Pointless Make initrd Code

For discussions about programming, programming questions/advice, and projects that don't really have anything to do with Puppy.
Post Reply
Message
Author
s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

Pointless Make initrd Code

#1 Post by s243a »

So I created some probably fairly pointless code but it does illustrate some concepts on how to make the initial ram disk. Basically this is a solution to a problem which doesn't really exist....or shouldn't exist unless you do something silly like I did (see here and here)

This is a solution for, "if you decompress you initrd.gz file to somewhere where you shouldn't (like /mnt/home) and you want to rebuild it". However, the easy way to solve this problem would probably just be to delete the directories and recopy initrd.gz from the CD.

However, this won't let us edit the files if we want to and I tried to add some variables in the script so it is easy to adapt the code to a more sensible place to decompress the ram disk.

The scripts hard codes all the initrd directories individually so that if for some reason you wanted to pull one or two directories from some place else it would be easy to do this. But then again would someone really want to do this? I don't know I said it was pointless code...well mostly pointless. This is practice code for me and I haven't even tested it yet. But you know it gives me something to look at when I'm away from my computer and want to think about it. I will updated this post with the modified code after I test it and fix the bugs.

The code is loosely based on the mkird script from an IBM tutorial.
https://www.ibm.com/developerworks/library/l-initrd/

but any remnants of the initial tutorial is mostly for reference.

Here is the code:

Code: Select all

#!/bin/bash

# s243a: Not sure I need this yet 
## Housekeeping...
#rm -f /tmp/ramdisk.img
#rm -f /tmp/ramdisk.img.gz

SOURCE=/mnt/home #If we decompressed initrd.gz without moving it
       #See: http://murga-linux.com/puppy/viewtopic.php?p=979708#979708
MOUNT_POINT=/mnt/new_initrd #s243a we can't mount at initrd because puppy already uses this.


# Ramdisk Constants
RDSIZE=1490 #This is the compressed size but what is the uncompressed size? 
BLKSIZE=1024
 
# Create an empty ramdisk image
dd if=/dev/zero of=/tmp/ramdisk.img bs=$BLKSIZE count=$RDSIZE
 
# Make it an ext2 mountable file system
/sbin/mke2fs -F -m 0 -b $BLKSIZE /tmp/ramdisk.img $RDSIZE
 
# Mount it so that we can populate
mount /tmp/ramdisk $MOUNT_POINT -t ext2 -o loop=/dev/loop0 #s243a removed the .img since puppy doesn't seem to use it.
 
#s243a Let's instead copy the files from an existing puppy initrd system
## Populate the filesystem (subdirectories)
#mkdir /mnt/initrd/bin
#mkdir /mnt/initrd/sys
#mkdir /mnt/initrd/dev
#mkdir /mnt/initrd/proc


# 
mkdir -p $MOUNT_POINT tmp

declare -a dirs= \
 ('bin' 'dev' 'etc' 'lib' 'locale' 'mnt' 'proc' 'sbin' 'sys' 'tmp' 'usr' 'var' 'pup_a' 'pup_f' 'pup_ro3' 'pup_ro4' \
  'pup_ro5' 'pup_ro6' 'pup_ro7' 'pup_ro8' 'pup_ro9' 'pup_y''pup_z' \
\  
 'pup_new' # The Layer being added in the /initrd/init script 
\
\           #See Lines #14 to #15 for def of puppy layers
\ #pmedia:         ram      atahd          usbhd         atahd       usbflash   Multisession
\ #savetype      pfx=ram   Full            Full         Frugal       Frugal     
\           #PUPMODE:5        6              7            12           13          77
  'pup_rw'   \ #   tmpfs     PDEV1          tmpfs    pup_save.3fs    tmpfs        tmpfs 
  'pup_ro1'  \ #   ----      -----         PDEV1     ------------  pup_save.3fs  folders  
  'pup_ro2') \ # pup_xxx.sfs pup_xxx.sfs pup_xxx.sfs pup_xxx.sfs   pup_xxx.sfs  pup_xxx.sfs

for d in "${dirs[@]}"; do
    mkdir -p "$MOUNT_POINT/$d"
    cp -a $SOURCE/$d/. $MOUNT_POINT
done 

#In the orginal tutorial there was a bunch stuff about copying busybox and making various symbolic links

cp -a $SOURCE/README.txt $MOUNT_POINT  
cp -a $SOURCE/DISTRO_SPECS $MOUNT_POINT 
cp -a $SOURCE/init $MOUNT_POINT

# Will use the existing init file instead  (See Previous Line)
## Create the init file
#cat >> /mnt/initrd/linuxrc << EOF
##!/bin/ash
#echo
#echo "Simple initrd is active"
#echo
#mount -t proc /proc /proc
#mount -t sysfs none /sys
#/bin/ash --login
#EOF

#Our init script should already have the correct permissions and is called init. 
#chmod +x /mnt/initrd/linuxrc
 
# Finish up...
umount /mnt/initrd
gzip -9 /tmp/initrd
cp /tmp/initrd.gz $SOURCE/ramdisk.gz #Stuf about .img extension removed because puppy doesn't use it.

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#2 Post by jamesbond »

The information from that IBM site is dated. There are two ways of creating "initrd", and the info you've got is the older one; applicable for puppies before 2.x I think.

There are two kinds of "initrd": The original, old one is the "initial ramdisk" - shortened to initrd, and the new one is "initial ram filesystem" - shortened to initramfs. Modern distros including puppies all use initramfs - although the name "initrd" unfortunately stuck and not many people are aware of the distinction.

To Linus credit, the original initrd functionality is retained and you can still use it to boot modern kernels; so the initrd created by your script will still work.
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#3 Post by s243a »

jamesbond wrote:The information from that IBM site is dated. There are two ways of creating "initrd", and the info you've got is the older one; applicable for puppies before 2.x I think.

There are two kinds of "initrd": The original, old one is the "initial ramdisk" - shortened to initrd, and the new one is "initial ram filesystem" - shortened to initramfs. Modern distros including puppies all use initramfs - although the name "initrd" unfortunately stuck and not many people are aware of the distinction.

To Linus credit, the original initrd functionality is retained and you can still use it to boot modern kernels; so the initrd created by your script will still work.
Thanks for the info :). I found something on this here:
Although the initrd method is all that is needed, it had a few drawbacks:

It is a full-fledged block device, requiring the overhead of an entire file system; it has a fixed size. Choosing an initrd that is too small and all needed scripts cannot fit. Make it too big and memory will be wasted.

Because it is a real, static device it consumes cache memory in the Linux kernel and is prone to the memory and file management methods in use (such as paging), this makes initrd greater in memory consumption.

To resolve these issues the initramfs was created.
https://wiki.gentoo.org/wiki/Initramfs/Guide

Looking at another link if I'm using initramfs instead then I don't need to use a loop device. I can just pipe all the files to cpio

Code: Select all

find . -print0 | cpio --null -ov --format=newc | gzip -9 > /boot/custom-initramfs.cpio.gz
https://wiki.gentoo.org/wiki/Custom_Initramfs

although some modification may be required.

on a related note I found this comment
It seems like when initrd is a cpio archive then "init" is executed
instead of "linuxrc".
https://www.redhat.com/archives/fedora- ... 01198.html

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#4 Post by s243a »

I made some modifications so that it calculates the size of the initial ramdisk. While jamesbond says that initramfs is more modern, I want to try it this way first because this way it is a complete file system (like a save file). Later I'll try it as initramfs -- maybe implemented as an option to the script or as a separate script.

Unfortunately this is still untested code. Other notes: I decided not to use /tmp like the IBM example and I added a variable in the script called OUTFILE for the name of the uncompressed image file. I also got rid of a lot of extraneous comments that explained the difference between this script and the IBM script.

Code: Select all

#!/bin/bash
# See Discussion: http://murga-linux.com/puppy/viewtopic.php?t=112511
# s243a: Not sure I need this yet 
## Housekeeping...
#rm -f /tmp/ramdisk.img
#rm -f /tmp/ramdisk.img.gz
BLKSIZE=1024
RDSIZE=1490 #Set this to how much free space you want in initrd. If BLKSIZE=1024 then this is in killobytes
OUTFILE='/mnt/sdd4/initrd'

SOURCE=/mnt/home #If we decompressed initrd.gz without moving it
       #See: http://murga-linux.com/puppy/viewtopic.php?p=979708#979708
MOUNT_POINT=/mnt/new_initrd #s243a we can't mount at initrd because puppy already uses this.

s243a_size(){
	return du --block-size=$BLKSIZE --total $1 | grep total | cut -f1	
}


# 
mkdir -p $MOUNT_POINT tmp

declare -a dirs= \
 ('bin' 'dev' 'etc' 'lib' 'locale' 'mnt' 'proc' 'sbin' 'sys' 'tmp' 'usr' 'var' 'pup_a' 'pup_f' 'pup_ro3' 'pup_ro4' \
  'pup_ro5' 'pup_ro6' 'pup_ro7' 'pup_ro8' 'pup_ro9' 'pup_y''pup_z' \
\  
 'pup_new' # The Layer being added in the /initrd/init script 
\
\           #See Lines #14 to #15 for def of puppy layers
\ #pmedia:         ram      atahd          usbhd         atahd       usbflash   Multisession
\ #savetype      pfx=ram   Full            Full         Frugal       Frugal     
\           #PUPMODE:5        6              7            12           13          77
  'pup_rw'   \ #   tmpfs     PDEV1          tmpfs    pup_save.3fs    tmpfs        tmpfs 
  'pup_ro1'  \ #   ----      -----         PDEV1     ------------  pup_save.3fs  folders  
  'pup_ro2') \ # pup_xxx.sfs pup_xxx.sfs pup_xxx.sfs pup_xxx.sfs   pup_xxx.sfs  pup_xxx.sfs

for d in "${dirs[@]}"; do
    RDSIZE=$RDSIZE+s243a_size d
    echo "RDSIZE=$RDSIZE"
done 

RDSIZE=$RDSIZE+(s243a_size $SOURCE/README.txt)
RDSIZE=$RDSIZE+(s243a_size $SOURCE/DISTRO_SPECS)
RDSIZE=$RDSIZE+(s243a_size $SOURCE/init)

# Create an empty ramdisk image
dd if=/dev/zero of=$OUTFILE bs=$BLKSIZE count=$RDSIZE
 
# Make it an ext2 mountable file system
/sbin/mke2fs -F -m 0 -b $BLKSIZE $OUTFILE $RDSIZE
 
# Mount it so that we can populate
mount $OUTFILE $MOUNT_POINT -t ext2 -o loop=/dev/loop0 #s243a removed the .img since puppy doesn't seem to use it.

for d in "${dirs[@]}"; do
    mkdir -p "$MOUNT_POINT/$d"
    cp -a $SOURCE/$d/. $MOUNT_POINT
done 

cp -a $SOURCE/README.txt $MOUNT_POINT  
cp -a $SOURCE/DISTRO_SPECS $MOUNT_POINT 
cp -a $SOURCE/init $MOUNT_POINT

# Finish up...
umount $OUTFILE
gzip -9 OUTFILE

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#5 Post by s243a »

Okay, Now I actually have tested code. I had to change the way the math was done. I don't know how to do it concisely like in other languages. I also found out you can't use a comment after a line continuation. Fortunately, you can put a comment inside an array definition.

The code is:

Code: Select all

#!/bin/bash
# See Discussion: http://murga-linux.com/puppy/viewtopic.php?t=112511
# s243a: Not sure I need this yet 
## Housekeeping...
#rm -f /tmp/ramdisk.img
#rm -f /tmp/ramdisk.img.gz
BLKSIZE=1024
RDSIZE=1490 #Set this to how much free space you want in initrd. If BLKSIZE=1024 then this is in killobytes
OUTFILE='/mnt/sdd4/initrd'
loop_dev='loop3'
SOURCE='/mnt/sdd4' #If we decompressed initrd.gz without moving it
       #See: http://murga-linux.com/puppy/viewtopic.php?p=979708#979708
MOUNT_POINT=/mnt/new_initrd #s243a we can't mount at initrd because puppy already uses this.


# 
mkdir -p $MOUNT_POINT tmp

declare -a dirs=\
('bin' 'dev' 'etc' 'lib' 'locale' 'mnt' 'proc' 'sbin' 'sys' 'tmp' 'usr' 'var' 'pup_a' 'pup_f' 'pup_ro3' 'pup_ro4' 
  'pup_ro5' 'pup_ro6' 'pup_ro7' 'pup_ro8' 'pup_ro9' 'pup_y' 'pup_z' 

 'pup_new' # The Layer being added in the /initrd/init script 
           #See Lines #14 to #15 for def of puppy layers
  #pmedia:         ram      atahd          usbhd         atahd       usbflash   Multisession
  #savetype      pfx=ram   Full            Full         Frugal       Frugal     
           #PUPMODE:5        6              7            12           13          77
  'pup_rw'    #   tmpfs     PDEV1          tmpfs    pup_save.3fs    tmpfs        tmpfs 
  'pup_ro1'   #   ----      -----         PDEV1     ------------  pup_save.3fs  folders  
  'pup_ro2')  # pup_xxx.sfs pup_xxx.sfs pup_xxx.sfs pup_xxx.sfs   pup_xxx.sfs  pup_xxx.sfs

for d in "${dirs[@]}"; do
    #delta=$(s243a_size "$d")
    delta=$(du --block-size=$BLKSIZE --total $d | grep total | cut -f1	)
    echo "delta=$delta" 
    (( RDSIZE = RDSIZE + delta ))
    echo "RDSIZE=$RDSIZE"
done 

delta=$(du --block-size=$BLKSIZE --total $SOURCE/README.txt | grep total | cut -f1 )
(( RDSIZE=$RDSIZE+delta ))
delta=$(du --block-size=$BLKSIZE --total $SOURCE/DISTRO_SPECS | grep total | cut -f1 )
(( RDSIZE=$RDSIZE+delta ))
delta=$(du --block-size=$BLKSIZE --total $SOURCE/init | grep total | cut -f1 )
(( RDSIZE=$RDSIZE+delta ))

# Create an empty ramdisk image
dd if=/dev/zero of=$OUTFILE bs=$BLKSIZE count=$RDSIZE
 
# Make it an ext2 mountable file system
/sbin/mke2fs -F -m 0 -b $BLKSIZE $OUTFILE $RDSIZE
 
# Mount it so that we can populate
mount $OUTFILE $MOUNT_POINT -t ext2 -o loop=/dev/$loop_dev #s243a removed the .img since puppy doesn't seem to use it.

for d in "${dirs[@]}"; do
    mkdir -p "$MOUNT_POINT/$d"
    cp -a $SOURCE/$d/. $MOUNT_POINT/$d
done 

cp -a $SOURCE/README.txt $MOUNT_POINT  
cp -a $SOURCE/DISTRO_SPECS $MOUNT_POINT 
cp -a $SOURCE/init $MOUNT_POINT

# Finish up...
umount $MOUNT_POINT
gzip -9 OUTFILE
If one wants to see the results then comment out the second last line. Also for some reason I'm not getting the .gz extension in my initrd file. I got to figure that out.

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#6 Post by s243a »

I made a version that uses cpio instead of an ext2 image. In either case the kernal wouldn't sync but I'm not sure the reason yet.

Anyway, since I couldn't get these approaches to work I did a search for "initramfs" on my system and ended up finding Barry's script

Code: Select all

/usr/sbin/edit-initramfs
Barry's script worked for me (see thread) but I'm going to post my failed cpio script bellow because I want to return to it after studying Barry's script and see if I can figure out why mine didn't work

Code: Select all

#!/bin/bash
# See Discussion: http://murga-linux.com/puppy/viewtopic.php?t=112511
OUTFILE='/mnt/sdd4/initrd'
SOURCE='.' #If we decompressed initrd.gz without moving it
       #See: http://murga-linux.com/puppy/viewtopic.php?p=979708#979708

declare -a dirs=\
('bin' 'dev' 'etc' 'lib' 'locale' 'mnt' 'proc' 'sbin' 'sys' 'tmp' 'usr' 'var' 'pup_a' 'pup_f' 'pup_ro3' 'pup_ro4' 
  'pup_ro5' 'pup_ro6' 'pup_ro7' 'pup_ro8' 'pup_ro9' 'pup_y' 'pup_z' 

 'pup_new' # The Layer being added in the /initrd/init script 
           #See Lines #14 to #15 for def of puppy layers
  #pmedia:         ram      atahd          usbhd         atahd       usbflash   Multisession
  #savetype      pfx=ram   Full            Full         Frugal       Frugal     
           #PUPMODE:5        6              7            12           13          77
  'pup_rw'    #   tmpfs     PDEV1          tmpfs    pup_save.3fs    tmpfs        tmpfs 
  'pup_ro1'   #   ----      -----         PDEV1     ------------  pup_save.3fs  folders  
  'pup_ro2')  # pup_xxx.sfs pup_xxx.sfs pup_xxx.sfs pup_xxx.sfs   pup_xxx.sfs  pup_xxx.sfs

  (echo "$SOURCE/init"
  echo "$SOURCE/README.txt"
  echo "$SOURCE/DISTRO_SPECS") |  tr '\n' '\0' | \
  ( for d in "${dirs[@]}"; do
    find "$SOURCE/$d" -print0
  done ) | cpio --null -ov --format=newc > $OUTFILE

# Finish up...
gzip -9 "$OUTFILE"

Post Reply