How to make Image.gz with usr_cram.fs inside for LAN Boot?

[sunburnt]

I like PizzaPup for IceWM & FireFox, so I chose to work with it.
Pizza's LAN booting to; no usr_cram.fs, solution; put it in the Image.gz file.
I know I'll need at least 128MB on client PC to run it... no problemo.

Read the Wiki; RemasterImageGz, but I don't grasp what's going on.
I don't get the "dd & mke2fs" lines & the copy lines I'm not sure.
I've done a little scripting, but not much, so bare with me.

1) Could someone please explain in detail all about the "dd & mke2fs" lines?
I'm sure I'd have to change some numbers in those lines to be able to insert the cram file.

2) Is there a script JUST for this (I looked), if not then I'll make a script out of it.

6 things below I need clarified (HELP).
===========================
copied files from image.gz -> /my-documents/imagep/ as follows
---------------------- # Make script from below code.
gunzip image.gz
mount -o loop image /mnt/data
# Would the next line be: "cp /mnt/data/* /my-doc"?
copied the data folder to /my-doc..
# Would the cram copy line be: "cp /usr_cram.fs /my-doc"?
# Don't we need to umount "/mnt/data" here?

then i added new files and made a new image.gz as follows
---------------------- # Please explain the next 2 lines in excruciating detail.
dd if=/dev/zero of=image bs=1k count=13312 (13 Mb)
mke2fs -F -m 0 -b 1024 image
mkdir /mnt/tmpimage
mount -o loop image /mnt/tmpimage
# Don't know what the next line would be, what's "../imagep"?
copy the files from /my-doc../imagep to /mnt/tmpimage
umount tmpimage
gzip image
=======
Nfs mounting usr_cram.fs would be great (smaller ramdisk so less memory's needed), that would require changing Barry's startup scripts, too much for me with my limited know how, but give me some time.
I read BladeHunters post at Freesco, but he stopped posting so I don't know what's happening with it.

[PeterSieg]

Hi, I am not en expert either, but:

Code: Select all

dd if=/dev/zero of=image bs=1k count=13312 (13 Mb)

This cmd creates a file of size 13312 kb with just zeros in it.
zeros are used, so that any space not used later on can be compressed well.
You will need to make this file just big enough to hold image(.gz)+usr_cram.fs in UMCOMPRESSED size!!
So we are talking app. of 12-13MB for image.gz and app. 100-120MB for
usr_cram.fs uncompressed. So without exact knowing I would try with 150MB

Code: Select all

mke2fs -F -m 0 -b 1024 image 

This creates an ext2 filesystem inside the file created before.
After that, you can mount the file as like a harddisk partition witk the loop device. cmd is like: mount -o loop image /mnt/data

In the copying, you should make sure that the copy copies all files, symlinks etc. without changes. Do a cp --help | more to see options...
I think it is something like cp -a or cp -dpR ...

PS

[PeterSieg]

Hi.

here I start trying a step by step guide...

1. Make sure you have enough space in pup001 file! You will need app.200MB
1.a Also mount your puppy live-cd to ex. /mnt/hdc
1.b go to $HOME (/root) with cd $HOME inside console terminal

2. create mount points
mkdir /mnt/image_old
mkdir /mnt/image_new

3. create new image file of 150MB (150*1024):
dd if=/dev/zero of=image_new bs=1k count=153600
mke2fs -F -m 0 -b 1024 image_new

4. mount new image file
mount -o loop image_new /mnt/image_new

5. mount old image file
# copy image.gz off live-cd
cp /mnt/hdc/image.gz /root
gunzip image.gz
mount -o loop image /mnt/image_old

6. copy old image to new image
cp -a /mnt/image_old/* /mnt/image_new
sync

7. usr_cram.fs should be mounted already at /usr ??
# don't know it needed? - if usr is already there not needed ??
mkdir /mnt/image_new/usr
cp -a /usr/* /mnt/image_new/usr
sync

8. make new image
umount /mnt/image_new
gzip -9 image_new
mv image_new.gz image.gz

9. clean ups
umount /mnt/image_old
rm image
rm /mnt/image_old
rm /mnt/image_new

10. copy new image.gz to where you need it (iso etc.)...


PS

[Flash]

Thank you, Peter. You left out of step 3 the excellent explanations of your previous post:

...This cmd creates a file of size 13312 kb with just zeroes in it.
zeros are used, so that any space not used later can be compressed well...

...This creates an ext2 filesystem inside the file created before.
After that, you can mount the file as like a harddisk partition with the loop device. [I would add: You can only mount filesystems.] cmd is like: mount -o loop image /mnt/data...

[sunburnt]

Thanks for the tremendous response guys!
This cleared up a misconception, I wondered if the usr_cram.fs file had to be inside the Image.gz file, or the Image file inside of it, ANSWER: wrong on both accounts, the expanded file structure of BOTH files needs to be merged.
PeterSieg didn't say you had to be running the Puppy release that you're modifying, but hinted it in the usr_cram.fs section.

Q#1) I modded his script to work from a iso file (I don't have any CDs of Puppy), so a line to expand the cram file is needed.
I looked, but didn't find command line examples, I think I'll look at the rc.sysinit file for a hint of how to do it.

ADDENDUM: I looked at rc.sysinit & it looks like to me that the usr_cram.fs file is intact inside of the Image.gz file, the commands for dealing with it are the same if the cram file's in Image.gz or if it's outside...... Barry... HELP!

QUOTE from "How Puppy Works ": Puppy will first look to see if usr_cram.fs is at "/" in the ramdisk (which meant that it was inside image.gz originally).
SO I just copy usr_cram.fs into the Image.gz file, or more likely into the Image file inside it (& maybe I wasn't wrong?)?

Q#2) Also PizzaPup has a EXTRAS dir. on the iso file, how do I include it in the new Image.gz file?

Q#3) One more Q, is there a way to get JUST the number of the total size of the complete working file system, to plug into the dd command in my script, perhaps with the commands: df & grep/sed (I've looked at them & WooWee... headache!).

This is what I did (screwed up?) rewriting what PeterSieg so graciously provided.
I added some lines & rearranged others for clarity, please tell me if & what's wrong with it & how to improve it.

Code: Select all

#!/bin/sh
# Script to join file systems of: Image.gz & usr_cram.fs into BIG new Image.gz.

# Command line arguments are: $1 = iso file name, $2 = new image size.
# Would like eliminate the new image size argument $2, & automatically obtain it.

# Make sure you have enough space in pup001 file! You will need app. 200MB.
# Running Puppy in a console terminal, go to $HOME (/root) by typing: cd $HOME

##### 1. Mount the Puppy iso file.
mount -o loop -t iso9660 $1 /mnt/data

# Copy Image.gz to /root, gunzip it & unmount CD or iso file.
cp /mnt/data/image.gz /root
gunzip image.gz
umount /mnt/data

##### 2. Create mount points.
mkdir /mnt/image_old
mkdir /mnt/image_new

##### 3. Create new image file & make ext2 file sys. on it. ~ 150MB (150*1024) = 153600
dd if=/dev/zero of=image_new bs=1k count=$2
mke2fs -F -m 0 -b 1024 image_new

##### 4. mount new image file & old image file.
mount -o loop image_new /mnt/image_new
mount -o loop image /mnt/image_old

##### 5. copy old image to new image
cp -a /mnt/image_old/* /mnt/image_new
sync

##### 6. usr_cram.fs should be mounted already at /usr ??
# don't know it needed? - if usr is already there not needed ??
mkdir /mnt/image_new/usr
cp -a /usr/* /mnt/image_new/usr
sync

##### 7. unmount new & old images
umount /mnt/image_old
umount /mnt/image_new

##### 8. make new image
gzip -9 image_new
mv image_new.gz image.gz

##### 9. clean ups
rm image
rm /mnt/image_old
rm /mnt/image_new

[Pizzasgood]

I agree that the usr_cram.fs seems like it should probably be intact, as that would make more sense. I've never done it though, so I don't knowl.

For the extras, I guess it depends. You could add it into the image, but that would defeat the purpose of having them separate (to not take up space in the ram disk, and allow choosing to install them without having to download).

The stuff in the extra folder is extra, as in removing it won't affect booting. It's just for the convenience of not having to download wireless drivers (to avoid chicken and egg scenarios) and to have nicer themes available without compromising image.gz size for them. So you could just leave it out and you'll be fine.

[Ted Dog]

Code: Select all

#!/bin/bash
rm -f /mnt/hdd1/mew.iso
gunzip -c /mnt/hdd1/livecd/image.gz | dd of=/tmp/image
rm -rf /mnt/image
mkdir /mnt/image
resize2fs -f /tmp/image 80M
e2fsck /tmp/image
losetup-FULL /dev/loop2 /tmp/image
mount -t ext2 /dev/loop2 /mnt/image
sync
rox "/mnt/image"
echo -n -e "Press Enter to reconstruct image.gz"
read nn4
rox -D "/mnt/image"
sync
dd if=/tmp/image | gzip -9 > /mnt/hdd1/livecd/image.gz
umount /mnt/image
rm -f /tmp/image
mkisofs -J -R -pad -b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -o /mnt/hdd1/mew.iso /mnt/hdd1/livecd/ 

qemu -m 120 -cdrom /mnt/hdd1/mew.iso -boot d -user-net

[sunburnt]

Thanks Pizzasgood, now I know what to do with it.

Ted Dog; I'm not sure what you've got, please explain it in some detail.

It looks like the Puppy source is a CD, I don't have any Puppy CDs, I just wanna use iso files for the source & leave the vmlinuz & Image.gz files outside the iso when I'm done, to be pxe booted from /tftpboot.

Live CDs are ok for trying out a distro., & are great for system rescue, but if I wanted portability, I'd use a USB flash drive.
My main interests for Puppy setups are HD, USB, & LAN boot, maybe even WIFI booting, how about a diskless laptop?

And if your good at script, maybe you could answer Q#3 above, getting the size of the new image for the dd comand.

[Pizzasgood]

From what I can tell, it takes an expanded iso (in other words the image.gz, usr_cram.fs, etc) from /mnt/hdd1/livecd/, decompresses the image to /tmp/image, mounts it at /mnt/image, opens a rox window, waits for you to hit enter (presumably so you can edit the image), then recompresses it, puts it back, and makes an iso at /mnt/hdd1/ called mew.iso. That single line at the end starts the iso in qemu to test it (but you need to install qemu separately).

If my understanding of the script and Puppy are correct, I'd be willing to bet that when it gets to the part where it pops up the rox window, you copy usr_cram.fs straight in, then close it up. I think this because if you have enough ram, when Puppy boots it copy's usr_cram.fs from wherever you have it to /, so if you stick it there, it won't need to copy it.

[Ted Dog]

`rox -D /mnt/cdrom ;umount /mnt/cdrom ; mount -o loop -t iso9660 -r "$1" /mnt/cdrom ; rox /mnt/cdrom`
is my run action for .iso
qemu is an easy install download
http://fabrice.bellard.free.fr/qemu/download.html
choose the second one qemu-0.7.2-i386.tar.gz Binary distribution for linux-i386 (untar it in '/').
that is it.
Pizzasgood you are correct in how it works, the booting freezes for unknown reason I hope MU or better John Murga could shed some light on what file changes needs to be done to correct the booting of usr_cram.fs from within image.gz[/code]

[sunburnt]

The How Puppy works says Puppy looks in / for the cram file first.
This means the cram file must be in the Image file inside of Image.gz.
It worries me when I don't understand how something works, is my code any good?

So all I need to do is add the extract iso line from my script, & remove the qemu line so no new iso file is made?
I don't grasp much of it (how 'bout some comments Ted Dog?), is any of the rox lines needed if noCD is burned?
How about this?
===========

Code: Select all

#!/bin/bash
rm -f /mnt/hdd1/mew.iso

### Mount the Puppy iso file.
mount -o loop -t iso9660 $1 /mnt/hdd1/livecd

gunzip -c /mnt/hdd1/livecd/image.gz | dd of=/tmp/image

### Added cram file copy
cp -a /mnt/hdd1/livecd/usr_cram.fs /tmp/image

rm -rf /mnt/image
mkdir /mnt/image
resize2fs -f /tmp/image 80M
e2fsck /tmp/image
losetup-FULL /dev/loop2 /tmp/image
mount -t ext2 /dev/loop2 /mnt/image
sync
#rox "/mnt/image"
#echo -n -e "Press Enter to reconstruct image.gz"
#read nn4
#rox -D "/mnt/image"
#sync
dd if=/tmp/image | gzip -9 > /mnt/hdd1/livecd/image.gz
umount /mnt/image
rm -f /tmp/image

### The next line I keep for making a new iso file, but I want to keep the separate files also. 
mkisofs -J -R -pad -b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -o /mnt/hdd1/mew.iso /mnt/hdd1/livecd/

[Ted Dog]

Code: Select all

#!/bin/bash
ISO="puppy.iso"
rm -f mew.iso
mkdir /livecd
### Mount the Puppy iso file.
mount -o loop -t iso9660 $ISO /livecd
gunzip -c /livecd/image.gz | dd of=/tmp/image
umount /livecd
rm -rf /mnt/image
mkdir /mnt/image
resize2fs -f /tmp/image 80M
e2fsck /tmp/image
losetup-FULL /dev/loop2 /tmp/image
mount -t ext2 /dev/loop2 /mnt/image
sync
#may stop here available loops are very limited
mount -o loop -t iso9660 $ISO /livecd
cp -a /livecd/usr_cram.fs /tmp/image
mkdir /tmp/livecd
sync
dd if=/tmp/image | gzip -9 > /tmp/livecd/image.gz
umount /mnt/image
rm -f /tmp/image
cp -a /livecd/* /tmp/livecd/
rm /tmp/livecd/usr_cram.fs
### The next line I keep for making a new iso file, but I want to keep the separate files also.
mkisofs -J -R -pad -b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -o mew.iso /tmp/livecd/

just changed the ordering and you had the jist of it.

[sunburnt]

Ted Dog; much thanks for the help, but now I'm really confused, some Qs if I might.
1) I had $1 for command line input of iso file name, this is how it's done, right?
2) The 80M size for the new image, is that good for most Puppy work, or is a closer size a good idea?
3) I'm not sure why you unmounted the iso file & then mounted it again.
4) Why do you remove the usr_cram.fs file just before making the iso file?
I hate to belabor a point, but I'm intending on using this to make many DOS & Lan boot setup images.
Thanks again... sunburnt

[Ted Dog]

1) Yes, I was going to test it via hard coded path
2) Yes, 48M sf+ 12M full image from gz + some overhead, needed. It will compress back to near 55M
It was set that was for larger custom puppies such as pizzasgoods' pizzapup, and Nathans' grafpup
3) Tried to use only one loop back at a time. Later found it cumbersome, using more than one loop back = problems
4) Save space on iso. With a copy already in image.gz, a second one would about double size of iso. It would be better if only files needed would be copied it would save memory used at one time.

[Pizzasgood]

1) I had $1 for command line input of iso file name, this is how it's done, right?

Yep. $0 is the program name (I think), $1 is the first parameter (I'm positive), and $2, $3, etc. are the second, third, and more parameters.

As for the size, decompress the image (so that it is image, not image.gz or a bunch of files. Just an image), and get the size. Then get the size for usr_cram.fs. Then just add them togeather, and that should be what you need.

One thing we all forgot is that the file /etc/ramdiskfssize needs to store the size of the ramdisk (image after usr_cram.fs is added, possibly with 448K of buffer space) Also, if you use an iso, the isolinux.cfg will need to be updated to reflect the new size. That won't apply for this, though.

[sunburnt]

Fantastic, everything you guys just said makes sense, I'm in the home stretch here folks.
I want to add a line right after the cram file copy to calculate the ramdiskfssize value.
But for the moment to make it work I'll add $2 to manually in put the value.
Is the end of this line: "resize2fs -f /tmp/image $2M" going to work?

Does the kernel arg.: ramdisk_size=50603 override the file: /etc/ramdiskfssize?
If so, as long as it's a kernel arg., the ramdiskfssize file needn't be changed.

I assume this is the same value to put in the line: resize2fs -f /tmp/image $rdfss.

Also there's the files: /etc/ramdisksize & /etc/sizeusecram, do they need to be set also?
ramdisksize has: 12288, & sizeusrcram has: 48078848

[Ted guest Dog]

I have got it, to this stage, and I'm stuck. There are multiple files which must change in image.gz (thats why my version shows files to edit with ROX) but after a few hours of hacking I couldn't get past the impasses. I need it for grub-er puppy version two. So multiple custom puppies could exist on one CD or DVD. If you do figure it out reply please, thanks -TD

[sunburnt]

I can only think of 2 was to have multi Puppies on a CD.
1) The same way I got it working on the HD, copy usr_cram.fs to hda1 or C:\.
2) The way I'm working on now, mount the versions iso file or dir. on the CD where the kernel, Image.gz, & usr_cram.fs files are at as a drive, so Puppy is fooled into thinking the usr_cram.fs file is in the root of a drive with the rest of the files.
The multi iso files on the CD & mounting one at a time might work the best & be the easist to do.

That's the way Puppy works, it looks in the ramdisk root, then the pup001 file root, then /hda1 root, then /CDROM root.
I don't know if this helps, but it's the only way I can see to do it until I can figure out the changes for the rc.sysinit file & probably some other files too.

Barry had no way of knowing Puppy would go this far, but as I see it, Puppy needs to be able to specify the path & the name for the cram file so problems like yours & mine would just go away!
This would save alot of headaches that folks are having trying to extend Puppy beyond Barry's original vision.

[Pizzasgood]

Also there's the files: /etc/ramdisksize & /etc/sizeusecram, do they need to be set also?
ramdisksize has: 12288, & sizeusrcram has: 48078848

/etc/sizeusrcram only needs to be changed if you change the size of usr_cram.fs. I don't know what /etc/ramdisksize does.

Does the kernel arg.: ramdisk_size=50603 override the file: /etc/ramdiskfssize?
If so, as long as it's a kernel arg., the ramdiskfssize file needn't be changed.

I don't know. /etc/ramdiskfssize might just be used to store the size so you don't have to write it down. I did find this in Barry's old remaster script:

#...note, /sbin/init will change this at bootup.

Better safe than sorry, though.

This will get the size of image. This is more or less copied and pasted from my HackyRemaster script:

Code: Select all

IMAGEUSED=`du -ks /mnt/image | cut -f 1`
IMAGEFREE=5120  #Buffer space
IMAGESIZE=`expr $IMAGEUSED + $IMAGEFREE`
USRCRAMSIZE="`cat /mnt/image/etc/sizeusrcram`"
TOTALSIZE=`expr $IMAGESIZE + $USRCRAMSIZE`

$TOTALSIZE will be the size with 5120 of buffer space.

If you get problems, try bumping up the buffer. More can't hurt, but less can. I don't know exactly what it needs, but the script I based mine on said this

#448K free space. we rely on /root0 getting deleted.
#...also, squashfs f.s. management uses most of that 448K.

If you haven't figured it out by tomorrow afternoon, I'll try it too with Personal-Pan Puppy. I'll be needing to test it by then anyways to make sure I haven't broken anything yet

[Ted guest Dog]

Code: Select all

#!/bin/bash
ISO=$1
IMAGESIZE=$2
WRKDIR=$3
#====== testing ===
ISO="/mnt/home/pizzapup105.iso"
IMAGESIZE="96000"
WRKDIR="/mnt/home"
#=================
rm -f $WRKDIR/new.iso
mkdir $WRKDIR/livecd
mount -o loop -t iso9660 $ISO $WRKDIR/livecd 
gunzip -c $WRKDIR/livecd/image.gz | dd of=/tmp/image
rm -rf /mnt/image
mkdir /mnt/image
resize2fs -f /tmp/image $IMAGESIZE
e2fsck /tmp/image
losetup-FULL /dev/loop4 /tmp/image
mount -t ext2 /dev/loop4 /mnt/image
sync
rox "/mnt/image"
echo -n -e "Press Enter to reconstruct image.gz"
read nn4
rox -D "/mnt/image" 
 echo -n "$IMAGESIZE" > /mnt/image/root0/.etc/ramdiskfssize
  echo -n "$IMAGESIZE" > /mnt/image/root0/.etc/ramdisksize
  echo "There is no need for special /mnt/image/sbin/init script..."
  rm -f /mnt/image/sbin/init
  ln -s /bin/busybox /mnt/image/sbin/init
  echo "Editing /mnt/image/sbin/check_space script to monitor /dev/ram0..."
  cat /mnt/image/sbin/check_space | sed -e 's/#DEVICES="\/dev\/ram0"/DEVICES="\/dev\/ram0"/g' > /tmp/checkspacetmp
  sync
  cat /tmp/checkspacetmp | sed -e 's/DEVICES="tmpfs"/#DEVICES="tmpfs"/g' > /mnt/image/sbin/check_space
  sync
mkdir $WRKDIR/tmp
mkdir $WRKDIR/tmp/livecd
cp -a $WRKDIR/livecd/ $WRKDIR/tmp/
rm $WRKDIR/tmp/livecd/usr_cram.fs
dd if=/tmp/image | gzip -9 > $WRKDIR/tmp/livecd/image.gz
umount /mnt/image
rm -f /tmp/image

umount $WRKDIR/livecd

mkisofs -J -R -pad -b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -o $WRKDIR/new.iso $WRKDIR/tmp/livecd/

qemu -m 120 -cdrom $WRKDIR/new.iso -boot d -user-net

found out what was causing the stopage.