Puppy Linux Discussion Forum

[heywoodj]

[Ted Dog]

Starting at line 739 of system-init:

[Sage]

[jamesbond]

[Ted Dog]

Thanks for looking into the zombie white-out possibility, your code once located is so easy to follow, so I'm encouraged to try my hand with labelled save session control (allowing user to select which session to skip or group )
However, the remaster process goes really slow due to my CPU's compression/decompression weakness, would there be a way to take the whole files in tmp during remaster, before the sfs compression and gzip them into initrd without sfs file? for a super huge init (actually only 20-25% larger initrd is still gzipped) so that the system would be uncompressed in RAM as an remaster option if machines have massive amounts of RAM, you check the disk usage of the full filesystem and report that before compression, I've still have 7G's free RAM on a fully decompressed FatDog64. Lets make this fat puppy sing....

[Ted Dog]

Furiously fast program execution, when running from RAM in uncompressed state.
I made this option for myself in order to speed up testing during my early days with puppylinux, so Its do able. Rule of thumb the system should have 9x the RAM size for the size of the gzipped initrd to run nicely ie make large changes, rezip reburn etc. For average user, 4x RAM is needed, no plans to remaster in RAM etc.
For example FatDog64 would be around 285M, so a machine with 2.8G's or more of RAM would be supported nicely and be usable with some headroom restrictions down to around 1.2Gs of RAM.

WARNING: the speed improvement is so drastic you can never be happy going back... Like driving a very fast car, true sports car, you will never be fully satisfied with anything less.....It's additive.

[jamesbond]

That is a fun idea, I can't resist
Here's one way you can do it.
1. Open terminal (call this as terminal #1), launch sandbox.sh, choose the layers you want to include in the "remaster". You will need to include all the bottom 3 layers (pup_init, pup_ro and kernel-modules).
2. Open another terminal (call this as terminal #2)
3. Run the following script inside terminal #2 - important, in terminal #2, NOT in the sandbox itself.
4. You will get initrd.gz in /tmp
5. I assume you know what to do with that initrd.gz
6. Close terminal #1 and #2.

The resulting initrd booted on my 2GB virtualbox instance, so it's not too bad
Expect some scripts to fail because when run this way, you're actually running without basesfs (all the files are in initrd).

[jamesbond]

Here is another idea how you can do it.
Instead of making squashfs of the root filesystem, you could create an ext2 image file large enough to hold the stuff (about 700MB at the moment), and put the content of squashfs there, uncompressed. Name the image file as "fd64-620.sfs" and put it in initrd, and then "cpio -o -H newc | gzip" as usual to produce the initrd stuff.

This is "more compatible" than the above because it still runs with basesfs, but the basesfs is uncompressed. The entire initrd is still compressed but it will be decompressed only once during boot-up. In my brief test this seems to have a better boot speed.

Note: just because the extension is "SFS", it doesn't mean it has to be squashfs file

Instruction:
1. Open terminal (call this as terminal #1), launch sandbox.sh, choose the layers you want to include in the "remaster". Only include pup_ro layers onwards, *DO NOT* include the bottom 2 layers (pup_init and kernel-modules).
2. Open another terminal (call this as terminal #2)
3. Type the commands in the following script one-by-one inside terminal #2 - important, in terminal #2, NOT in the sandbox itself.
4. Replace /path/to/original/initrd with - well, the path your original initrd. Its contents will be changed so make sure you have a handy backup copy, and it must be on a writable medium.
5. Replace /tmp/initrd-XXXXX with the random path generated by filemnt.
6. At the end of the script, the your /path/to/original/initrd would have been updated, which then you can use for making iso etc.
7. Close terminal #1 and #2.

[gcmartin]

This is a great idea for a "local" adaptation of FD. It offers the creation of an ISO by the local user which will run a system minus the overhead of compression, once booted.

Excellent insight from @TedDog and implementation examples from @JamesBond.

I wonder if this could be a future addition to the general system or an option to the Remaster tool.

I do understand the RAM impact, but, the performance offers a positive impact and user experience. Would this qualify as a "Real-Time" category?

Thanks.

[Ted Dog]

Should also be possible without having to rebuild it uncompressed, if the entire contents of regular SFS would be copied into RAM (decompressing) at the time prior to switch-root, and then the SFS could be deleted, or if in my case just remain behind to aid in compatibility I've got the RAM room to spare. RAM would need to be 10X the size of compressed SFS to function without issue.
When I did my methods years ago I used 10X size is a warning layer since I found scripting x10 math easier than x9 I slapped a character for the letter zero to the end of the string and use that to compare.
I had a single script closer to the last one above that then created an on the fly iso and burned it to the DVD+RW, then called the reboot command. It was nice, hit the icon on the desktop, leave to eat or due something else for a few minutes. If my desktop was in view then that test was successful.

A single bootcode could be added to do this.

[Ted Dog]

Actually you should make running slow as a boot flag not FAST.

I always have to add a waitdev of 2 to 4 seconds for my machine to function correctly.

Those 2 to 4 seconds would be all we need to decompress the sfs if it already loaded in RAM.

Check for 10x RAM and decompress unless they told us not too.

Its the repetitively decompressing the same files when accessed individually that slows the system.

Also this reminds me of something I read online ( can't recall source ) that redhat uses sfs as a package container for a single file which is a filesystem like ext2 that is used by the system like a block device, so that different layer of reading code is used by kernel drivers and a performance is increased since the files most likely to be used next are within the block driver RAM buffer.

I did not understand how that actually worked but if you are already so close to reproduce that method why not see if this holds true.

[jamesbond]

Replace process_base2ram with this one.
Beware that the initial expansion process is a bit slow.
To use, boot with base2ram=expand

[heywoodj]

[Ted Dog]

Great, a turbo boost option from the start, without a remaster for the next version of FatDog. I think this will become popular since most 64bit computers come with enough RAM.

I, of course, will try this first, then stair-step the fast remaster option to solve my original issue. Thanks for all the scripts and how-to knowledge...

[Ted Dog]

Boot option base2ram=expand works nice, it does not work in a huge init without editing the code that resets base2ram=no. since its already loaded

But so far, loading apps with lots of files SeaMonkey, Gimp etc. its very noticeable in a good way. System over all seems a bit less clogged.