Performance differences between frugal vs full install?

[nic007]

Any difference if run fully in RAM? What about full install on linux partition vs frugal install (anywhere) with swapfile and limited resources (in instances not fully running in RAM).

[mikeb]

full will start quicker and use a bit less ram.

frugal will have apps opening faster if the sfs loads to ram.

the rest is down to the usual...ease of fixing/using sfs/quirks and so on.

mike

[nic007]

full will start quicker and use a bit less ram.

frugal will have apps opening faster if the sfs loads to ram.

the rest is down to the usual...ease of fixing/using sfs/quirks and so on.

mike

Full does not load totally to RAM (if enough available)? Wary Puppy is slightly sluggish on my old 384Mb RAM machine (windows XP is faster). Also, it hardly ever uses my swapfile which is on fat32 partition (as is the SFS and savefile). So apparently a full install will be of no significance in this situation? Better to stick with windows then. To think that I ran windows xp on only 128Mb RAM at one stage and it was still workable. What an operating system, so much more practical and efficient than the new, bloated rubbish with popups whenever you want to do something.

[mikeb]

yes full does not load the system files to ram at all.

With that spec I doubt if you would notice any difference plus puppy has some joke funnies with full install...

XP was the best on my old 1996 kayak partly because it had the correct video driver and linux did not support it (cirrus...funny really thats the QEMU vid card)

64MB ram was ok on windows too..... all this 256/512 ram for linux seems a bit vista like....


fun and games

[nic007]

yes full does not load the system files to ram at all.

With that spec I doubt if you would notice any difference plus puppy has some joke funnies with full install...

XP was the best on my old 1996 kayak partly because it had the correct video driver and linux did not support it (cirrus...funny really thats the QEMU vid card)

64MB ram was ok on windows too..... all this 256/512 ram for linux seems a bit vista like....


fun and games

Well, puppy 5 starts to get sluggish when you only have 256MB RAM in fact with 384MB RAM it's still a bit sluggish. So i think one can safely say you need at least 512MB RAM and more if you want to run puppy 5 and later without a bit of lag time. I don't know if this was the intention when puppy started off to consume so much system resources? It now seems to get closer and closer to the bigger operating systems in terms of "resource hungriness" which leaves the question whether it's going to stay a viable alternative.

[Ted Dog]

never underestimate the speed loss due to continual decompression of sfs text files, Unlike other distros puppylinux is mostly scripts masquerading as programs squashfs is VERY SLOW with text due to lack of block cache and the way it forces the text type files together as a mega file for compression gains. A tiny script is grouped with other tiny scripts and compressed as a block. Each time called it decompresses the entire mega block of text just to return a tiny script. Add the push to segregated scripts into function call blocks in a different script that is called by the first script.. I once readup on HOW TO CORRECT THIS but lost the link and can't talk the developers into trying this method. The correction is to loop mount a block based filesystem like ext3 as a single file into a sfs. the squashfs contains one file, a ext3 filesystem. This method provides a read ahead cache of already decompressed files.
Along with this method is using a fast decmpression type. Currently its xz, but gz style is faster and doesn't block multicore AMD CPUs into single core like xz does.
Ok this is getting a bit long in the tooth. There is another method like frugal install but not as frugal on harddrive space. Use squashfs as just packed only not compressed or better move all files into a savefile ext4 etc.
But for some reason developers do not directly support this but should, sfs is only 1/3 the size of orginal files. so for a savefile in the 1G range would comfortable fit a fat frugal install without the continual decompression.

[Ted Dog]

@nic.. Current or methods used by BK before his retirement can't live much longer and I believe the rash of less then stable late generation of wolf based puppylinux are due to internal timeouts of non responsive scripts IE takes too long to decompress those thousands of tiny scripts and the scripts that call the other scripts repeatively.
There is NO error trapping in linux for scripts faults due to not being returned to start a file that is not yet ready to read.

HOWEVER the way BK is doing it now with his hobby post-retirements Quirky builds corrects this issue.

So at this time only Fatdog64 with a kernel flag asked by me and BK newest Quirky will run in RAM uncompressed. And its like greased lighting...

the fat frugal type install ( decompressed within a file ) on the harddrive would free up RAM and speed up puppylinux spins and remove the tiny script slowing down the system overall. I feel this is the best direction that puppylinux should go.. hope those main developers can add the needed code in wolf start scripts.

[Ted Dog]

there is a forum member making and remaking a special remaster script ( program lol ) I tried to step up and add my goodies ( I have made such systems for my self, It tests those changes above but its a complex task done by hand )
But he went into a teardown and rebuild effort just as I wanted to work with him.

So hopefully with his tools a fat frugal methodology could be an option. But that is a stop gap. The best way to do it is BK new way.

[rufwoof]

There is another method like frugal install but not as frugal on harddrive space. Use squashfs as just packed only not compressed or better move all files into a savefile ext4 etc.

Thanks Ted

Just tried a HDD based frugal (ram boot, no save file used) remastered with no compression i.e. -noI -noD -noF mksquashfs parameters, with the puppy sfs merged into initrd and initrd uncompressed. Boot time seems a little slower (1 second or so) and general operation seems quicker (so fast anyway its difficult to be sure). I do like how it remasters in seconds.

[mikeb]

sort of drifted into another topic though must say lzma/xz does slow things down noticeable for both compressing and decompressing.. We made some gz only builds with varying compression levels and the difference was definitely marked... perhaps less zealous space reduction would be a simple step to take.

I did make a save option for pup that basically installed the sfs into a large save file and ran without layering at all like a full...but installation resembled a frugal...ie any file system as host...first tested in puppy 2.12.

Worked rather well with the usual need to allow enough space for growth of the system.

I am sure there are technical reasons for scripts being handled slower than optimum but I also find many of those pup scripts are painfully inefficient in themselves...Ie better scripting could result in some large speed ups (I get lucid booting 4 times faster for example.)

Non of this affects the apparent need for magnitudes more ram for the running system however its put together though.

mike

Mike

[Ted Dog]

@rufwoof

could you post the command used I have not rebuilt mine in a while and would have a long time finding my code. check the .history file its easy to cut and paste from it. Also if you would post it as text within a code section so we can cut and paste online.
Yes I had done this for the fast remaster in the early days of developing DVD multisession. You can recompress the initrd and I think people would be amazed how well it looks just like older method puppylinux size included.
That was the direction I wanted fatdog64 to go with the expand option.

[rufwoof]

In /usr/sbin/remasterpup2 I added/changed

Code: Select all

COPTIONS="-noI -noD -noF"

to add the no compression parameters

... and later ...

Code: Select all

squash / $WKGMNTPT/puppylivecdbuild/$PUPPYSFS ${COPTIONS} -e /etc /proc /initrd /var /tmp /archive /mnt /root $TOPPLCDB $ANOTHER_REMOVE $DIRHOME $DIRSYS $DIRLOST $TOPPUPSFS $DIRDEVSNDFILES $DIRDEVUDEV $ICONCACHE   icon-theme.cache 121021

and for the script to make initrd.gz into initrd with pup sfs installed

Code: Select all

#!/bin/sh
mkdir newdir
cd newdir
zcat ../initrd.gz | cpio -i -H newc -d
cp -p ../pup*.sfs .
find | cpio -o -H newc > ../initrd
cd ..
rm -rf ./newdir

[Ted Dog]

cool thanks...

[mikeb]

I apologise if my reply was tooo simple

mike

[rerwin]

In /usr/sbin/remasterpup2 I added/changed

Code: Select all

COPTIONS="-noI -noD -noF"

to add the no compression parameters

... and later ...

Code: Select all

squash / $WKGMNTPT/puppylivecdbuild/$PUPPYSFS ${COPTIONS} -e /etc /proc /initrd /var /tmp /archive /mnt /root $TOPPLCDB $ANOTHER_REMOVE $DIRHOME $DIRSYS $DIRLOST $TOPPUPSFS $DIRDEVSNDFILES $DIRDEVUDEV $ICONCACHE   icon-theme.cache 121021

This seems like a desirable update for lupu 5.2.8.7 to improve performance in low-spec PCs, with little impact for those with 512 MB or more. I am reluctant to mess with the initrd file, so would like your opinion on confining the improvement to only the squash file.

I am sure there are technical reasons for scripts being handled slower than optimum but I also find many of those pup scripts are painfully inefficient in themselves...Ie better scripting could result in some large speed ups (I get lucid booting 4 times faster for example.)

If you made changes to the lucid scripts, I would like to consider including them in an update to 5.2.8.7, assuming you feel that to be reasonable.
Richard

[mikeb]

If you made changes to the lucid scripts, I would like to consider including them in an update to 5.2.8.7, assuming you feel that to be reasona

I can do.... some changes are in the initrd.... I will see whats in there and the main boot scripts and pass on anything that may be usable without major rewrites

mike

[Ted Dog]

I am not sure you would need to change initrd if you do not move sfs in to it. I am working on using these code and Idea for use with a EFI boot loader in a zip file. Zip compresses everything so should shrink nicely.

[rufwoof]

This seems like a desirable update for lupu 5.2.8.7 to improve performance in low-spec PCs, with little impact for those with 512 MB or more. I am reluctant to mess with the initrd file, so would like your opinion on confining the improvement to only the squash file.

Merging puppy sfs into initrd means that initrd doesn't have to search for it during bootup (quicker boot). Also means that PXE based booting is simpler (i.e. I also sometimes boot a Win machine via network boot from my main puppy machine).

With squash file uncompressed I am noticing differences after the day or so that I've been using that. I ram boot and have no savefile as my core puppy is basic (leafpad, calc, mplayer, usual basic puppy programs) and the rest I load as SFS's as/when required (libre etc) and I run with portable firefox (that's installed on HDD). All data/docs I also store outside of puppy (HDD).

So I run 100% in ram - excepting program config files that I sym link to HDD so that program config changes are preserved across reboots, and data/docs files I/O, and Firefox (I do have a script to download the latest firefox into ram and run that if I want a quicker running firefox).

I boot via Grub, so a large puppy sfs (uncompressed) that's read from HDD isn't much of a delay - around 1 or 2 seconds longer bootup. That uncompressed puppy sfs is around 265MB. For CD boot I use a high compressed puppy sfs on the CD, that's around 85MB (as CD's are relatively slow at booting when large files are being loaded) i.e. quicker boot, slower general running once loaded (CD) and a highly compressed puppy sfs is used, not much slower boot, faster running once loaded when Frugal boot using HDD/Grub and a non-compressed puppy sfs is used.

[mikeb]

again heavy compression seems to be the problem here...drives are faster, RAM is bigger ...size is less stringent. A beast like libreoffice IS slower with xz compared to gz when starting up....just cut back on the compression...easy....no compatibility problems and one tiny change in one script line.

And thats on pentium 3 machines and if there is a puppy cut off line I am sure such machines are on it.

Merging puppy sfs into initrd means that initrd doesn't have to search for it during bootup (quicker boot). Also means that PXE based booting is simpler (i.e. I also sometimes boot a Win machine via network boot from my main puppy machine).

ok well to me that highlights a failure in logic.

Standard initrd with no boot parameters scans sub (and sub sub? ) folders...that is a painful user of time and if there is windows around takes ages.... and for no point. If a frugal install is made then the location (folder) is supplied in the boot stanza.
If its done manually the same. Using the boot flags has been considered standard procedure on puppy for years (just how slax has always done it as it happens... they don't play guess the location because why would you)
So how exactly does the sfs (and save for that matter) get into this mysterious folder in the first place to warrant the ridiculous long search.
Eliminate that (hell looking in the root of the drive works just fine for parameterless booting for me...) and you immediately save a chunk of boot time....modification involves changing -maxdepth to 1 from 2!!!!

I know why its still like that.... boot from CD users.... scared of truly installing (WHY??) only instance of where the system files are scattered around media without abject silliness. Even then usually the sfs and save end up on the hard drive or usb but of course with the optical media you cannot add yer parameters.
By the way using drive root only in this instance solves the slowdown problem.

A reappraisal of such booting options and install options would help make for some easy speed ups.

I also suggest special modes for flash media are pointless... anything usb1 will be in the 'not boot from usb' category and its so damn slow whatever you do.... Usb 2/3... well hardly warrants special handling.
Means usb drivers can be built into the kernel...a pile of 'wait for usb to initialise' scripts can go too. (hate to mention again but slax from 2006/8 took this approach and it makes life faster and simpler)
To me the best replacement for PUPMODE=13 is sfs/tar saving of the tmpfs...but thats just my bias...otherwise have a save file or folder.
Oh and make sure sfs at boot are loaded to ram.

With any project its always worth re-evaluating why things are done the way the are now and then.

mike

[Keisha]

In /usr/sbin/remasterpup2 I added/changed...

... and later ...

Code: Select all

squash / $WKGMNTPT/puppylivecdbuild/$PUPPYSFS ${COPTIONS} -e /etc /proc /initrd /var /tmp /archive /mnt /root $TOPPLCDB $ANOTHER_REMOVE $DIRHOME $DIRSYS $DIRLOST $TOPPUPSFS $DIRDEVSNDFILES $DIRDEVUDEV $ICONCACHE   icon-theme.cache 121021

I suspect you mean:

Code: Select all

squash / $WKGMNTPT/puppylivecdbuild/$PUPPYSFS ${COPTIONS} -e /etc /proc /initrd /var /tmp /archive /mnt /root $TOPPLCDB $ANOTHER_REMOVE $DIRHOME $DIRSYS $DIRLOST $TOPPUPSFS $DIRDEVSNDFILES $DIRDEVUDEV $ICONCACHE  #120605 end #120721 avoid wildcards option, icon-theme.cache 121021

which is the line (line 483) from slackob64-5.9.1 remasterpup2