Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Thu 19 Oct 2017, 16:33
All times are UTC - 4
 Forum index » House Training » Users ( For the regulars )
SFS files vs program folders: test results are in!
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [13 Posts]  
Author Message
mikeslr


Joined: 16 Jun 2008
Posts: 1929
Location: 500 seconds from Sol

PostPosted: Mon 18 Feb 2013, 22:18    Post subject:  SFS files vs program folders: test results are in!  

Hi All,

Pets vs. SFSes vs. Program Folders? And the winner is Program Folders! Maybe.

As the attached Chart Shows, Program Folders use less of your computer's resources, which means they can more efficiently be run on computers with less resources. This is particularly important if limited RAM is a concern. Or the size of your SaveFile: note in particular that to successfully install LibreOffice as a pet required 500 megs of a SaveFile just for itself, requiring the SaveFile to be resized from 512 to 768 Mbs. Only SFSes use available Hard-drive (including USB-Key drives if you have that type of installation) space more efficiently. Using 01micko's Get-LibreOffice pet to download LibreOffice4 created a 144 Mb SFS in xz compression. Converting to gz compression resulted in a 173 Mb file. Decompression resulted in a folder occupying 543 Mbs of harddrive. But is Hard-drive space a significant concern today? Similarly, Program Folders make fewer demands on one's CPU.
Perhaps this is why there are advocates of Full Installs. Like Full Installs, the files in Program Folders are not compressed. Or why Kirk and Jamesbond have developed the use of a Save Folder as an alternative to a SaveFile in Fatdog64?
So why the “Maybe”?
Although almost any application can be restructure as a Program Folder, if aesthetics is a concern you'll want to do something more than drop the application's executable on the desktop and give it a pretty icon. Adding the application to the menu or to Wbar or other launchers takes some time-consuming grunt work. That use of your time has to be balanced against the savings in resource usage. It may only make sense if the application is large –or a “suite” not all applications of which must be opened and running at the same time-- or is frequently put to use.
The "almost" at the beginning of the last paragraph exists because not every application can be run as a Program Folder. Some expect your OS to use a version of glib not present. Others may require settings contained in a pinstall which may take some programming knowledge I don't have, and you may not have either.
But if they can be used they are particularly appropriate if you have one of the following concerns:
1. In your desire for security you run your Frugal Puppy without a SaveFile. Once you've created a pet to install the links between your Puppy and the executable in the program folder, installing that pet into your Temporary RAMDISK –it uses only a couple of kilobytes-- takes about about a minute and, thereafter, until you shut down, starting the application is convenient. In fact, you can create one pet to link all your commonly used applications. Or, of course, you can just click the executable in the Program's Folder.
2. Yours is a Full Install. With a Full install, your Program Folder can either exist outside of the partition used for your install, or be moved outside of that partition when you intend to upgrade or otherwise change your Puppy. No need to start from scratch to use that application in your new Puppy. To try out a different version of an application, just create a folder for that version and test it before changing the links and deleting the old version. A Program Folder (except for your system's links to it) is self-contained. No need to figure out how to uninstall all its components.
3. The Application you sometimes want to use conflicts with something else. Program Folders are self-contained. Using them won't overwrite the libs of any other program.

Procedure used in running the tests:
As mentioned above, I started by downloading Libreoffice4 using 01micko's Get-LibreOffice pet. Having read that Lupus couldn't work with SFSes xz compressed, I decided to convert it to gz compression. In the middle of doing so, I realized that an uncompressed version was available. So I copied it to a folder. Later, I copied that folder into a temporary folder and used dir2pet to create an installable pet. That gave me 4 versions of the otherwise identical application: 01micko's xz SFS, an installable pet, a Program Folder and a gz SFS. The last wasn't used in the tests.
In retrospect, the tests would have been more informative if I had run them on my 10 year old single-core Thinkpad T42 rather than my Quad-Core 4x AMD Phenom(tm) II X4 945 Processor with 3375MB of RAM. The Thinkpad uses a Swap file, and CPU usage would have been more intense but also better defined than the broad strokes shown by of swings between 0% and 5%. Consequently, the recordings of cache and buffer usage –as they show activity the CPU has just engaged in-- are probably more informative than the CPU activity reports. But this started out as just a simple question to be answered in what I thought would take an hour. Only a stubborn streak kept me from abandoning it after 10 hours. The problem I ran into was testing the installed pet. I was on my third test vehicle -01micko's ThinSlacko-- and second LibreOffice Version (having tried version 3.6.3 when Version 4 initially failed) having previously assumed that failures resulted from some files being absent in Barebones-Precise and PreciseNOP or that I had somehow screwed up dir2pet when the installation failed in ThinSlacko. Only then did it occur to me that perhaps a SaveFile larger than 512 Mbs was necessary. You have my blessing if you want to perform the tests on your own less powerful computer using a 768 Mb SaveFile throughout. Please post your results.
At any rate, I chose ThinSlacko because it included few applications by default. To it, at first boot pfix=ram, I set my location per First Run Dialog and set up wireless using SNS (on the assumption everyone would do that) and only added Sys-info-1.4.2-noarch.pet and Pup-Sysinfo-2.1.7.pet. I then cleared /tmp and used PupSysInfo>System-Specs>Memory and System-Apps>Task Mgr to obtain readings. PupSysInfo...Memory takes a snapshot, while Task Mgr shows CPU usage as it takes place. That provided the figures shown in the first row. I then shutdown creating a 512 SaveFile, rebooting into a different Pup.
While in that other Pup, I deleted the “ram” argument from menu.lst, created a protected folder and copied the SaveFile into it. I then rebooted into ThinSlacko, installed the pet providing access to Libreoffice4-External and rebooted. After rebooting, since SysInfo>Memory provides a snapshot, the next step I took was to use rox to browse to /tmp, show hidden files and delete every one. I deleted those files before initiating each procedure (except obtaining the CPU readings) so that unnecessary files, no matter how small, would not be counted: that is, before taking readings after installing pets/loading the SFS, after opening Libreoffice4 writer, and after opening a 38.2 Kb document in writer. This wasn't done before using Task Mgr as it reports activity currently taking place. The CPU change from Zero to a different percentage was generated by dragging the document from one position on the screen to another when the document was on the desktop, and when the document wasn't, dragging the SysInfo window from one position to another.
After taking all the reading pertaining to Libreoffice4-External, I booted into a different Pup, deleted the SaveFile and copied the Protected SaveFile into ThinSlacko's folder, booted into ThinSlacko, loaded the SFS on the fly, rebooted and re-ran the tests. I similarly replaced the “already used” SaveFile before testing Libreoffice4 installed as a pet. With hindsight, I should have tested that version first. Had I done so, I probably wouldn't have spent hours recording data I couldn't use because I didn't have comparable readings from an installed pet version.

For instructions on how to obtain/create your own Program Folder, see: http://www.murga-linux.com/puppy/viewtopic.php?p=686580#686580.
Free Storage in the following Table refers to the reported amount of Free Space remaining in the SaveFile.
mikesLr
Comparison.png
 Description   Comparison of System Usage by Pets, Sfses & Program Folders
 Filesize   105.06 KB
 Viewed   1175 Time(s)

Comparison.png


Last edited by mikeslr on Mon 08 Jun 2015, 08:22; edited 1 time in total
Back to top
View user's profile Send private message 
cowboy


Joined: 03 Feb 2011
Posts: 252
Location: North America; the Western Hemisphere; Yonder

PostPosted: Mon 25 Feb 2013, 21:31    Post subject: program folders post  

mikeslr,

I shudder when I think of the time this must have cost you - just wanted you to know that your experiments, and explanations, of the uses of program folders are great reading.

I appreciate the efforts that you make in discovering more efficient ways to operate in Puppy. Always pleased to see a "mikeslr" post on the forum.

Thanks.

_________________
"you fix what you can fix and you let the rest go.." - Cormac McCarthy - No Country For Old Men.
Back to top
View user's profile Send private message 
mikeslr


Joined: 16 Jun 2008
Posts: 1929
Location: 500 seconds from Sol

PostPosted: Tue 26 Feb 2013, 11:10    Post subject: LibreOffice4 Progrm Folder Crash  

Hi Cowboy,

Thanks for the appreciation.

LibreOffice4 run from a program folder crashed in both Precise and retro-precise. It didn't in Thin Slacko. I don't know if the problem is with the build, the Pup or other. So on my desktop, where resources are not an issue,
Edit: LibreOffice4 both as an xz SFS and as a Program Folder crashed in Exprimo. I've reverted to 3.6.3 gz SFS.
I do strongly recommend FreeOffice for use when resources are limited. It was designed to run "as a portable" --from a folder-- which only occupies 99 Mbs of disk space. Think Puppy from a USB-Key. Reads and writes Microsoft file format, although the first time you save a document you'll have to select other than its native format. If set as default, clicking an ".odt" text document, an ".odg" presentation or an ".xls" spreadsheet will open it, but doesn't recognze the ".ods" --default Libre/OpenOffice-- spreadsheet file format however you try to open it.
Other than the perversity of Microsoft and others, I don't know why .xls isn't the default spreadsheet format in all spreadsheet apps. Or for that matter, there isn't a universally used file format for text and presentation files. It's not like graphic, sound and video where formats indicate different compression or containers used, is it? Who buys/uses business applications because of the unique file formats of their applications?

mikesLr
Back to top
View user's profile Send private message 
Pelo

Joined: 10 Sep 2011
Posts: 12591
Location: Mer méditerrannée (1 kms°)

PostPosted: Wed 23 Aug 2017, 11:42    Post subject: mikeslr that is real good work,full study.  

mikeslr that is real good work,full study. I use SFS a lot, but Libre Office, quite never (version Windows in case of need, in a far future)
"whereas,
Care of CPU, this is where apps fail or not. RAM is there to help, use it.
basic Linux idea is that unused memory is wasted memory.
" And i agree with that. ".so it
is typical in Linux to see RAM full or near full all the time...and,
it will stay that way, and dump out the oldest stuff as soon as
something _new_ is opened which needs space.." to Swap, is existing..

RAM is a Kitchen, the cooker needs the more as possible untill the cooker can open the fridge door or the oven. Or in an aircraft it would be the Cargo compratment, that avoid the pilot to land to load some food or kerozen.

I load as SFS full distro , (famous underdog) and its rare that my laptop crashes. It happened with my acer aspire 512mb RAM. But with 512MB RAM Libre office will crash as soon you work with a document with some 'mise en page', i mean titles, graphics and images included. Not because of RAM, Processor (only one often) cannot manage all changes to do just by typing a new line, that will change pages, summary and so on..

The problem with a suite, it's when only writer is needed.. if ever it is needed by Puppy users. Never writer will knock down processors... Look how big is writer in Libre Office..
Often processors are too busy at upload, opening docs, but not after..
Libre Office is very slow, but more with calc.. Or Big documents, with images included..But Microsoft office was not better.. Word suddenly garbled everything, when i was at the office.
Don't deny Abiword, because it is a light text maker, processors will loose the thread later..

Try not to include Java.
"Libreoffice calc uses all memory until operating system freezes, when using external references to websites" official bug in
Then when your document is 5GB, of course it is needed to upload it full in RAM, spreadsheet or text, if your RAM is 4GB, processors cannnot send anything to swap.

_________________
Passenger Pelo ! don't ask him to repair the aircraft. Don't use him as a demining dog .... pleeease.
Back to top
View user's profile Send private message Yahoo Messenger 
foxpup


Joined: 29 Jul 2016
Posts: 238
Location: europa near northsea

PostPosted: Wed 23 Aug 2017, 15:29    Post subject:  

Nice work mikeslr! What hard work!

It confirms a lot of my impressions.

I used to use LibreOffice a lot. I used it with Wary5.5. The last version of LO for Wary is the last 5.0.1 version iirc, because of general lib issues.
I also preferred the portable mode: just unpack on /mnt/home/ in a folder and copy .desktop files into /usr/share/applications. The .desktop files need some editing to define the correct paths. That's all.
Firefox is similar! And there is also the fine firefox-portable from Shinobar that helps you to set it up.

One other reason I like portable is this: I always have a few puppies installed (frugally) to choose from. They can all use the same portable! Well not entirely, it depends a bit: 32 or 64 bit, more recent or older puppies, but in general, a lot of puppies can share the same portable.

For java in LibreOffice: it is rarely needed except for the database.

mikeslr wrote:
Like Full Installs, the files in Program Folders are not compressed. Or why Kirk and Jamesbond have developed the use of a Save Folder as an alternative to a SaveFile in Fatdog64?
Savefiles are not compressed either, are they? One advantage of a folder over a file that I can think of, is, you do not need to resize, the size is not fixed.
Back to top
View user's profile Send private message 
mikeslr


Joined: 16 Jun 2008
Posts: 1929
Location: 500 seconds from Sol

PostPosted: Wed 23 Aug 2017, 17:43    Post subject:  

Hi foxpup,

Well, a SaveFile, having a fixed size, isn't, itself, compressed. But the files you install into them are. Hence the acronym "sfs" --squashed file system. The compression and decompression take place 'behind the scene' managed by Puppy when you've chosen a Frugal Install.

If you compare the size of an SFS to that of a pet of the same application you might not realize that SFSes are compressed because pets are also compressed.

Usually, gzip compression is used. This chart will provides some idea of how much compression takes place, and also the time and computer resources involved. https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO. The time and memory factors were one of the reasons a "Full Install" was developed to be used by those computer systems which have less than 256 Mbs of RAM.


mikesLr
Back to top
View user's profile Send private message 
Peterm321

Joined: 29 Jan 2009
Posts: 382

PostPosted: Wed 23 Aug 2017, 22:34    Post subject:  

A while ago I recompiled mksquashfs to use gzip with compression level 1. The default compression was slow. For a relatively small increase in SFS size, thje time taken to make the SFS was greatly reduced. I presume there might be some benefit in using fast compression: faster decompression, eg if using SFS to store an archive of programs. I never did any benchmarking to be sure.

The disadvantage of the version I uploaded is that it does not support xz / bzip etc compression.

http://murga-linux.com/puppy/viewtopic.php?t=89173
Back to top
View user's profile Send private message 
LazY Puppy


Joined: 21 Nov 2014
Posts: 2007
Location: Germany

PostPosted: Wed 23 Aug 2017, 22:50    Post subject:  

Peterm321 wrote:
A while ago I recompiled mksquashfs to use gzip with compression level 1. The default compression was slow. For a relatively small increase in SFS size, thje time taken to make the SFS was greatly reduced. I presume there might be some benefit in using fast compression: faster decompression, eg if using SFS to store an archive of programs. I never did any benchmarking to be sure.

The disadvantage of the version I uploaded is that it does not support xz / bzip etc compression.

http://murga-linux.com/puppy/viewtopic.php?t=89173

I'm still using it in all my 32bit derivatives!

Would be cool to have a 64bit version?

_________________
RSH

"you only wanted to work your Puppies in German", "you are a separatist in that you want Germany to secede from Europe" (musher0) Laughing

No, but I gave my old drum kit away for free to a music store collecting instruments for refugees! Wink
Back to top
View user's profile Send private message 
Pelo

Joined: 10 Sep 2011
Posts: 12591
Location: Mer méditerrannée (1 kms°)

PostPosted: Thu 24 Aug 2017, 02:02    Post subject: Highly compressed does not mean less RAM,
Subject description: it means lot of work.
 

Compressed means less size for storage, generally by removing all blanks (not only). like the air of inflatable boats, you will have to blow to re-inflate.
Is the question really a problem ? I like SFS. When choice if offered i choose SFS. Pet only if stuff is needed permanently. So you could understand why i am against Libre Office included in ISOs, huge ISOs that needs time to download. L.O is not only writer, even if you don't use calc, maths, and databases. We know that, look at 'business' chapter of the forum.
if you need writer, use Abiword or tiny text makers. 7MB i/o 250, no need to flat and unflat. to zip and unzip, that will slow your computer and get your old processor explode at opening your old laptop.
Highly compressed does not mean less RAM, it means lot of work.
SFS on the fly should be unmounted when shutdown.
Use tools for Puppy, i know this is not easy nowadays. Debian Packages are raw... nobody to trim the fat. Libre office should be trimmed too. We don't need the English dictionnary as we don't type in English, if ever we had to type. PuPPy Linux users are not office workers. Devs, please stop playing businessmen.. Nobody use Puppy to work. To play developpers, yes, there are a lot (not so many, 200 ?)
Use light OOo. Not enough for you ? wait for the passengers claims,
Sfs Kde Office is 47MB.. well it is full of bugs.

_________________
Passenger Pelo ! don't ask him to repair the aircraft. Don't use him as a demining dog .... pleeease.
Back to top
View user's profile Send private message Yahoo Messenger 
Pelo

Joined: 10 Sep 2011
Posts: 12591
Location: Mer méditerrannée (1 kms°)

PostPosted: Thu 24 Aug 2017, 02:25    Post subject: computers with 256Mb RAM...?  

computers with 256Mb RAM.... with 512MB on my Acer Laptop. It is difficult to have fun. Compression has nothing to do, everything has to be decompressed to run. That is more work, not less. Evil or Very Mad
_________________
Passenger Pelo ! don't ask him to repair the aircraft. Don't use him as a demining dog .... pleeease.
Back to top
View user's profile Send private message Yahoo Messenger 
Peterm321

Joined: 29 Jan 2009
Posts: 382

PostPosted: Fri 25 Aug 2017, 02:06    Post subject:  

LazY Puppy Wed 23 Aug 2017, 22:50 wrote:

Would be cool to have a 64bit version?


It would have to be compliled in a 64 Bit Puppy to work.

With 2GB of RAM, I have never even tried a PAE / 64 Bit OS of any
type, never mind downloaded one with the DEVX module which would be
needed for compiling.

But the hack (if even you could call it a hack) was simplicity itself.
If memory recall is right, a change to just one line of code. NB to
replace the default gzip compression of 9 to 1. If you knew where that
9 was (presumably an INT or SHORTINT 32/16 bits) it might be
adjustable by binary editor.
Back to top
View user's profile Send private message 
mikeslr


Joined: 16 Jun 2008
Posts: 1929
Location: 500 seconds from Sol

PostPosted: Fri 25 Aug 2017, 14:52    Post subject: Program Folders -- Still Useful, but not as much  

Hi All,

Thanks pelo and foxpup for digging up this old thread. IIRC, it predates gyro's development of SaveFolders which may be why tests using SaveFolders weren't conducted.

Unlike a SaveFile, which has a fixed size and whose content is compressed, a SaveFolder has no fixed size --starts small, and as entries into it are made expands automatically to, if necessary, the entire available space of the partition/drive on which it is located-- and the contents you install aren't compressed.

Consequently, many of the benefits derived from using a Program Folder can be obtained simply by using a SaveFolder rather than a SaveFile.

Additionally, the amount of RAM in (or which can be installed into) modern computers has reduced the advantage. IIRC, the computer I ran the test on had 4 Gbs of RAM. My current Desktop has 8 Gbs. How much significance does an additional 100 or so Mbs (or even 500 Mbs) of RAM for applications usages have?

There remain some circumstances where I recommend the use of Program Folders even when a SaveFolder is used. My experience has been that despite taking every precaution I could think of to prevent applications from being automatically written to a SaveFolder --remove Automatic Save, configure PPM to install to /tmpfs-- occasionally and randomly some applications or (worse yet) parts of applications were automatically written to SaveFolders. More an annoyance than anything else, as I haven't known those occasions to actually cause problems. But, obviously, if in building a yet-to-be-tested application it never was permitted to install anything, even that annoyance would have been avoided.

My current practice, therefore, is to primarily use SFSes of large applications; especially when they've been packaged by someone else. Smile

Tahrpup64 and Xenialpup64 (and their remasters) don't manage multi-architecture systems (64 +32 bit) and their libraries the same way as the Ubuntus whose binaries were used in building them. A program folder, like an SFS, isolates an application's libraries from the folders of a Puppy's shared libraries. Such isolation may enable the application to locate the libraries it requires.

I find several applications which can be 'run without installing' useful. Among those are blender, cool-reader, foxitreader, and masterpdfeditor. Basically, you just download their respective package, decompress it anywhere, and click its executable/binary*. [Discussions elsewhere on how to create menu entries or run by dragging binary to desktop]. When a new version is available, I can try it out without overwriting the old version. So, essentially, they have already been structured as 'Program Folders' and the only reason I can think of for including them within a Puppy --such as in /opt-- would be to make it easy to include them in a remaster.

mikesLr

* Some of these have as dependencies Qt or other libraries. I do install those libraries. Still, running a 'pre-packaged' application which requires the installation of a few libraries is easier than having to build it from scratch.
Back to top
View user's profile Send private message 
Pelo

Joined: 10 Sep 2011
Posts: 12591
Location: Mer méditerrannée (1 kms°)

PostPosted: Wed 30 Aug 2017, 10:46    Post subject: to remember  

"I find several applications which can be 'run without installing' useful. Among those are blender, cool-reader, foxitreader, and masterpdfeditor"
_________________
Passenger Pelo ! don't ask him to repair the aircraft. Don't use him as a demining dog .... pleeease.
Back to top
View user's profile Send private message Yahoo Messenger 
Display posts from previous:   Sort by:   
Page 1 of 1 [13 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » Users ( For the regulars )
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 1.1763s ][ Queries: 13 (0.8187s) ][ GZIP on ]