Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 26 Nov 2014, 12:26
All times are UTC - 4
 Forum index » Advanced Topics » Puppy Projects » Next Puppy Development
Possibilities based on working with SFS files in pup-431
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
Page 1 of 1 Posts_count  
Author Message

Joined: 28 Oct 2008
Posts: 523
Location: Brisbane, Australia

PostPosted: Mon 11 Jan 2010, 08:55    Post_subject:  Possibilities based on working with SFS files in pup-431
Sub_title: including dynamic loading and unloading

NOTE: all this stuff has been done on a frugal install of pup-431.

1) Myth - dynamic loading and unloading of SFS files dosen't work with aufs2 in pup-431 - busted!

True, the "load_sfs" script in common use at the time of the release of 431 did not work. But the problem lay with the script, not aufs2.
The "append" and "del" functions of aufs still work fine with the new stripped down version of aufs.
I have modified versions of "load_sfs" and "unload_sfs" that work in pup-431.

2) When an sfs file is removed, sometimes not all files disappear, and if you delete them, the old files don't reappear.

One of the useful things about using sfs files, is that when one is added, all the files just appear in the filesystem, no decompressimg or copying reuired. Then when the sfs file is removed, the files just disappear.
Unless, of course, a file in the sfs has been 'modified'. The new version of the file does not disappear because it is not in the sfs, but in the pup_rw layer.
An example, lets say you installed a "firefox-3.0.1.sfs" some time ago, and allowed it to automatically update to firefox 3.0.17. Now you remove the "firefox-3.0.1.sfs" and replace it with "firefox-3.5.7.sfs". The problem is that all the updates of firefox-3.0.1 to firefox-3.0.17 are still in the pup_rw layer, and 'cover' any firefox-3.5.7 files with the same filename. Hence you now have stuffed up firefox.
This is when aufs can seem to be a pain. It's clear what needs to be done, the offending files in the pup_rw layer, need to be deleted. But normal deletion won't work at this stage, since aufs will create ".wh." files so that the 'covered' files in the new sfs appear to be deleted.

An apparrently simple solution is to remove the new sfs, delete the offending files, and then add the new sfs again. This will most likely work, provided that none of the files have the same name as a file in any still active sfs files, in which case aufs will create a ".wh." file, and that file will disappear.
A reliable solution, is to boot into a different copy of puppy, i.e. one that uses a different pupsave file, mount the pupsave file containing the offending files as a loop device, and then delete the offending files. This works because the files in the pupsave are now mounted in a unique path within the filesystem.

I now have a couple of scripts that can safely remove offending files from the current pup_rw layer (e.g. pupsave.2fs). "rmfile-rw.sh" removes a single file or directory. "rmlist-rw.sh" removes all the files listed in the text file provides as a parameter. Both these have been developed on a PUPMODE=13 system, so they remove files from both pup_rw and pup_ro1 if necessary. Currently, the aufs cache gets fixed, but not the rox-filer cache, so it might be necessary to refresh an open rod-filer window.
In the firefox example above, simply running "rmfile-rw.sh /usr/local/firefox" would fix the problem on my machine, since I install firefox in "/usr/local/firefox/".

3) Modifying "init" so that pup-431.sfs gets loaded at the bottom of the aufs stack.

One of the problems with the current way of loading sfs files in Puppy, is that they can't overwrite any thing in pup-431.sfs. A typical example would be a "firefox.sfs" wanting to overwrite "/usr/local/bin/defaultbrowser".
One solution is to modify "init" to seperate pup-431.sfs from the pupsave stuff when defining the aufs stack. This means pup-431.sfs can be loaded last instead of straight underneath the pupsave stuff (pup_rw). This means that a "firefox.sfs" can now simply 'cover' "/usr/local/defaultbrowser".
Unfortunately this produced a number of nasty side-effects. The problem is that none of our current crop of sfs files have been only been tested while being loaded under pup-431.sfs, so when they are loaded above pup-431.sfs some unwanted things appear. A simple example is devx_431.sfs, this contains the normal "man" program, which usually gets 'covered' by the puppy specific "man" in pup-431.sfs, not a problem. But with devx_431.sfs above pup-431.sfs, the useful "man" in pup-431.sfs gets 'covered' by the useless "man" in devx_431.sfs.
So doing this could result in strange thins happening until all sfs files where revised. Perhaps a corageous course.

I have implemented a possible compromise, a modified pup-431 that introduces a new sfs file that I have called "zad_431.sfs" which gets loaded at the bottom of the aufs stack. The idea is that this sfs file contains all the stuff that currently resides in pup-431.sfs, that folk would want to overwrite. And the slightly smaller pup-431.sfs still gets loaded above other sfs files, as is the current situation.
This sfs file contains all the application decouplers, i.e. all the "/usr/local/default*" scripts + /"usr/bin/xterm". It currently also contains all the "/usr/share/applications/*.desktop" files.
So a "forefox.sfs" can 'cover' "/usr/local/defaultbrowser" but devx_431.sfs will not 'cover' "man".

A possible extension of this concept would be to divide each puppy release into two sfs files, a system sfs (pup-431.sfs) and an applications sfs (apps-431.sfs). Then folk could create alternate applications sfs's. Or simply load the system sfs high and the applications sfs low.

Another possibility which I haven't tried yet, would be to introduce another sfs file called pat-431.sfs.
This sfs file, if it exists, contains patches to pup-431.sfs, and is always loaded above pup-431.sfs.

4) Manipulating sfs files.

Confession: My current 'production' puppy 431 system, runs in a .pet free state.

I have a sctipt called "mixfs.sh" that can combine several sfs files, or directories containing 'filesystems' into a single sfs.
If I want to use a .pet I extract it to a directory and then add the name of that directory to the list of directories to be included in my single "my-apps-sfs4.sfs".
I use the same mechanism to directly patch pup-431.sfs and devx_431.sfs.

a) When running Puppy from the SD card of my EeePC, it was very very slow in saving pup_save.2fs to the SD card. So I began to consider how I might minimise the size of the pupsave file.
b) I realised that most of any software package is inherantly read only, so it is a waste of valuable read/write space to install it there, it would be more appropriately stored in an sfs file.
c) Since I discovered Puppy's propensity for defecating in it's own nest by not cleanly unmounting pupsave files in a frugal install; I simply don't trust pupsave to be reliable, and I want to be able to discard pupsave with a minimum of inconvenience.

I have attached a ".tar.gz" file containing the mentioned scripts, and a ".gz" file containing zad_431.sfs

Description  The scripts I use as mentioned above.

Filename  gyro-sfs-scripts.tar.gz 
Filesize  2.76 KB 
Downloaded  1048 Time(s) 
Description  An example of suggested extra sfs for puppy releases.

Filename  zad_431.sfs.gz 
Filesize  11.75 KB 
Downloaded  973 Time(s) 
Back to top
View user's profile Send_private_message 

Joined: 05 Aug 2007
Posts: 1738
Location: Baltimore, Maryland USA

PostPosted: Mon 11 Jan 2010, 10:54    Post_subject:  

Sounds interesting....do your scripts overcome the low number of SFS files that 4.3.1 is able to load at startup?

Back to top
View user's profile Send_private_message Visit_website 

Joined: 28 Oct 2008
Posts: 523
Location: Brisbane, Australia

PostPosted: Mon 11 Jan 2010, 12:31    Post_subject:  

tlchost wrote:
Sounds interesting....do your scripts overcome the low number of SFS files that 4.3.1 is able to load at startup?
My memory is that other folk on the forum have addressed this issue.

Back to top
View user's profile Send_private_message 

Joined: 29 Jun 2008
Posts: 10557
Location: SwedenEurope

PostPosted: Fri 02 Apr 2010, 08:49    Post_subject:  

Sounds as a good thing you have going there.

doesn't the other devs have to know about it. Would it be too "much" to send a pm to them to look at this thread.

I would next puppy to have these features to load and unload sfs files.

I use Google Search on Puppy Forum
not an ideal solution though
Back to top
View user's profile Send_private_message 

Joined: 10 Nov 2009
Posts: 618
Location: Austria

PostPosted: Sat 08 Jan 2011, 16:12    Post_subject: How to combine  

goingnuts wrote OTF SFS Loader which can dynamically load sfs files.
It is in the pupngo thread
Until now it just works with aufs1 (till puppy 4.2). I think your script can make it work also for newer puppies.
Great ... Very Happy
Back to top
View user's profile Send_private_message Visit_website 
Display_posts:   Sort by:   
Page 1 of 1 Posts_count  
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
 Forum index » Advanced Topics » Puppy Projects » Next Puppy Development
Jump to:  

You cannot attach files in this forum
You can download files in this forum

Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0662s ][ Queries: 13 (0.0051s) ][ GZIP on ]