Setup for advanced "sfs" Squash files & sfsManager.

What features/apps/bugfixes needed in a future Puppy
Post Reply
Message
Author
User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

Setup for advanced "sfs" Squash files & sfsManager.

#1 Post by sunburnt »

Usually files must be copied up to the Save file layer to install sfs files (MegaPup).

To use squash files as auto. install packages:
If a hidden dir.: /.install is found on the sfs file, sfsManager copies it's contense up to the Save file layer.
This would allow auto. sfs file install of even heavy apps. (KDE, Gnome, & even LanPuppy).

For dynamic menus & desktop icons:
Another hidden dir.: /.sfs would have menu additions for JWM, IceWM, Xfce, & maybe KDE & Gnome.
Icons for the ROX desktop & maybe other desktops are also in: /sfs
sfsManager would handle placing the menu items & icons.

Removing an sfs file:
The sfsManager would first check a list: /.sfs/apps.lst against "ps"
& show a warning for any apps. in the list that're running.
Then the menu items & icons are removed & the sfs file's all cleaned up.
If there's problems with the install files in the Save file, sfsManager could
also remove them with a tracking list: /sfs/install.lst


This is a rough draft for a system of swappable auto. install sfs files.
I hope a discussion will ensue & a good solid plan will emerge.
I think I've covered most of the basics that're required for workablity.

NOTE: The current version of sfsManager only adds or removes sfs files.
For Puppy-1.x.x versions ONLY, a TEST copy of sfsManager is at:
http://208.109.22.214/puppy/viewtopic.p ... 0ff0f38733

So..... What do you think Puppyites? Ideas, thoughts, rants, etc.?

User avatar
Gekko
Posts: 443
Joined: Sat 22 Jul 2006, 09:57
Location: Sydney, New South Wales

#2 Post by Gekko »

This could be used as an extensive form of compression?

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#3 Post by sunburnt »

Hi Gekko; I'm not sure what you mean, a Squash file squashes dirs. & files 2 to 3 times.
It's used as live r/o file system, read directly by the kernel, but not written to.

If your thinking of compression like Zip of Gzip I suppose it could be used that way.
It wouldn't be as widely useful like Gzip which is an executable & runs on any Linux & Win.
SquashFS is a Linux kernel module, that's needed to use the files as a live file system.

The utility: /usr/bin/mksquashfs makes a new Squash file or adds to it.

GuestToo
Puppy Master
Posts: 4083
Joined: Wed 04 May 2005, 18:11

#4 Post by GuestToo »

i thought the way .sfs files usually work, is that the file system in the .sfs file is mounted as a unionfs layer ... so the files in the .sfs file are not copied at all (unless you have a full hd install, in which case there is not pup_save file and the file system does not consist of unionfs layers)

a tar.gz file could be used as a container for files that would be copied to the file system ... what makes .sfs files useful, is that the file system in the sfs file is not copied, it is mounted, which doesn't use up space ... but having too many unionfs layers slows things down

you could set up a Run Action / File Association so that when a .sfs file is clicked, the files in the sfs file system would be copied to the correct locations ... or it might be mounted using unionfs when the file is clicked

DavidBell
Posts: 132
Joined: Fri 24 Nov 2006, 21:44

#5 Post by DavidBell »

... but having too many unionfs layers slows things down
Is there a practical limit to the number of sfs files? The reason I ask is that, speaking as a puppy novice, it occured to me it would be nice if the applications came in their own sfs separate from pup_###.sfs. So it would be something like ...

boot (?) : vmliniz, initrd.gz
pup_###.sfs - contains core functions, no applications at all, only X, maybe rvxt, rox
zdrv_###.sfs - drivers
apps_###.sfs - current application suite
pup_save.3fs - user data

The reason I think it would be neat would be that for people making modified puppies they would just make their own AlternateApps_###.sfs, and users could try different versions just by swapping the one sfs, so they would have roughly 30MB less to download to test different versions.

Also, provided having a lot of sfs wasn't a problem you could break apps_###.sfs into several like InternetApps.sfs, OfficeApps.sfs, MediaApps.sfs, UtilityApps.sfs, so the user could pick and choose which parts they wanted, similar to the way you can now by adding devx_###.sfs.

Top it all off with nice tools for switching them in and out of puppy without rebooting, plus a way for a user to make their own MyApps.sfs and it would be great system IMO, and very flexible. It would be extra good if the *Apps.sfs's could be more or less version independent, so to upgrade in a lot of cases you would only need to update the basic pup_###.sfs, or the other way round - leave your base as is and just update the apps.

Just some thoughts, that have probably been raised before. DB

User avatar
MU
Posts: 13649
Joined: Wed 24 Aug 2005, 16:52
Location: Karlsruhe, Germany
Contact:

#6 Post by MU »

David, the more .sfs files are used, the slower the system works (it is said, that this has to do with the number of loop-devices supported by the Kernel).
I think Puppy supports 8 loop-devices, 4 reserved for .sfs.

I made an approach to deliver a "user"-.sfs with a CD:
http://www.murga-linux.com/puppy/viewtopic.php?t=13396

I'd be happy, if such a mechanism could make it officially into Puppy, so that I don't have to patch initrd.gz from Muppy with every new Puppy-version.

Background-info (loop-devices):
http://www.murga.org/~puppy/viewtopic.php?p=32709#32709
http://www.murga.org/~puppy/viewtopic.php?p=40113#40113
http://www.murga-linux.com/puppy/viewtopic.php?t=7482

Mark

DavidBell
Posts: 132
Joined: Fri 24 Nov 2006, 21:44

#7 Post by DavidBell »

Well I didn't understand what was going on in any of those links :-) ... but fools rush in ....

I wonder if it would be possible to build pup_###.sfs on the fly at boot. So then it would still only use one sfs but still have the flexibility. Eg if there was a script that ran during boot that found Pupconfig.ini, that would tell it to create pup_###.sfs by merging core_###.???, PuppyApps.???, AnotherAppSuite.???, etc.??? (by .??? mean I don't know what format you would use).

I'm guessing because sfs are only mounted rebuilding one would be a lot slower than opening one, but it could rebuild only if it detected a change in what was available (like make), and then maybe present the user with options. So first boot might be slow but after that it would be normal. Once booted you could have wizard for switching bits in and out.

Does that make sense?

DB

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#8 Post by sunburnt »

David Bell; I think 5 unions is the max, the sfs files would best be made by job type.
Like: graphics design, audio-video, OpenOffice, developer, servers, & etc.
Each sfs file is ~ 50 to 100 MB, larger ones can easily be made from smaller ones.
A boot menu could be used for Puppy 2, in Puppy 1 you can cange them anytime.
Building one on the fly could probably be done from Puppy unleashed source files.
The remaster script on the menu makes an sfs file, but only from CD-DVD & full HD installs.
MU's correct, the cpu has to check multiple file sys. to access any file IN THE UNION.

GuestToo; My utility: xFileMount mounts sfs files with a click in ROX.
Perfect example of the copy up thing: Samba server has to copy smb.conf to /etc.
Puppy 2 unions extra sfs files at the bottom of the stack, so any changes must be copied up.
The sfs file's whole dir. tree except /usr would be copied to the save layer automatically.
A parent dir. for the tree is a good idea, or as you said, perhaps a tar file.
The menu addons could be in a tar file, but it seems that dirs. are easier.
The sfsManager would write the changes to the menus of JWM, IceWM, etc.
And it'd copy icons to the ROX desktop if a checkbox on it was set to do it.
Attachments
sfsManager.png
This the working sfsManager I made with NathanF's help.
(21.04 KiB) Downloaded 1709 times

DavidBell
Posts: 132
Joined: Fri 24 Nov 2006, 21:44

#9 Post by DavidBell »

Thats the sort of thing I was talking about. Do you have to reboot after running? Actually configuring with something like that followed by reboot wouldn't really be inconvenient because you wouldn't do it very often.

Pity about the limit of five sfs, but again from my novice point of view I think it would be nice if applications were outside of pup_###.sfs - and put in a separate sfs, for arguments sake called apps.sfs.

While this would use up one of the five spots, if you had a tool like yours that let you mix match and merge other sfs's to create your own version of apps.sfs, then it wouldn't matter because you would only ever really need one sfs spot from a user POV. I think it would add a lot of flexibility, people making alternate versions could use it and individuals too.

David

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#10 Post by sunburnt »

David; The sfs files ARE the apps., Xwin is a Linux app. also & in the sfs file.
The sfsManager utility above is for Unioning sfs files, not building them.

For Puppy-2 the sfsManager would show only at bootup, as it can only union then.
Puppy-1 however can swap sfs files in & out at random anytime.
So the sfsManager would be on the Puppy-1 menu/desktop.

An advanced sfs file builder would be nice addon app. for both Puppies!

DavidBell
Posts: 132
Joined: Fri 24 Nov 2006, 21:44

#11 Post by DavidBell »

Yes I think I understand that now. Actually I was looking at Marks post above again and see now he was talking about something very similar to what I was trying to express. I guess when I say move the applications to a separate sfs I really mean just productivity apps like spreadsheets etc, whereas the core applications like X would stay in pup_###.sfs.

In addition to Marks system I am wishing for an easy way to contruct your own (productivity) application suite sfs that you could replace the standard one with. I did download unleashed a while back, I really need to have a look at that as I might be asking for stuff that's already possible, or not difficult with a few modifications. No time this weekend though.

Thanks DB

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#12 Post by sunburnt »

Yep, I was thinking about the advantages of this sfs file setup
for upgrading Puppy or syncing versions as MU pointed out.
A side benefit I hadn't really thought about when I formed the idea.
It's really the same setup the Linux distro. Morphix has,
except Morphix doesn't swap files after booting.
Morphix is Debian & uses cloop (Compressed LOOP ) files instead of Squash files.

User avatar
Gn2
Posts: 943
Joined: Mon 16 Oct 2006, 05:33
Location: virtual - Veni vidi, nihil est adpulerit

#13 Post by Gn2 »

Sunburnt

Yes FWIW, totally agree - an Optical device is a questionable choice :
Longevity, reliability & ease of maintenance/scalability is foremost concerns to server DB use
The prime reason device hot-swapping and one of RAID mirroring solutions is employed for system critical usage

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#14 Post by sunburnt »

Gn2; It seems your answering the post line: LanPuppy HowTo.
Flash suggested a secure DB loaded from DVD.

But hey... what do you think of the Advanced Squash file idea?

User avatar
Gn2
Posts: 943
Joined: Mon 16 Oct 2006, 05:33
Location: virtual - Veni vidi, nihil est adpulerit

#15 Post by Gn2 »

Yes > to both (noticed too late mixup in threads - knew you would do as always - 8) figure it all out. )

Post Reply