Multiple SFS support in initrd

This is where threads concerning the development of the next version of Puppy live.
Message
Author
User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

Multiple SFS support in initrd

#1 Post by mavrothal »

Jemimah introduced in Saluki the ADRV sfs. An SFS that will mount in the union with the main puppy sfs which was used to hold applications. The idea being that could be easily replaced without affecting the core functions.
Her work was never made it into woof, so when Archpup wanted to go that way I adapted jemimah' s code for the more recent initrd that Archpup uses.
Then I took it one step further adding a YDRV, since Archpup people wanted a separate sfs for the Desktop/Window managers.

So here are a couple of patches for Woof2 ver:8fa56fa1c1d5fb12 (today's).
Ones adds the adrv and the other both adrv and ydrv.
Please also notice the additional mount-point folders that should be added in the initrd tree as indicated, as well as the required changes in DISTRO_SPECS.

Obviously the code could be expand to encompass the entire alphabet :P but without the woof changes to build the separate SFSs is not of much use :roll: .
Still I put it here since a modular puppy may not be such a bad idea. Could facilitate building the different versions of a puppy that become all the more often like "retro", "fat", "XFCE" etc, without the need to replace/rebuild the entire ISO.
Attachments
Woof_initrd-tree0_patches.tar.gz
Initrd support for multiple SFS
(3.64 KiB) Downloaded 1333 times
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

D Drive

#2 Post by scsijon »

@mavrothal

I had also looked at Jemimas's way of doing it as I wanted similar with my Development Puppy qtpy, I have added further a d-drive for the Development tools \ apps SFS to mount into as they are quite a size (naturally) and do get regularly updated.

However, I was wondering how \ if we could force some of the apps to be added to the new SFS's instead of the savefile with a 'marker tickbox' in PPM, since there are a few decent sfs packaging apps out there now.

I was \ am also considering how to split the existing puppy system up so that /root lives on it's own partition \ savefile with the intention of having a common /root available across a number of puppy versions. The problem as far as I can see is how to deal with the 'differances' between the /root settings files. But I wonder how different it is nowadays other than the 'set' and 'menu' items.

User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

Re: D Drive

#3 Post by mavrothal »

scsijon wrote:However, I was wondering how \ if we could force some of the apps to be added to the new SFS's instead of the savefile with a 'marker tickbox' in PPM, since there are a few decent sfs packaging apps out there now.
The SFS is read only and you can not add things to it after is made.
However, you can define [a,d,y]drv is DISTRO_SPECS as, say Apps_mypuppy_123.sfs and then let the user or developer make the Apps_mypuppy_123.sfs with apps of their liking. That's what Archpup_13.2 is doing currently.
A down side is that these SFSs will load in RAM (if available) so you may need 1GB+ to take full advantage of this system.
scsijon wrote:I was \ am also considering how to split the existing puppy system up so that /root lives on it's own partition \ savefile with the intention of having a common /root available across a number of puppy versions. The problem as far as I can see is how to deal with the 'differances' between the /root settings files. But I wonder how different it is nowadays other than the 'set' and 'menu' items.
I _think_ that the init script and snapmerge could be modified to use an additional "root_savefile" but I'm not sure if it worths the effort and the potential problems. Also I doubt about the benefits since unless you use very similar pupplets can be a source of problems.
If you have some settings you want to carry from pupplet to pupplet you can make an SFS to "load_on_the_fly" or just a tarball.
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==

User avatar
Q5sys
Posts: 1105
Joined: Thu 11 Dec 2008, 19:49
Contact:

Re: D Drive

#4 Post by Q5sys »

mavrothal wrote:
scsijon wrote:However, I was wondering how \ if we could force some of the apps to be added to the new SFS's instead of the savefile with a 'marker tickbox' in PPM, since there are a few decent sfs packaging apps out there now.
The SFS is read only and you can not add things to it after is made.
However, you can define [a,d,y]drv is DISTRO_SPECS as, say Apps_mypuppy_123.sfs and then let the user or developer make the Apps_mypuppy_123.sfs with apps of their liking. That's what Archpup_13.2 is doing currently.
A down side is that these SFSs will load in RAM (if available) so you may need 1GB+ to take full advantage of this system.
I'm sure someone could hack the PPM to utilize the Edit-SFS package and give you that functionality... but honestly I'd think its a bad thing. One of the benefits of using a layered file system that we do, is if something causes a problem... its easy to reverse. Example, install a PET that uses a newer version of some core lib. If its a pet you can simply uninstall the pet, because the original lib is still in the base SFS package. If you edit the SFS by adding programs through the PPM, if you overwrite a important lib... you've just completely nuked your system.

If there are specific programs you use and know will work, you can always use the Edit-SFS package yourself to change the base SFS. But I think making this an option in the PPM is only going to cause more problems than it'd solve.

mavrothal, do you agree with that assement or do you have another opinion on it?

User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

Re: D Drive

#5 Post by mavrothal »

Q5sys wrote:
mavrothal wrote:
scsijon wrote:However, I was wondering how \ if we could force some of the apps to be added to the new SFS's instead of the savefile with a 'marker tickbox' in PPM, since there are a few decent sfs packaging apps out there now.
The SFS is read only and you can not add things to it after is made.
However, you can define [a,d,y]drv is DISTRO_SPECS as, say Apps_mypuppy_123.sfs and then let the user or developer make the Apps_mypuppy_123.sfs with apps of their liking. That's what Archpup_13.2 is doing currently.
A down side is that these SFSs will load in RAM (if available) so you may need 1GB+ to take full advantage of this system.
I'm sure someone could hack the PPM to utilize the Edit-SFS package and give you that functionality... but honestly I'd think its a bad thing. One of the benefits of using a layered file system that we do, is if something causes a problem... its easy to reverse. Example, install a PET that uses a newer version of some core lib. If its a pet you can simply uninstall the pet, because the original lib is still in the base SFS package. If you edit the SFS by adding programs through the PPM, if you overwrite a important lib... you've just completely nuked your system.

If there are specific programs you use and know will work, you can always use the Edit-SFS package yourself to change the base SFS. But I think making this an option in the PPM is only going to cause more problems than it'd solve.

mavrothal, do you agree with that assement or do you have another opinion on it?
Given the current pet QA I do agree that is not a good idea to mess up the pristine state of the original SFSs.
An option could be to install new pets directly in an "additional_apps.sfs". If anything goes wrong you can always drop it and go to pristine state.
Theoretically after a new pet download/installation your "additional_apps.sfs" expands the new pet is/are added the original sfs gets backed-up and umounted and the new one squashed and mounted (so you have the previous version if something goes wrong)
I still find it unpractical though and I can hardly see the advantages over installing in the savefile.
If you eventually have a set of additional apps that you feel happy with you just remaster.
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==

R-S-H
Posts: 487
Joined: Mon 18 Feb 2013, 12:47

SFS? Yes, yes, yes!

#6 Post by R-S-H »

Hi.

First I have to say: I really like the idea of using SFS files instead of installing applications.

So, I run my LazY Puppy (by now Version 3.0.0, not published yet) from USB flash (and USB hdd) totally in RAM, no save file is used and all programs are coming as SFS file. There are SFS files in sizes of several 100 MB and also some of sizes in less KB - by now there are 334 SFS files in the LazY Puppy boot directory.

All these SFS files are loaded automatically by a RunScript, which can be created with right-click action on SFS files. The RunScripts are executed by a menu entry - just like as the program would have been installed (downloading the SFS directly to the boot directory from server smokey01.com is included).

LazY Puppy can load every SFS file directly at boot up (mounted like the zdrv SFS) just by giving the name of the SFS or by predefined boot menu entries in menu.lst, using predefined constants for the SFS Suites.

There is just one single PET to install for me - if I want to go online with my USB GPRS Modem!

BUT: I have to make sure all Data is saved outside LazY Puppy before shutting down the computer - each and every day.

That's why I have invented the Personal Data SFS in LazY Puppy 3.0.0. This can be loaded automatically at boot up (by boot option in menu.lst) and can also be saved automatically before shutting down the computer - selectable option in the shutdown-gui of LazY Puppy. If I'm saving the Personal Data SFS at shut down, an existing Personal Data SFS is renamed (date is added to the file name) before that!

Just by a right-click I can set files and directories to be added to the Personal Data SFS or to be removed from the Personal Data SFS.

Everything is temporary until I save the Personal Data SFS manually or at shut down.

This is my vision of Puppy and SFS!

And it's become REAL!

RSH

User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

Re: SFS? Yes, yes, yes!

#7 Post by mavrothal »

R-S-H wrote: This is my vision of Puppy and SFS!
And it's become REAL!
How is that different/better from tinycorelinux?

The idea of puppies with 2-3 distinct SFSs instead of the usual monolithic one, has to do only with the ease of development/customization of a complete system/ISO.
Not with the philosophy of the puppy setup.

As mentioned, there are other distros to do that.

(sorry for the "loud" mark up. Just try to be clear)
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==

R-S-H
Posts: 487
Joined: Mon 18 Feb 2013, 12:47

#8 Post by R-S-H »

How is that different/better from tinycore linux?
I do not know Tiny Core Linux and I did never ever see one running or read any specific information about Tiny Core.

What I've found now by your presented links is totally new to me - however: I don't believe, that Tiny Core uses RunScripts to be able to do the following:

Clicking a GIMP Image (Gimp is not installed and there is no SFS file at local computer available), automatically downloading the SFS into the boot directory, automatically loading the SFS after download, automatically starting GIMP application and automatically loading the GIMP Image into GIMP - ready to edit by just a single click on a menu entry.

Tiny Core surely can NOT do this.

Sorry, but that's all and that's much more as I've ever seen in any Linux (there are not much, but some)
As mentioned, there are other distros to do that.
No!

They might do something similar, but not exactly the same!

R-S-H
Posts: 487
Joined: Mon 18 Feb 2013, 12:47

#9 Post by R-S-H »

Thanks to this thread I've discovered and examined the adrv option of Saluki. Found it very useful and surely will try to add such feature to LazY Puppy. I've had a quick look at the init script in Saluki's initrd.gz - seems not to be too much code. :)
[b][url=http://lazy-puppy.weebly.com]LazY Puppy Home
The new LazY Puppy Information Centre[/url][/b]

User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

#10 Post by mavrothal »

R-S-H wrote: Clicking a GIMP Image (Gimp is not installed and there is no SFS file at local computer available), automatically downloading the SFS into the boot directory, automatically loading the SFS after download, automatically starting GIMP application and automatically loading the GIMP Image into GIMP - ready to edit by just a single click on a menu entry.
This can be handy at times.
The reason that distros do not do that (ie download an SFS or package) for a missing MIME type/file extension, is that usually more than 1 app can handle a file.
Some distros (not tinycore) do provide a list of suggestions if a file can not be handle by the installed applications
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==

R-S-H
Posts: 487
Joined: Mon 18 Feb 2013, 12:47

#11 Post by R-S-H »

Some distros (not tinycore) do provide a list of suggestions if a file can not be handle by the installed applications
Yes, this might be also a nice feature in LazY Puppy. :)

However...

I've tried to put in the "adrv"-option into LazY Puppy using the code taken from the Saluki initrd.gz. I did get the adrv sfs mounted in pup_a. All files where in the directory and I was able to open f.e. scripts in geany.

BUT, the sfs seems not to be mounted like to have the menu entries and to be able to execute the applications. :cry:

To me it seems like it is mounted to be able to grab some files off it, but not mounted to the "layered filesystem".

What am I doing wrong?

I did try this also using the exactly code for the zdrv modified to adrv - didn't work also.

Is there anything else I would have to do to get the adrv mounted like the main sfs is mounted (to be able to execute its applications from menu)?

Thanks

P.S.

No, "fixmenus" did not "the job" :lol:
[b][url=http://lazy-puppy.weebly.com]LazY Puppy Home
The new LazY Puppy Information Centre[/url][/b]

User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

#12 Post by mavrothal »

R-S-H wrote: I've tried to put in the "adrv"-option into LazY Puppy using the code taken from the Saluki initrd.gz. I did get the adrv sfs mounted in pup_a. All files where in the directory and I was able to open f.e. scripts in geany.

BUT, the sfs seems not to be mounted like to have the menu entries and to be able to execute the applications. :cry:

To me it seems like it is mounted to be able to grab some files off it, but not mounted to the "layered filesystem".

What am I doing wrong?

I did try this also using the exactly code for the zdrv modified to adrv - didn't work also.

Is there anything else I would have to do to get the adrv mounted like the main sfs is mounted (to be able to execute its applications from menu)?
Besides the changes in initrd/init you also need to have the mount points (folders) in initrd. Did you?
Did you also added the adrv to DISTRO_SPECS
If you want post or PM all the changes you have made in initrd, to take a look, (or compare with the patches above. There is both the adrv only and the adrv+ydrv patches. Did you look at them?...)
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==

R-S-H
Posts: 487
Joined: Mon 18 Feb 2013, 12:47

#13 Post by R-S-H »

Yes, I did everything. pup_a is in initrd.gz.

The 'adrv' directory is created by init script - like the zdrv is.

I did not add the adrv as DISTRO_ADRVSFS to DISTRO_SPECS - I do submit this from the grub bootmenu. The file is then in pup_a, I can see the files and I can open scripts in geany, so it is successfully submitted, but the .desktop files do not appear in the menu and none of the program runs.

No, I did not have a look into the above attached patches, because I'm not in woof. I just do remaster.

Should I?
Attachments
image-2.jpg
Content of initrd.gz
(101.1 KiB) Downloaded 2537 times
Last edited by R-S-H on Fri 08 Mar 2013, 01:00, edited 1 time in total.
[b][url=http://lazy-puppy.weebly.com]LazY Puppy Home
The new LazY Puppy Information Centre[/url][/b]

User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

#14 Post by mavrothal »

R-S-H wrote:Yes, I did everything. pup_a is in initrd.gz.

The 'adrv' directory is created by init script - like the zdrv is

I did not add the DISTRO_ZDRVSFS to DISTRO_SPECS - I do submit this from the grub bootmenu. The file is then in pup_a, I can see the files and I can open scripts in geany, so it is successfully submitted, but the .desktop files do not appear in the menu and none of the program runs.

I'm afraid descriptions are not gong to do it.
You may want to post your init and the contents of your /initrd/tmp folder to see what are you doing exactly, during boot.
Also /tmp/xerrs.log can be informative (or maybe just run pdiag)
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==

R-S-H
Posts: 487
Joined: Mon 18 Feb 2013, 12:47

#15 Post by R-S-H »

Here is the init script from my LazY Puppy 3. Saluki code included - commented out, because it failed and I did try it with an code-copy of the zdrv code. I have made a lot of modifications also, so it's a bit more than a usual init script.

Script removed!
Last edited by R-S-H on Sun 10 Mar 2013, 20:38, edited 1 time in total.
[b][url=http://lazy-puppy.weebly.com]LazY Puppy Home
The new LazY Puppy Information Centre[/url][/b]

User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

#16 Post by mavrothal »

R-S-H wrote:Here is the init script from my LazY Puppy 3. Saluki code included - commented out, because it failed and I did try it with an code-copy of the zdrv code. I have made a lot of modifications also, so it's a bit more than a usual init script.
"a bit more"?...That's a lot to shift through!

I think that the crucial part you miss is the definition of the layers (lines872-888) and thus the union mounting at 1778.

Look at the ADRV_Woof.patch in the download above. should be clear the part you are missing (and is much smaller than your init too :-P)
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==

R-S-H
Posts: 487
Joined: Mon 18 Feb 2013, 12:47

#17 Post by R-S-H »

Hi, mavrothal
I think that the crucial part you miss is the definition of the layers (lines872-888) and thus the union mounting at 1778.
I did try two times.

First, after doing all needed changes, I forgot to enable the saluki adrv code again and to remove the modified zdrv code - so, I've booted with all changes made and ended up into kernel panic... :lol:

Second go, I've enabled the saluki adrv code again and removed the other one.

Result:

I'm currently posting from LazY Puppy 3.0.0 (only openbox is installed) using the LP2_WindowManagers.sfs for the adrv mount point plus the LP2_XorgHigh.sfs for the zdrv mount point - I use this for the LazY Puppy Extension.sfs (but can be any sfs I want).

FbPanel is there right after X is started and Stellarium runs out of the box (because of the XorgHigh).

I do this from a boot menu entry in menu.lst

Code: Select all

title LazY Puppy 3.0.0 Standard (RAM, ADRV-Test, Fenstermanager, +XorgHigh)
  find --set-root --ignore-floppies /LazY/initrd.gz
  kernel /LazY/vmlinuz lpextsfs=LP2_XorgHigh.sfs lpadrvsfs=LP2_WindowManagers.sfs psubdir=LazY pmedia=atahd pfix=ram,fsck
  initrd /LazY/initrd.gz
Thank you very much, mavrothal! Really!

But, be aware: my lovely Augenstern will not be impressed, because second result ends up in "waiting for me until sunday" - for her... :wink:

I assume, I would have to do this exactly again and again for each a-, b-, cdrv I want to have. At this point I have changed something like this:

Code: Select all

OLDFILESMNTPT="/pup_ro1";NEWFILESMNTPT="/pup_ro1";UMNTMAIN="/pup_rw=rw:/pup_ro1=ro+wh";;
to this

Code: Select all

OLDFILESMNTPT="/pup_ro1";NEWFILESMNTPT="/pup_ro1";UMNTMAIN="/pup_rw=rw:";UMNTRO0="/pup_ro1=ro+wh";;
almost inserting

Code: Select all

UMNTRO0
and

Code: Select all

UMNTRO1
The other part is changing this

Code: Select all

mount -t aufs -o udba=reval,diropq=w,dirs=${UMNTMAIN}${ZLAYER}${UMNTRO} unionfs /pup_new

mount -t unionfs -o dirs=${UMNTMAIN}${ZLAYER}${UMNTRO} unionfs /pup_new
to this

Code: Select all

 mount -t aufs -o udba=reval,diropq=w,dirs=${UMNTMAIN}${UMNTRO0}${ALAYER}${UMNTRO1}${ZLAYER}${UMNTRO} unionfs /pup_new

 mount -t unionfs -o dirs=${UMNTMAIN}${UMNTRO0}${ALAYER}${UMNTRO1}${ZLAYER}${UMNTRO} unionfs /pup_new
Just inserting this code

Code: Select all

${UMNTRO0}${ALAYER}${UMNTRO1}$
But I can't see the system behind this!

Could you please give me something like a "form" or a "path" how to change and/or to add severals "adrv" mount points. I want to have at least: adrv (admin), ldrv (language), wdrv (WindowManagers), rdrv (RunScrips etc.), xdrv (extensions) - I think there will be even more. :lol:

Thanks again, 8)

RSH
[b][url=http://lazy-puppy.weebly.com]LazY Puppy Home
The new LazY Puppy Information Centre[/url][/b]

R-S-H
Posts: 487
Joined: Mon 18 Feb 2013, 12:47

#18 Post by R-S-H »

Addition to the post above...

I've had a look into the above attached .tar.gz file. Patch file ADRV-YDRV_Woof did let me get a bit more into this. Although I did not change UMNTMAIN to UMNTRW it works. Do I have to change this also or is it because of newer woof or newer puppies (kernels)?

And, oh, by the way: where is this mounted - I mean in which order at the pup_roX list (ro1, ro2, ro3 etc.)?
[b][url=http://lazy-puppy.weebly.com]LazY Puppy Home
The new LazY Puppy Information Centre[/url][/b]

R-S-H
Posts: 487
Joined: Mon 18 Feb 2013, 12:47

#19 Post by R-S-H »

Oh, that's interesting!

To me it seems like the adrv is mounted on top of the main sfs file. There's a image for the fbpanel menu in main sfs. But the menu shows the image from LP2_WindowManagers.sfs!

This might make possible to boot using settings from sfs that would "overwrite" existing files from main sfs ? ! ? ! ? !

Or is it only because i did not change UMNTMAIN to UMNTRW?

Hm...

I know, it's late... ...though, anyone here, with knowledge on such issues?

EDIT:

Yes!

Copying the image from /initrd/pup_ro2/usr/share/fbpanel/images/start-button.png to /usr/share/fbpanel/images/start-button.png shows the image in fbpanel menu (start button)...

So, is this 8) or is it :? ???
[b][url=http://lazy-puppy.weebly.com]LazY Puppy Home
The new LazY Puppy Information Centre[/url][/b]

R-S-H
Posts: 487
Joined: Mon 18 Feb 2013, 12:47

#20 Post by R-S-H »

Ooohhh, this is working fun - or funny working!

Lazy Puppy 3.0.0 booting to RAM, no save file

- adrv = LP2_WindowManagers.sfs
- ddrv = LP2_AudioStudio.sfs --> for testings

- zdrv = LP2_XorgHigh.sfs

FbPanel is there, Stellarium runs out of the box and QJackCtl from LP2_AudioStudio.sfs runs as well!

Next stop: fdrv --> for fonts

---

The icon from fbpanel is still on top...
[b][url=http://lazy-puppy.weebly.com]LazY Puppy Home
The new LazY Puppy Information Centre[/url][/b]

Post Reply