Another re-write of the "init" script - using OverlayFs?

[Keef]

@peebee

I used UUID and it works for me.

Code: Select all

title tahrpup
  uuid ba259b38-29a0-40e7-84bc-032f1d46204b
  kernel /tahr/vmlinuz psubdir=tahr pmedia=atahd  pdrv=sda2:/tahr/ zdrv=sda2:/tahr/zdrv_tahr_6.0.6.sfs
  initrd /tahr/initrd.gz

[peebee]

Sorry - found my error - I'd put the wrong "version" of DISTRO_SPECS into the frugal subdirectory - I'd used the version which goes into the woof-ce process rather than the output of woof-ce and therefore does not include the *drv specifications.....

With the correct DISTRO_SPECS, boots as expected!

[gyro]

@Keef,
You shouldn't need the "zdrv=" parameter.
I'm successfully booting xenial 7.0.8.1 with this boot entry:

Code: Select all

title Puppy xenial 7.0.8.1 (sdc2/xenial)
  uuid e9e64b47-ccd4-42c5-aeff-e771e264af3c
  kernel /puppy/xenial/vmlinuz acpi_osi=Linux libata.noacpi=1 intel_pstate=disable pdrv=isoSave:/puppy/xenial/ pfix=nosave
  initrd /puppy/xenial/initrd.gz

And it picks up my ydrv and my zdrv.

@peebee,
Great, it now works for you.

Note: I using xenial 7.0.8.1 as my testbed because it has overlays enabled in it's 4.9.13 kernel. And now I'm playing with overlays.

gyro

[gyro]

I've done it.
The new version I've just uploaded to http://www.fishprogs.software/puppy/initrd/init and http://www.fishprogs.software/puppy/initrd/initrd.gz uses overlays to create the stack instead of aufs.

I used xenial 7.0.8.1 as my test bed, since it's kernel has overlay support as a module.

A normal first boot will boot like a normal pupmode=5 with sfs's copied to ram and the rw layer in tmpfs.
It you add a "pfix=nocopy" boot parameter and the frugal install directory is on a linux partition, it will create and use a savefolder for rw. On second boot it will do a pupmode=12 type boot with the savefolder as the rw layer.

Note: If you see the normal first shutdown config, always click on the "No" button.

Of course this is just first step, a proof of concept, there are lot's of things it can't do.
And a significant amount of effort to sortout exactly what it can do.

Observations:
It seems rather slow at shutting down.
I wonder how much of the rest of puppy has difficulties when booted with this init because aufs is assumed?

gyro

[gyro]

I've just uploaded a couple of files that I find useful with this init.

http://www.fishprogs.software/puppy/ini ... downconfig is a patched "/ust/sbin/shutdownconfig" that honours "pfix=nosave" by doing absolutely nothing.

http://www.fishprogs.software/puppy/initrd/TIME_ZONE is an example file that enables "init" to set the time zone for itself and for the booted puppy. It has to be "/TIME_ZONE" in the "initrd.gz".
The TZ part before the ':' sets the time for "init", and the part after the ':' sets the time for the booted puppy.
Can completely remove all "Time of last mount...." error messages from fsck.

gyro

[peebee]

Just to report that the initrd.gz of 9-jun used together with the xenial-7.0.8.1 kernel 4.9.13 booted fine for me.....
also renamed my chromium .sfs to an adrv and that was loaded correctly.
Cheers
peebee

[gyro]

@peebee,
I've finally worked out why your deficient DISTRO_SPECS resulted in the misleading error message it did.
The function that deciphers the "pdrv=" boot parameter, does nothing if there's no default filename provided, thus ignoring any valid information that might be provided in the "pdrv=" parameter.
I'll try to fix this in the next version.

gyro

[gyro]

Thanks to all the folk who have taken the time to download and try this stuff, I do appreciate it.
gyro

[gyro]

I've uploaded new versions to http://www.fishprogs.software/puppy/initrd/init and http://www.fishprogs.software/puppy/initrd/initrd.gz.

Changes in this 11 June version:
1. Improved error reporting, particularly in the early "sorting out boot parameters" phase.
2. Interpretation of "pdrv=sdb2:dir/filename.sfs" has changed from "sdb2:/dir/filename.sfs" to "sdb2:/psubdir/dir/filename"
3. Checks for the existence of the "overlay" module, and error exits if it's not found.
4. Writes a "POVERLAY='yes'" line to PUPSTATE, if overlay support is found.
5. Builds the stack in 2 phases, first an ro sfss stack, mounted on "/pup_sfss" and then the full rw stack at "/pup_new".

I've also uploaded a couple of scripts I use a lot when working with "initrd.gz":
http://www.fishprogs.software/puppy/initrd/file2initrd and http://www.fishprogs.software/puppy/initrd/initrd2file.
Just execute them without any parameters to see the "Usage" info.

gyro

[Moose On The Loose]

I've uploaded new versions to http://www.fishprogs.software/puppy/initrd/init and http://www.fishprogs.software/puppy/initrd/initrd.gz.

Changes in this 11 June version:
1. Improved error reporting, particularly in the early "sorting out boot parameters" phase.
2. Interpretation of "pdrv=sdb2:dir/filename.sfs" has changed from "sdb2:/dir/filename.sfs" to "sdb2:/psubdir/dir/filename"
3. Checks for the existence of the "overlay" module, and error exits if it's not found.
4. Writes a "POVERLAY='yes'" line to PUPSTATE, if overlay support is found.
5. Builds the stack in 2 phases, first an ro sfss stack, mounted on "/pup_sfss" and then the full rw stack at "/pup_new".

I've also uploaded a couple of scripts I use a lot when working with "initrd.gz":
http://www.fishprogs.software/puppy/initrd/file2initrd and http://www.fishprogs.software/puppy/initrd/initrd2file.
Just execute them without any parameters to see the "Usage" info.

gyro

I haven't been watching closely but had an idea
Has the loading of an SFS on the fly been made possible yet?
If not perhaps a "fast reboot" could be added that doesn't reboot but instead pivots off the overlayed file system makes changes and goes back onto the modified overlayed system. This would be a lot faster than a complete reboot.

[gyro]

Has the loading of an SFS on the fly been made possible yet?

No, I have still not found a way to modify an existing overlay stack, but testing of this is not yet complete.

If not perhaps a "fast reboot" could be added that doesn't reboot but instead pivots off the overlayed file system makes changes and goes back onto the modified overlayed system. This would be a lot faster than a complete reboot.

Or could try building a new "fixed" stack and pivot root to that.

gyro

[Rangan Masti]

Wish you success guys.

[Moose On The Loose]

Has the loading of an SFS on the fly been made possible yet?

No, I have still not found a way to modify an existing overlay stack, but testing of this is not yet complete.

If not perhaps a "fast reboot" could be added that doesn't reboot but instead pivots off the overlayed file system makes changes and goes back onto the modified overlayed system. This would be a lot faster than a complete reboot.

Or could try building a new "fixed" stack and pivot root to that.

gyro

Yes, if a modified stack can be done, it seems like a faster way to get to a changed system. It may take some trickery to make the changes up to that point stay but not override the new SFS.

[gyro]

When I tried to change the list of ro directories in an overlay with a "remount", there were no error messages, but the overlay remained unchanged. So this seems to be a dead-end.

But I was able to create a new overlay using some directories from the old overlay (including the rw dir, "upperdir" and "workdir") + a new ro directory. Then unmounting the old overlay and moving the new overlay to the original mount point. This worked.
Although this was just using play directories, not the actual / overlay.
So there is still some hope of being able to create a new overlay at /pup_new and doing some sort of pivot root to /pup_new, (again).

Unfortunately, it is probably safer to start by returning to the ancient puppy method of building the whole stack in "init" and not modifying it at all.

I've uploaded a new version of the "init" and "initrd.gz".
This one creates an ro overlay containing just the sfs's at /pup_sfss, and after sorting out what directory to use as an rw directory, creates the full overlay on /pup_new, using /pup_sfss as the "lowerdir".
This restores the logic of dealing with the sfs's first, then sorting out the save layer.
This also means that in the running system /initrd/pup_sfss contains the files of all the sfs's.

This version is also the first one that writes a PUPSTATE where the variables that are written are fully compatible with current puppies.

gyro

[Rangan Masti]

Hi Good morning, where to find the new initrd uploaded.for testing?
Thank you and have a great day!

[gyro]

where to find the new initrd uploaded.for testing?

The url's of all the files I've uploaded for this project are now provided in the 1st post.
But here's the one for the "initrd.gz" any way, http://www.fishprogs.software/puppy/initrd/initrd.gz.
Note: If the kernel in your puppy does not support "overlayfs", "init" will fail.

gyro

[gyro]

I've had no success with trying to modify an overlay that is mounted as /, so I'm moving on to try some save layer techniques.

I've uploaded http://www.fishprogs.software/puppy/ini ... verlay.sfs, to help with testing. It currently only includes "shutdowconfig" but in the next version it will also include "rc.shutdown".
And I've added some information on how to use the files, to the first post.

This version can boot automatically to pupmode=12 using a savefolder.
Just boot with boot parameters similar to the following:

Code: Select all

pdrv=sdc2:/slacko/ ydrv=:ydrv_overlay.sfs pfix=nocopy

If you already have a ydrv but no adrv you could specify "adrv=:ydrv_overlay.sfs" instead.
It should boot as pupmode=5 for the first boot. On first shutdown there should be no "shutdownconfig".
If your frugal install directory is on a Linux partition, a reboot should result in pupmode=12.

There is no logical reason why the mount point of a savefile could not be used as the rw directory of the overlay instead of a savefolder.

Because there is no way to modify an overlay mounted as /, and modifying any ro directory in an overlay leads to an "undefined" result, pupmode=13 simply cannot work. So I'm working on a pseudo pupmode=13.

gyro

[Rangan Masti]

Hi, good afternoon. Testing initrd with my compiled kernel 4.12.0-rc6 with
built-in overlayfs.Thank you for the link last sent. Have a good day.

[gyro]

Hi, good afternoon. Testing initrd with my compiled kernel 4.12.0-rc6 with
built-in overlayfs.Thank you for the link last sent. Have a good day.

You're welcome, and thanks for testing.
Just for information, overlayfs support can be builtin or a module; if it's a module, the "init" script will "insmod" it.

Just one question, does your kernel have aufs support?
This should work without aufs support, but the puppy kernel's I use for testing all include aufs support. And it's on my todo list to test it using a kernel that doesn't have aufs support. So if it works with your kernel and it doesn't have aufs support, I can cross that test off my list.

gyro

[gyro]

Yes, if a modified stack can be done, it seems like a faster way to get to a changed system. It may take some trickery to make the changes up to that point stay but not override the new SFS.

There is no problem creating a new overlay stack that includes the current rw directory and a different set of ro directories. The problem is getting that new overlay stack to replace the current /, in a running system.
So, I'm putting this into the too hard basket for now.

gyro