Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sun 20 May 2018, 18:06
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
Another re-write of the "init" script - using OverlayFs?
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 12 of 12 [173 Posts]   Goto page: Previous 1, 2, 3, ..., 10, 11, 12
Author Message

Joined: 28 Oct 2008
Posts: 1498
Location: Brisbane, Australia

PostPosted: Thu 19 Apr 2018, 05:47    Post subject:  

A simple fix to "install2usb", change:
printf "1\ny\n"| fsck.vfat $FIXFATPART
printf "1\ny\n"| fsck.vfat -r $FIXFATPART
"-r" is the default in recent versions of "fsck.vfat" but not in older versions.
I tested "install2usb" on Tahr 6.0.6 and the vfat partition was not fixed, until I made this change.
The changed code still works in recent versions.

Back to top
View user's profile Send private message 

Joined: 28 Oct 2008
Posts: 1498
Location: Brisbane, Australia

PostPosted: Fri 20 Apr 2018, 01:59    Post subject: Indicating that vmlinuz is in differnt partition  

I'm having second thoughts about how installers like "install2usb" should indicate that "vmlinuz" is on a partition other than the one containing the frugal install directory with the sfs files.
The problem is that while the boot parameter method works fine in the running target Puppy, it does not help a script that's running on a host Puppy, but trying to update the frugal install directory of a target Puppy, e.g. "overlay-setup-frugal".

My alternative would be for the installer to write a file called "pbootlocation.txt" (or "puppyboot.txt") in the frugal install directory alongside the sfs files.
This file would contain something like the following:
Where 'VFAT' is the label assigned to the vfat partition, and '/xenialpup-7.5' is the sub-directory of that partition containing "vmlinuz".

This approach should work for any script processing the frugal install directory, simplifies the code in the installer, and does not involve changing "init" or "PUPSTATE".

The aim of either approach is to support not only the "install2usb" script, but also setups where the the boot partition is a usb device but the Puppy frugal install directories reside on a different device, e.g. a Linux partition on an internal HD.

Back to top
View user's profile Send private message 

Joined: 28 Oct 2008
Posts: 1498
Location: Brisbane, Australia

PostPosted: Fri 20 Apr 2018, 23:41    Post subject: "vmlinuz" not in frugal install directory issue  

"vmlinuz" not in frugal install directory issue:
There is a simpler way.

"install2usb" doesn't do anything about the the issue.
"change_kernels" looks for "vmlinuz" in the frugal install directory, if it doesn't find it, it opens a "select file" dialog, so the user can locate it.
Silimarly, if it doesn't find a "zdrv...sfs" in the frugal install, it opens a "select file" dialog so the user can locate it.

This approach can even cope with the situation where the user has "specified" the "zdrv...sfs" to be a file with a different name, e.g. "/mnt/sdc7/kernels/kernel_modules-4.9.92.sfs".

But if the frugal install directory is "normal", then things proceed as before.

Back to top
View user's profile Send private message 

Joined: 28 Oct 2008
Posts: 1498
Location: Brisbane, Australia

PostPosted: Wed 25 Apr 2018, 20:44    Post subject:  

I have attached:

"overlay_installers-0.9.sfs", which includes my latest hack of 01micko's "install2usb", and "overlay-setup-frugal".
This "install2usb" does nothing about the "vmlinuz" in a different partition issue.

"change_kernels" the corresponding hack of this script.
If it can't find "vmlinuz" in the frugal install directory and it's a usb device, it also looks in PSUBDIR on the 1 partition of the device.
Otherwise it opens a file selction dialog for the user to specify the location of the appropriate "vmlinuz".

"install2usb9.diff" my patch to "install2usb".

"change_kernels9.diff" my patch to "change_kernels"

Description  gunzip to produce "overlay_installers-0.9.sfs"

Filename  overlay_installers-0.9.sfs.gz 
Filesize  9.86 KB 
Downloaded  15 Time(s) 
Description  gunzip to produce "change_kernels" script

Filename  change_kernels.gz 
Filesize  4.53 KB 
Downloaded  15 Time(s) 
Description  gunzip to produce "install2usb9.diff"

Filename  install2usb9.diff.gz 
Filesize  2 KB 
Downloaded  14 Time(s) 
Description  gunzip to produce "change_kernels9.diff"

Filename  change_kernels9.diff.gz 
Filesize  2.04 KB 
Downloaded  15 Time(s) 
Back to top
View user's profile Send private message 

Joined: 28 Oct 2008
Posts: 1498
Location: Brisbane, Australia

PostPosted: Wed 25 Apr 2018, 20:47    Post subject: Release of overlay_init-0.9  

Release of overlay_init-0.9

The version jump to 0.9 is to indicate that this project is almost done.
I don't expect the "init" script to get any more significant changes.
The utilities are now pretty much functionally complete.

Unfortunately this project has become a little muddied by some of the existing Puppy files that it replaces, changing in woof-ce 'testing'.
To ensure that the patched version in this project properly reflects the version in the parent Puppy, there are multiple releases, each for a particular Puppy.
This issue will be resolved when/if I can get compatible versions of these files into woof-ce 'testing', and then they can be removed from "overlay_init-1.0.tar".

for artfulpup-17.11+16 http://www.fishprogs.software/puppy/initrd/overlay_init_artful-0.9.tar.
for LxPupSc-18.04+3T-k64 http://www.fishprogs.software/puppy/initrd/overlay_init_lxpupsc-0.9.tar.
for slacko-700-RC3_k4.4.70-PAE_2018-04-05_d553a8c-patched http://www.fishprogs.software/puppy/initrd/overlay_init_slacko-0.9.tar.
for slacko64- http://www.fishprogs.software/puppy/initrd/overlay_init_slacko64-0.9.tar.
for Upup_Bionic_Beaver-18.04+7 http://www.fishprogs.software/puppy/initrd/overlay_init_upupbb-0.9.tar.
for xenialpup-7.5-uefi http://www.fishprogs.software/puppy/initrd/overlay_init_xenial-0.9.tar.

for artfulpup-17.11+16.iso http://www.fishprogs.software/puppy/initrd/artfulpup-17.11+16.iso___artfulpup_overlay0.9-17.11+16.iso.delta.
for LxPupSc-18.04+3T-k64.iso http://www.fishprogs.software/puppy/initrd/LxPupSc-18.04+3T-k64.iso___LxPupSc_overlay0.9-18.04+3T-k64.iso.delta.
for slacko-700-RC3_k4.4.70-PAE_2018-04-05_d553a8c-patched.iso http://www.fishprogs.software/puppy/initrd/slacko-700-RC3_k4.4.70-PAE_2018-04-05_d553a8c-patched.iso___slacko_overlay0.9-700-RC3_k4.4.70-PAE_2018-04-05_d553a8c-patched.iso.delta.
for slacko64- http://www.fishprogs.software/puppy/initrd/slacko64-
for Upup_Bionic_Beaver-18.04+7.iso http://www.fishprogs.software/puppy/initrd/Upup_Bionic_Beaver-18.04+7.iso___Upup_Bionic_Beaver_overlay0.9-18.04+7.iso.delta.
for xenialpup-7.5-uefi.iso http://www.fishprogs.software/puppy/initrd/xenialpup-7.5-uefi.iso___xenialpup_overlay0.9-7.5-uefi.iso.delta.

1. Simplified the "Squash" facility in "Overlay init setup".
This used to consist of a "split" process on shutdown, and a "purge" process after the next boot.
The "split" process has been optimised so that the gain from a "purge" process is not worth the complication, so the "purge" process had been eliminated.
The "Squash" process used to produce an update by "rc.update", this is now skipped.
2. Re-versioned all the untilities to 0.9
3. Removed "/usr/sbin/bootmanager" from most versions because the modification is now part of woof-ce 'testing'.
4. Removed support for "pbootpart" and "pbootdir" boot parameters from "init" script.
5. Renamed "OVERLAY_SPECS" to "initrdBOOTspecs".
6. Fixed bug, multisession DVD not saving when using "Archive" save mechanism.

1. Download a delta file for a Puppy iso file that you already have, and use it to produce an "overlay" iso file.
Do a manual frugal install using this "overlay" iso,
install it on a usb stick using 01micko's "install2usb", see "overlay_installers-0.9.sfs.gz" in a previous post.
if you don't require UEFI boot, use rcrsn51's ISObooter, but answer "n" in response to it's "splash screen" question.
2. Download one of the tar files into a fresh frugal install of the appropriate Puppy,
run "overlay-setup-frugal", see "overlay_installers-0.9.sfs.gz" in a previous post.

Review, what's so interesting about this package:

1. It uses "overlayfs" not "aufs" to build the union stack.
This is suppoosed to be good because "overlayfs" is in the kernel, and so has a future,
"aufs" is not, and is never likely to be, so some predict that it will die.
"overlayfs" is supposed to be efficient, since all reading and writing of files goes directly to the actual files.

2. It contains a new facility called "bootspecs", this consists of a file that creates a mechanism for the running Puppy to communicate with the "init" script.
This means that the "init" script can more truly be said to "only do what it's told".
This facility includes scripts that make it very easy for any utility in the running system, to communicate with "init", e.g. "initmodules".

3. The complicated boot parameters for specifying what system sfs files to load, are gone.
Instead, in the running Puppy, the "Overlay init setup" GUI utility is used to "slelect" the sfs files to load and their order in 2 system sfs lists.
The "ABOVE" list (defult adrv,ydrv) that gets mounted in the stack above the "MAIN" list (default puppy,fdrv,zdrv).
These lists can contain virtually any number of sfs files with any name and with any location.
The sfs lists facility relies on the "bootspecs" facility.

4. While there is support for "psavepart" (or it's alias "psave") and "psavedir" boot parameters, these are only very rarely needed.
The "saveconfig" utility, (accessed via "Overlay init setup") can be run at any time to "select" the save mode and "select" the location where the save is stored.
So you can copy your savefolder to a different location, or change your save mechanism from "Folder" to "Archive".
The "saveconfig" utility is run automatically on first shutdown.
On first shutdown, if your are not certain that you have a suitable Linux partition for your savefolder, simply choose the "Archive" save mechanism.
When you have sorted your Linux partition, run "Overlay init setup", "Saveconfig", choose "Folder" and "Select" a directory in your Linux partition.
Again, the relevant information is communicated to "init" via the "bootspecs" facility.
And when you choose a savefolder, you always get a proper pupmode=12, never a fiddly pupmode=13.

5. To avoid any chance that Puppy will boot a different copy of the same Puppy version,
the "psfspart" boot parameter (or one of it's aliases, "pdev1", "pupsfs", "pdrv") is required.
And if the Puppy sfs files are stored in a sub-directory of the defined partition, "psfsdir" (or it's alias "psubdir") is required.
Note, that it does not care where the "vmlinuz" and "initrd.gz" files reside, these are already loaded into memory before "init" starts.
The only exception is if you are booting an iso or DVD, i.e. with "pmedia=cd" in which case the "iso9660" partition is chosen.

6. The "saveconfig" utility supports the creation and use of a Luks encrypted partition to contain savefolders.
Instead of having a Luks savefile for a single Puppy, you have a real Luks partition, that can be used to contain the savefolders of many Puppies.

7. The "Archive" save mechanism provides a means of having your cake and eating it too,
i.e having Puppy running totally in RAM, but having changes saved on shutdown and restored on boot.
Good for a limited use Puppy on a cheap usb stick.

8. Multisession DVD support has been enhanced to include the facilities provided by "Overlay init setup", including the selection of sfs file.
So, it's possible to "upgrade" a multisession DVD by writing a replacement sfs file to it,
and then using "Overlay init setup" to select the new sfs file instead of the old sfs file in the appropriate list.
Or you can simply select sfs files located anywhere on a HD, or select a savefolder location on a HD.
The "Archive" save mechanism is implemented for a multisession DVD, each archive file is appended to the DVD at shutdown.

9. "Overlay init setup" also contains a facility to specify some of the "pfix=" boot flags.
This includes a "nosave" facility that works with all save mechanisms, including what we are used to calling "pupmode=12".

10. The "Extra-SFS manager" supports the "Extra" sfs file list, that gets loaded into the stack below the "MAIN" list.
Again, you can select almost any sfs file, to be included in this list.
Unfortunately, extra sfs files are not loaded immediately, they are loaded by "init" because with "overlayfs", the stack cannot be modified after it is mounted.

11. Replacing an sfs file with another of the same name, now leads to "rc.update" displaying the dreaded "next boot will be faster" message.
The information passed from "init" to "rc.update" has been enhanced to include a modification time stamp.
The updating mechanism in "rc.update" has also been enhanced to be a lot more selective to do only what the contents of the "changed" sfs file suggests is needed.

12. Data security is enhanced by "init" always preforming an fsck on a partition before it's first mount.
This is the default behaviour, it can be disabled by booting with "pfix=nofsckp".

13. The timezone that applies during "init" and the subsequent running system can now be specified in "initrd.gz".
When an "overlay_init" Puppy is installed to a usb stick via the "install2usb" script, this is done automatically.
It is also done automatically when "overlay-setup-frugal" is used to convert a "normal" Puppy to an "overlay" Puppy.

14. A read-only stack is mounted as "/initrd/pup_sfs", this contains all the sfs's.
So it's easy to determine if a file exists in any of the loaded sfs's, and if so, read it.

Limitations imposed by "overlayfs":

1. Layers can neither be added to, nor removed from, an "overlayfs" stack.
This means the "sfs_load" as we know it, cannot be supported.

2. The directories that makeup the "overlayfs" stack, cannot be modified directly.
This means there can be no "snapmergepuppy", nor can "PPM" install directly tp "pup_ro1".

3. A read/write "overlayfs" stack, requires 2 directories on the same filesystem for it's top layer.
I have implemented this requirement by always having a "work" directory beside the "save" directory.
This means we can't use a mount point as the top directory, it would have to be 2 directories inside the mount point directory.
So, a "savefile" mechanism could not be implemented in the same manner as current.
Also using a partition as a save layer is not possible.

A note on not "searching":

Whenever there is "searching", there is the chance that more than 1 file exists that matches the search critera,
and the "found" file, might not be the one expected.
There is an advantage in avoiding "searching" if at all possible.

When you install a Puppy, you must know exactly where you installed it.
So to avoid any chance of surprises, why not tell "init" exactly where you installed it.
This "init" insists that you do.

When you select where your changed files should be saved, this "saveconfig" tells "init" exactly where it is saved, via "bootspecs".

Back to top
View user's profile Send private message 

Joined: 28 Oct 2008
Posts: 1498
Location: Brisbane, Australia

PostPosted: Wed 25 Apr 2018, 21:09    Post subject: Cleanup  

Over the next few days, I will cleanup the download site to remove all the old stuff, leaving just the version 0.9 downloads.
Back to top
View user's profile Send private message 

Joined: 28 Oct 2008
Posts: 1498
Location: Brisbane, Australia

PostPosted: Mon 30 Apr 2018, 00:45    Post subject: upupbb-18.05.iso  

I have uploaded a delta for upupbb-18.05.iso http://www.fishprogs.software/puppy/initrd/upupbb-18.05.iso___upupbb_overlay0.9-18.05.iso.delta.

And re-uploaded http://www.fishprogs.software/puppy/initrd/overlay_init_upupbb-0.9.tar to match it.

Back to top
View user's profile Send private message 

Joined: 28 Oct 2008
Posts: 1498
Location: Brisbane, Australia

PostPosted: Mon 30 Apr 2018, 07:00    Post subject:  

I have uploaded http://www.fishprogs.software/puppy/initrd/overlay_compat_upupbb-0.2.pet for upupbb-18.05.
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 12 of 12 [173 Posts]   Goto page: Previous 1, 2, 3, ..., 10, 11, 12
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum

Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0875s ][ Queries: 12 (0.0376s) ][ GZIP on ]