Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Mon 18 Dec 2017, 08:59
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
Another re-write of the "init" script - using OverlayFs?
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 4 of 7 [96 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7 Next
Author Message
gyro

Joined: 28 Oct 2008
Posts: 1409
Location: Brisbane, Australia

PostPosted: Tue 27 Jun 2017, 17:24    Post subject:  

peebee wrote:
Is this a temporary fix or a design decision for how the new init will always work?

Putting pfix=nocopy does not seem intuitive to me - I want Puppy loaded into ram to maintain the speed advantages.....I've never used nocopy so I'm vaguely remembering that that's what it means..
Sorry, I should make it clear what is actualy part of the feature under test.
The use of pfix=nocopy as the trigger is just a testing convenience, not a new way for puppy to operate.
The possible feature is, that we can have normal pupmode=12 with overlayfs and without any shutdownconfig.
In the next version, pupmode=12 will be triggered by the existence of a save directory, either empty or populated.

Note:
I've never been a fan of the concept of users being presented with shutdownconfig on first shutdown.
I am pursuing the idea of having a puppy that has persistence, without any shutdownconfig. Although there would still need to be a save layer configuration utility for those who like to fiddle.

For the record, "pfix=nocopy" means "don't copy the sfs's to ram".

Gyro
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1409
Location: Brisbane, Australia

PostPosted: Thu 29 Jun 2017, 21:14    Post subject: Updated files  

I've uploaded new versions to http://www.fishprogs.software/puppy/initrd/initrd.gz and http://www.fishprogs.software/puppy/initrd/ydrv_overlay.sfs.
Both need to be downloaded, since the "ydrv_overlay.sfs" now contains modified versions of "shutdownconfig", "rc.update", and "rc.shutdown".

Changes in this 30 June version:

1. The pupmode=12 is now triggered by the existence of a save folder.
If you create a new directory apporpriately named as a save folder immediately after a fresh frugal install, it will boot in pupmode=5 except for using the save folder as it's rw directory.
On reboot it will come up as a normal pupmode=12, (no shutdownconfig involved).

2. There's a new pupmode=21 (5+16), this is automatic persistence following a normal pupmode=5 boot.
Simply do a fresh frugal install, ensure that the boot parameters include ones similar to either "pdrv=sdc2:/puppy/slacko/" or "pdev1=sdc2 psubdir=/puppy/slacko".
Booting will produce the usual first boot.
A reboot should not produce a first boot, but it should contain the changes you made during first boot.
Make further configuration changes and reboot, the changes should be retained.
When you are finished making changes, add a "pfix=nosave" boot parameter.
Now, changes made during a session should be ignored.
This facility should work on any filesystem, I've tested it on ext4 and vfat.

How does this pupmode=21 work:
On shutdown, the contents of the "pup_rw" in ram, are written to an sfs file.
In "init", the contents of this sfs file are copied into "pup_rw" before the main overlay is created.

It all begins with a limitation of overlayfs:
Can't modify any read-only diretory in the stack while it is mounted, (result is "undefined").
This means that "snapmergepuppy" can't do it's thing, in either the running system, or during shutdown, since the main overlay stack is still mounted.
So if we're not running pupmode=12, about the only thing we can do during shutdown, is take a copy of the "pup_rw" in ram, and sort it out in "init" before the main overlay stack is mounted.
The copy of "pup_rw" is an sfs because it can safely store all linux files on any filesystem, and it can be read directly.
pupmode=21 is the simplist thing that "init" can do with this copy of the previous "pup_rw".
The next thing on the todo list is to produce a pupode=29 (13+16), that will be a pseudo pupmode=13.

Note1: Booting with "pfix=ram", will always produce a first boot, ignoring any saved stuff.

gyro
Back to top
View user's profile Send private message 
peebee


Joined: 21 Sep 2008
Posts: 3042
Location: Worcestershire, UK

PostPosted: Fri 30 Jun 2017, 04:00    Post subject:  

Looking good.....
Code:
Pup-SysInfo Hardware Report (Summary), Fri 30 Jun 2017

Linux Kernel: 4.11.8-lxpup-32-pae (i686)
Kernel Version: #1 SMP Fri Jun 30 18:06:51 +08 2017
PAE Enabled: Yes

Kernel Command Line:
pmedia=atahd pdrv=sda3 psubdir=devtest pfix=fsck

Distro: LxPup-Sc 17.06.26
Desktop Panel: lxpanel 0.9.3
Window Manager: Openbox 3.6.1
Desktop Start: xwin startlxde

Boot File System: ext4
Boot Media: atahd

PUPMODE=21
PUPSFS=sda3,ext4,/devtest/puppy_LxPupSc_17.06.26.sfs
PUPSAVE=

Booted - rebooted - made wifi connection - rebooted - wifi came up Smile

_________________
LxPup = Puppy + LXDE
Back to top
View user's profile Send private message Visit poster's website 
gyro

Joined: 28 Oct 2008
Posts: 1409
Location: Brisbane, Australia

PostPosted: Fri 30 Jun 2017, 06:35    Post subject:  

peebee wrote:
Booted - rebooted - made wifi connection - rebooted - wifi came up Smile
Great.
Thanks for testing.
gyro
Back to top
View user's profile Send private message 
Rangan Masti

Joined: 01 Jan 2013
Posts: 34
Location: Germany, Berlin

PostPosted: Fri 30 Jun 2017, 12:49    Post subject:  

Hi, good evening. Testing your new initrd.All the BEST. Laughing
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1409
Location: Brisbane, Australia

PostPosted: Fri 30 Jun 2017, 21:09    Post subject:  

@Rangan Masti,
Thanks.
gyro
Back to top
View user's profile Send private message 
Keef


Joined: 20 Dec 2007
Posts: 876
Location: Staffordshire

PostPosted: Sat 01 Jul 2017, 05:04    Post subject:  

I've tried this with the current 32bit Slacko (RC3). It works as advertised (well once I read the instructions properly), but as well as the manually created save directory, there is a 'slackowork' directory. Inside that is another empty directory called 'work''.
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1409
Location: Brisbane, Australia

PostPosted: Sat 01 Jul 2017, 08:44    Post subject:  

Keef wrote:
but as well as the manually created save directory, there is a 'slackowork' directory. Inside that is another empty directory called 'work''.
Yep, that's a directoty required by overlayfs.
If you "cat /proc/mounts" and look at the line for the overlay mounted on /, you will see the "workdir" mentioned.
Thanks for testing.

PS: Have you tried not creating a save directory?
Just do a clean frugal install and change stuff, and reboot, and change stuff, and reboot.
You should notice that there is persistence, even when running completely in ram, pupmode=21. This is a facility we don't have in current puppy.

gyro
Back to top
View user's profile Send private message 
Keef


Joined: 20 Dec 2007
Posts: 876
Location: Staffordshire

PostPosted: Sat 01 Jul 2017, 13:01    Post subject:  

I'd rather stupidly forgot to rename the ydrv Embarassed I was wandering why I was being prompted to save when I shutdown. Which I didn't do, but I did get persistence if I made savefolder myself. Need to pay more attention.

Having done what I should have done in the first place, yes it does save changes without any intervention from me. Posting from the install now, after first reboot.
Back to top
View user's profile Send private message 
Keef


Joined: 20 Dec 2007
Posts: 876
Location: Staffordshire

PostPosted: Sun 02 Jul 2017, 10:25    Post subject:  

I've been experimenting with loading sfs files. First used a small sfs for the Alacritty terminal. Get the warning that Pupmode 21 is experimental, but it works ok. Then tried the devx. Took quite a while to load (compared to the normal situation), but it still worked correctly.
Both stay loaded after reboot.
The most noticable thing is that with a large sfs like the devx installed, saving takes a lot longer. I assume that is because of the time it takes the slackosave.ram.sfs to be built.
No option with sfs_load to uninstall the sfs, but I suppose this could be done manually - I haven't tried yet.
[EDIT] Scratch that - it eventually dawned on my why that is not possible.
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1409
Location: Brisbane, Australia

PostPosted: Mon 03 Jul 2017, 01:03    Post subject:  

@Keef,
I'm surprised loading extra sfs's did anything, no aufs stack.
And I haven't done any overlayfs support for extra sfs's.

If you were really desperate to load the devx, you could rename the zdrv to the fdrv and rename the devx to zdrv, or some similar fiddle.
Note: if you try doing anything like this, don't load the devx as an adrv or ydrv, it should always go under the main puppy...sfs.

gyro
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1409
Location: Brisbane, Australia

PostPosted: Mon 03 Jul 2017, 05:00    Post subject: Announcing a new version  

I've uploaded new versions to http://www.fishprogs.software/puppy/initrd/initrd.gz and http://www.fishprogs.software/puppy/initrd/ydrv_overlay.sfs.
Both need to be downloaded, since this version of "ydrv_overlay.sfs" contains an updated version of "rc.shutdown", and the command line utilities "lsoverlay" and "showoverlay".

Changes in this 3 July version:

1. Contains a new pupmode=29 a pseudo pupmode=13 that uses tmpfs for the rw directory and the save folder is the top ro directory. I'm not using pupmode=13 for this, so as not to confuse the running puppy, e.g. "snapmergepuppy".
2. Includes "pfix=rosave" to control use of pupmode=29 instead of pupmode=12, and "pfix=nosave" to stop saving during shutdown.

During first boot, any empty save folder is ignored, the boot proceeds as a normal pupmode=5 boot, all in ram.
During second boot:
If the first boot included "pfix=nosave" another first boot will be performed.
Else if there is no save folder present it will boot pupmode=21, all in ram, but remembering changes made in the previous session.
Else if a save folder exists and "pfix=rosave" is specified it will boot pupmode=29.
Else if a save folder exists, it will boot pupmode=12.

During any boot, if the conditions for one of pupmode=29, pupmode=12 or pupmode=21 are not met, it will do a normal pupmode=5 boot.

Since with overlayfs, there can be no writing to any ro directory while the main overlay is mounted, "snapmergepuppy" type processing is done by "init".
As per the previous version the only saving done in "rc.shutdown", is to create an sfs file on disk, from the "pup_rw" in ram, unless "pfix=nosave" has been specified.

"pfix=nosave" has no effect on pupmode=12, because all writing has already been done before shutdown.

Overlayfs documentation says that a directory on any filesystem supported by Linux can be used as a read-only directory.
So theoretically pupmode=29 should be able to use a save folder any any filesystem, if the "snapmergepuppy" type code in "init" was careful about what files were written to disk.
I tried this with a directory on a "vfat" partition and the overlay mount failed.
I tried this with a directory on an "ntfs" partition and the overlay mount worked.
But the result was quite annoying, in ROX filer, a single click on a text document appeared to do nothing, since the file had "execute permission" set, ROX filer executed it.
I then abandoned this concept, the pupmode=29 implemented here only works on Linux filesystems.

Running "lsoverlay" in a console will produce a list of the directories in the "/" overlay, in the order top to bottom.
"showoverlay" goes one step further and unwinds any directories that are mountpoints for loop devices to show the mounted files.

gyro
Back to top
View user's profile Send private message 
peebee


Joined: 21 Sep 2008
Posts: 3042
Location: Worcestershire, UK

PostPosted: Sat 08 Jul 2017, 06:14    Post subject:  

sfs loading next....??
Code:
# lsoverlay
/initrd/mnt/tmpfs/pup_rw
/initrd/pup_a
/initrd/pup_y
/initrd/pup_ro2
/initrd/pup_f
/initrd/pup_z
# lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0    7:0    0 181.1M  1 loop /initrd/pup_ro2
loop1    7:1    0  20.6M  1 loop /initrd/pup_a
loop2    7:2    0    28K  1 loop /initrd/pup_y
loop3    7:3    0  15.6M  1 loop /initrd/pup_f
loop4    7:4    0  22.9M  1 loop /initrd/pup_z

_________________
LxPup = Puppy + LXDE
Back to top
View user's profile Send private message Visit poster's website 
gyro

Joined: 28 Oct 2008
Posts: 1409
Location: Brisbane, Australia

PostPosted: Wed 12 Jul 2017, 02:47    Post subject:  

peebee wrote:
sfs loading next....??
Yes.
I'm currently thinking of an "init" that automatically loads any extra sfs's found in the frugal install directory.
gyro
Back to top
View user's profile Send private message 
Rangan Masti

Joined: 01 Jan 2013
Posts: 34
Location: Germany, Berlin

PostPosted: Wed 12 Jul 2017, 04:43    Post subject:  

Hi, Good morning.
May be Light House Puppy init could be just an example!!!
wish you good success.

Rangan
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 4 of 7 [96 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0569s ][ Queries: 14 (0.0048s) ][ GZIP on ]