The time now is Sun 15 Dec 2019, 01:52
All times are UTC - 4 |
Page 5 of 14 [206 Posts] |
Goto page: Previous 1, 2, 3, 4, 5, 6, 7, ..., 12, 13, 14 Next |
Author |
Message |
gyro
Joined: 28 Oct 2008 Posts: 1689 Location: Brisbane, Australia
|
Posted: Sat 15 Jul 2017, 06:56 Post subject:
|
|
Rangan Masti wrote: | May be Light House Puppy init could be just an example!!! | Thanks, but on further thought, I've decided not to go this way. It's simple and obvious when setting it up, but there might be some difficulties when trying to remove a live sfs from the running system.
So I've implemented one that uses a list, which unfortunately then needs a utility to maintain that list. But more on that when I get to the point of releasing it.
Note: Many of the things I try, are ideas I've seen in the forum over the years. Many of these have never been incorporated into official puppy. e.g. using sfs files as a save mechanism.
gyro
|
Back to top
|
|
 |
Rangan Masti
Joined: 01 Jan 2013 Posts: 38 Location: Germany, Berlin
|
Posted: Sat 15 Jul 2017, 07:50 Post subject:
|
|
Hi, thank you for the reponse. Do not give up!!. Try further. Wish you a good success!!
Rangan
|
Back to top
|
|
 |
gyro
Joined: 28 Oct 2008 Posts: 1689 Location: Brisbane, Australia
|
Posted: Mon 17 Jul 2017, 11:35 Post subject:
Updated version |
|
I've uploaded new versions to http://www.fishprogs.software/puppy/initrd/initrd.gz and http://www.fishprogs.software/puppy/initrd/ydrv_overlay.sfs.
Both need to be downloaded, since the "ydrv_overlay.sfs" now contains a new utility "extra_sfs", and the command line utilities "file2initrd" and "initrd2file".
The significant change in this 18 July version is, extra sfs support.
The "extra_sfs" utility is a replacement for "sfs_load", in the sense that it masks "sfs_load" and takes its place in the menu.
But it does not load or unload any sfs files.
It simply maintains a list of extra sfs files that are loaded by "init" into the "pup_sfss" overlay, before the main overlay is created.
If "init" fails to load any extra sfs file, an appropriate message is written to "/initrd/tmp/bootinit.log", but the file remains in the list.
You can then either use "extra_sfs" to remove the failing entry, or fix the problem so that it will load successfully next boot.
The list is stored as a variable called "XTRA_SFS" in the "BOOT_SPECS" file.
This file can reside in the frugal install directory or in "initrd.gz", but it must reside in the same place as "DISTRO_SPECS".
So if you use the "file2initrd" utility to move "DISTRO_SPECS" into "initrd.gz" then you should do the same with any "BOOT_SPECS".
Also if you move these files into "initrd.gz", it will confuse "extra_sfs" if copies still exist in the frugal install directory. So either delete them, or rename them.
It's easier to just leave these files in the frugal install directory.
Note on DISTRO_SPECS:
This file defines the puppy you are booting, so the same "initrd.gz" can boot many different puppies provided it has access to the appropriate DISTRO_SPECS.
So, having this file outside "initrd.gz" means the same "initrd.gz" file can be tested with many puppies without change, I've booted both 64bit and 32bit puppies with the same "initrd.gz".
The limitation of the current "initrd.gz" is that the puppies kernel must support overlayfs.
gyro
|
Back to top
|
|
 |
Rangan Masti
Joined: 01 Jan 2013 Posts: 38 Location: Germany, Berlin
|
Posted: Mon 17 Jul 2017, 15:40 Post subject:
|
|
Hi, good work !
--Testing!!. Have a nice evening.
Rangan
|
Back to top
|
|
 |
peebee

Joined: 21 Sep 2008 Posts: 4098 Location: Worcestershire, UK
|
Posted: Mon 17 Jul 2017, 18:30 Post subject:
|
|
Looking good here - although I did have an oddity when I did a reboot - I got the saving message but this was followed by a dropout to the terminal prompt and neither reboot nor poweroff actually worked - had to press the power off button....
Code: | # lsoverlay
/initrd/mnt/tmpfs/pup_rw
/initrd/pup_a
/initrd/pup_y
/initrd/pup_ro2
/initrd/pup_f
/initrd/pup_z
/initrd/pup_5 |
Description |
|
Filesize |
147.39 KB |
Viewed |
277 Time(s) |

|
_________________
LxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64
|
Back to top
|
|
 |
gyro
Joined: 28 Oct 2008 Posts: 1689 Location: Brisbane, Australia
|
Posted: Mon 17 Jul 2017, 18:46 Post subject:
|
|
peebee wrote: | Looking good here - although I did have an oddity when I did a reboot - I got the saving message but this was followed by a dropout to the terminal prompt and neither reboot nor poweroff actually worked - had to press the power off button.... | Hmm.., I've never seen "rc.shutdown" dropout to a terminal.
Did you try Ctrl+Alt+Delete?
Does it do it every shutdown?
What pupmode are you running? (see /etc/rc.d/PUPSTATE.)
Edit: Never mind the pupmode question, it looks like you are running in pupmode=21.
gyro
|
Back to top
|
|
 |
peebee

Joined: 21 Sep 2008 Posts: 4098 Location: Worcestershire, UK
|
Posted: Tue 18 Jul 2017, 06:29 Post subject:
|
|
gyro wrote: | peebee wrote: | Looking good here - although I did have an oddity when I did a reboot - I got the saving message but this was followed by a dropout to the terminal prompt and neither reboot nor poweroff actually worked - had to press the power off button.... | Hmm.., I've never seen "rc.shutdown" dropout to a terminal.
Did you try Ctrl+Alt+Delete?
Does it do it every shutdown?
What pupmode are you running? (see /etc/rc.d/PUPSTATE.)
Edit: Never mind the pupmode question, it looks like you are running in pupmode=21.
gyro |
Ignore me - it was just an oddity - am unable to replicate/repeat the problem....
One quirk/feature I notice is that the frugal install partition is not mounted after boot which is what you would expect with the aufs initrd.....so for example extra sfs manager does not work until the partition is mounted.....
_________________
LxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64
|
Back to top
|
|
 |
gyro
Joined: 28 Oct 2008 Posts: 1689 Location: Brisbane, Australia
|
Posted: Tue 18 Jul 2017, 07:02 Post subject:
|
|
peebee wrote: | Ignore me - it was just an oddity - am unable to replicate/repeat the problem.... | I also have been unable to replicate.
I just tried it with a fresh frugal install of LxPupSc-17.07.23T.iso.
peebee wrote: | One quirk/feature I notice is that the frugal install partition is not mounted after boot which is what you would expect with the aufs initrd.....so for example extra sfs manager does not work until the partition is mounted..... | Pupmode=21 is an extension of pupmode=5, everything is in ram, even extra sfs's. So no partitions are mounted by default in the running system.
"extra_sfs" needs direct access to files in the frugal install directory, so it won't run until the partition is mounted manually, in pupmode=5 and pupmode=21.
Code could be added to "extra_sfs" to mount the partition if required, after all "rc.shutdown" has to do just that so it can write the save sfs. I'll think about it.
I tend to avoid this issue by creating an empty save folder, thus switching to pupmode=12 on the next boot.
gyro
|
Back to top
|
|
 |
peebee

Joined: 21 Sep 2008 Posts: 4098 Location: Worcestershire, UK
|
Posted: Tue 18 Jul 2017, 07:39 Post subject:
|
|
gyro wrote: | I tend to avoid this issue by creating an empty save folder, thus switching to pupmode=12 on the next boot.
gyro |
Ahhhh - hadn't twigged you could / should do that.....
If I do....
what is the ....work directory for?
what is the relationship of the save folder to the ...save.sfs.bak?
Thanks
peebee
_________________
LxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64
|
Back to top
|
|
 |
gyro
Joined: 28 Oct 2008 Posts: 1689 Location: Brisbane, Australia
|
Posted: Tue 18 Jul 2017, 08:16 Post subject:
|
|
@peebee,
Before I answer your questions there is just 1 thing:
Adding Code: | CONFIG_TMPFS_XATTR=y | to your kernel config should improve reliability when tmpfs is used as the rw layer of the main overlay, i.e. pupmode=5,21,29.
I noticed in my recent LxPupSc test that in dmesg, overlayfs complained that the upper fs does not support xattr.
peebee wrote: | what is the ....work directory for? | It's required by overlayfs for it's own internal use.
In pupmode=5,21,29 it's there as "/initrd/mnt/tmpfs/pup_wk".
peebee wrote: | what is the relationship of the save folder to the ...save.sfs.bak | When "init" finds an empty save folder, it copies the content of ...save.ram.sfs to the save folder and then uses that as the rw layer directly. This provides a seamless transition from pupmode=21 to pupmode=12.
After this all changes are written directly to the save folder. No new ...save.ram.sfs is produced and ...save.ram.sfs.bak is ignored.
So ...save.ram.sfs.bak is the sfs that was used to populate the initialy empty save folder.
gyro
|
Back to top
|
|
 |
peebee

Joined: 21 Sep 2008 Posts: 4098 Location: Worcestershire, UK
|
Posted: Tue 18 Jul 2017, 11:31 Post subject:
|
|
gyro wrote: | Adding Code: | CONFIG_TMPFS_XATTR=y | to your kernel config
It's required by overlayfs for it's own internal use.
...save.ram.sfs.bak is ignored.
|
Thanks....
config changed for next kernel build....
does ....work have to be in the frugal install directory? perhaps it can be in /tmp or deleted at shutdown??
perhaps ....save.sfs.bak can also be deleted when no longer needed?
_________________
LxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64
|
Back to top
|
|
 |
gyro
Joined: 28 Oct 2008 Posts: 1689 Location: Brisbane, Australia
|
Posted: Tue 18 Jul 2017, 12:20 Post subject:
|
|
peebee wrote: | config changed for next kernel build.... | Thanks, I look forward to that.
peebee wrote: | does ....work have to be in the frugal install directory? perhaps it can be in /tmp or deleted at shutdown?? | According to the overlayfs documentation, it has to be in the same file system as the rw directory.
It seemed like a simple idea to always put it beside the rw directory.
It's required, why should it not be obvious?
peebee wrote: | perhaps ....save.sfs.bak can also be deleted when no longer needed? | The ...save.ram.sfs.bak file could be deleted at any time. It's just some backup, it's never actually used.
It's not even necessary to create it. It simply provides an extra recovery option if things go pear-shape. This is still a testing environment.
gyro
|
Back to top
|
|
 |
gyro
Joined: 28 Oct 2008 Posts: 1689 Location: Brisbane, Australia
|
Posted: Fri 21 Jul 2017, 10:57 Post subject:
|
|
peebee wrote: | config changed for next kernel build.... | I just tried a fresh frugal install of LxPupSc 10.07.25, and it's 4.12.3 kernel seems to have fixed the issue.
gyro
|
Back to top
|
|
 |
gyro
Joined: 28 Oct 2008 Posts: 1689 Location: Brisbane, Australia
|
Posted: Fri 21 Jul 2017, 14:06 Post subject:
A reflection on where it's at |
|
So what have we learnt from this project?
Issues:
1. It has been reported on the net that there is a problem with using tmpfs as the upper fs for an overlay.
This is of concern because this init script does this for pupmode=5, pupmode=21 and pupmode=29.
Apparently this problem is related to a lack of extended attribute support in tmpfs, which can be fixed by compiling the kernel with "CONFIG_TMPFS_XATTR=y".
Fortunately the kernels in slacko 6.9.9.9 and LxPupSc 17.07.25 meet this requirement. Unfortunately the released kernel in xenialpup 7.0.8.1 does not.
The issue can be avoided by running in pupmoode=12.
Note: I have never experienced any problem, although I did notice recent kernels did complain with "overlayfs: uppper fs does not support xattr" in dmesg.
2. Compared to aufs, overlayfs has some significant limitations, i.e. can't add or delete directories in a stack, can't directly modify the contents of a directory in a stack.
This init script works within these limitations.
Things we can do:
1. We can boot and run normal woof-ce puppies using overlayfs for the stack.
The upside is, overlayfs is supposed to be very efficient, and is supported directly in the kernel, (no aufs patch needed).
The downside is, "sfs_load" is gone, the stack has to be completely created by init.
We can still have extra sfs's, it's just that it takes a reboot, to actually load or unload an sfs.
2. The same initrd.gz can be used to boot many different puppies.
I introduced the facility for init to use a DISTRO_SPECS from outside the initrd.gz to simplify testing.
But it has demonstrated that, just as we can move huge-kernels from one puppy to another, it's possible to move initrd.gz's from one puppy to another.
So I can take this initrd.gz, push my "TIME_ZONE" file into it, and expect to be able to boot any woof-ce puppy with it, after extracting the puppies DISTRO_SPECS from the release initrd.gz.
3. We can easily specify things like extra sfs's before first boot, using a BOOT_SPECS file, beside the DISTRO_SPECS file.
(And any specified extra sfs's will be loaded by first boot.)
4. We can have a puppy that has automatic persistence, without shutdownconfig.
Of course pupmode=21 has significant limitations, i.e. it all has to fit back into the tmpfs on next boot.
So it would not be recommended for use as a "work" puppy.
Other new things:
1. Enable save folder by simply creating an appropriately named directory.
If done before first boot, e.g. as part of installation, it will be ignored until first reboot.
2. "pfix=nosave" boot parameter, disables automatic saving at shutdown in pupmode=5, pupmode=21 and pupmode=29, i.e. when /initrd/mnt/tmpfs/pup_rw is the rw directory of the main overlay.
3. "pfix=rosave" boot parameter, use a read-only save layer, i.e. choose pupmode=29 instead of pupmode=12.
Pupmode=29 is a replacement for pupmode=13, but I didn't want to confuse puppy utilities like snapmergepuppy.
4. "pfix=nobs" boot parameter, disable processing of BOOT_SPECS file.
5. "pfix=nofsckp" boot parameter, disable fsck of each partition before first mount.
The previous "pfix=fsckp" processing is now the default.
gyro
|
Back to top
|
|
 |
woodenshoe-wi
Joined: 28 Jul 2017 Posts: 104 Location: Wisconsin
|
Posted: Sat 29 Jul 2017, 00:33 Post subject:
RE: A reflection on where it's at |
|
I just wanted to let you know that after replacing the 386 binaries in the initrd.gz with ARM binaries I was able to use your init script on my Pi2 (1GB RAM) with only two small modifications.
Code: |
--- original_init 2017-07-17 09:25:51.000000000 -0500
+++ edited_init 2017-07-19 13:02:38.000000000 -0500
@@ -574,6 +574,12 @@
[ -d /mnt/tmpfs ] || mkdir -p /mnt/tmpfs
mount -t tmpfs tmpfs /mnt/tmpfs
+#raspberry pi, load squashfs kernel module
+[ -d /mnt/boot] || mkdir -p /mnt/boot
+mount /dev/mmcblk0p1 /mnt/boot
+insmod "/mnt/boot/modules/$KERNELVER/squashfs.ko"
+umount /mnt/boot
+
#mount pdrv
mount_sfs "$PDRV_SPEC" "pdrv"
[ $? -eq 0 ] || fatal_error "$mount_sfs_RET"
@@ -615,7 +621,9 @@
fi
fi
ZDRV_MNTPT=''
-sortout_sfs_spec "$DISTRO_ZDRVSFS" "$ZDRV"
+#sortout_sfs_spec "$DISTRO_ZDRVSFS" "$ZDRV"
+#raspberry pi, need to support two different zdrvs for armv6 and armv7 kernels
+sortout_sfs_spec "zdrv_raspup_$(uname -r).sfs" "mmcblk0p1:/zdrv_raspup_$(uname -r).sfs"
if [ "$sortout_sfs_spec_RET" ];then
ZDRV_SPEC="$sortout_sfs_spec_RET"
mount_sfs "$ZDRV_SPEC" "zdrv"
|
Is this re-write of the init script going to eventually be merged with the init in woof-CE, or is it a separate experiment?
I'm not sure if the Pi1/Pi0 has enough RAM (512MB) to run completely in RAM, but it opens the possibility to running frugally in a FAT formatted SD card without reformatting/writing an image file to it.
I am planning to (eventually) figure out how to cross compile a kernel with AUFS support, but it is nice to already be able to play with the pre-compiled kernels.
|
Back to top
|
|
 |
|
Page 5 of 14 [206 Posts] |
Goto page: Previous 1, 2, 3, 4, 5, 6, 7, ..., 12, 13, 14 Next |
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You can download files in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|