Another re-write of the "init" script - using OverlayFs?

[01micko]

The problem wih this is that the superblock times in a Linux partition can get skewed into the future either when you reboot to this Puppy or when you reboot to another Puppy, depending on which side of your actual timezone the hardcoded timezone is set to.

What about for ISO booting purposes, hard coding to Pacific/Kiritimati? That is the eastern most time zone so therefore anywhere after must be in the past. I notice fsck doesn't care if the time is in the past. I just checked by skipping my clock back an hour (future superblock error is reported) and skipping it forward an hour and no error reported. Of course I have Australia/Queensland set in my TIME_ZONE file.

In other news, I did the deed with a fresh slacko64 install with kernel updated to 4.9.73 with necessary configs. I approached it a bit differently though. I did the fresh frugal and then booted it, set timezone and from within that install performed all of your modifications. Seems to be working quite well. I have a google-chrome sfs loaded running as 'spot' with added support packages added from PPM (gtk+3, updated mozilla-nss etc).

I'll keep this as a daily driver for a while.

Thanks!

[peebee]

Tested with:
LxPupSc-18.01+4T-k64
Kernel 4.14.13 64-bit

All seems fine
Cheers
peebee

menu.lst=

Code: Select all

title OverlayFS (sda4/overlayfs)
  uuid 9bb32631-a1f9-425f-81bb-9f816347e8fc
  kernel /overlayfs/vmlinuz  pmedia=atahd pdrv=sda4 psubdir=overlayfs
  initrd /overlayfs/initrd.gz

Code: Select all

====> /initrd/tmp/bootinit.log <====

'FATAL' messages may be insignificant.

Doing fsck of /dev/sda4 as ext4 with e2fsck.
e2fsck 1.43.3 (04-Sep-2016)
Linux-3: clean, 303495/5693440 files, 15090214/22761984 blocks
Mounting /dev/sda4 on /mnt/sda4 as ext4.
Using /mnt/sda4/overlayfs/BOOT_SPECS
Using 'puppy_LxPupSc_18.01.sfs' as sfs0 file...
 Copying /mnt/sda4/overlayfs/puppy_LxPupSc_18.01.sfs to /mnt/tmpfs/
 Mounting /mnt/tmpfs/puppy_LxPupSc_18.01.sfs on /pup_sfs0.
Using 'fdrv_LxPupSc_18.01.sfs' as sfs1 file...
 Copying /mnt/sda4/overlayfs/fdrv_LxPupSc_18.01.sfs to /mnt/tmpfs/
 Mounting /mnt/tmpfs/fdrv_LxPupSc_18.01.sfs on /pup_sfs1.
Using 'zdrv_LxPupSc_18.01.sfs' as sfs2 file...
 Copying /mnt/sda4/overlayfs/zdrv_LxPupSc_18.01.sfs to /mnt/tmpfs/
 Mounting /mnt/tmpfs/zdrv_LxPupSc_18.01.sfs on /pup_sfs2.
Using 'overlay_mods.sfs' as sfs3 file...
 Copying /mnt/sda4/overlayfs/overlay_mods.sfs to /mnt/tmpfs/
 Mounting /mnt/tmpfs/overlay_mods.sfs on /pup_sfs3.
Using 'adrv_LxPupSc_18.01.sfs' as sfs4 file...
 Copying /mnt/sda4/overlayfs/adrv_LxPupSc_18.01.sfs to /mnt/tmpfs/
 Mounting /mnt/tmpfs/adrv_LxPupSc_18.01.sfs on /pup_sfs4.
Adding module kernel/fs/overlayfs/overlay.ko
Mounting pup_sfs3:pup_sfs4:pup_sfs0:pup_sfs1:pup_sfs2 on /pup_sfs...
umount_unneeded - keeping /dev/sda4
Booting with PUPMODE=12
Using personal storage /overlayfs/LxPupScsave [sda4].
Mounting LxPupScsave:pup_sfs on /pup_new...
mount -o move /pup_sfs0 /pup_new/initrd/pup_sfs0
mount -o move /pup_sfs1 /pup_new/initrd/pup_sfs1
mount -o move /pup_sfs2 /pup_new/initrd/pup_sfs2
mount -o move /pup_sfs3 /pup_new/initrd/pup_sfs3
mount -o move /pup_sfs4 /pup_new/initrd/pup_sfs4
mount -o move /pup_sfs /pup_new/initrd/pup_sfs
mount -o move /mnt/sda4 /pup_new/initrd/mnt/sda4
mount -o move /mnt/tmpfs /pup_new/initrd/mnt/tmpfs
'/pup_new/initrd/pup_rw' -> '/initrd/mnt/sda4/overlayfs/LxPupScsave'
'/pup_new/tmp' -> '/initrd/mnt/tmpfs/tmp'

[jamesbond]

The problem wih this is that the superblock times in a Linux partition can get skewed into the future either when you reboot to this Puppy or when you reboot to another Puppy, depending on which side of your actual timezone the hardcoded timezone is set to.

I got interested when I read this, so I've done a few tests.

I found out that this superblock problem only manifest itself when your hardware clock is set to localtime - which is the default settings in most puppies, to accomodate dual booting with Windows. If hwclock is set in UTC, all is good.

I don't know how you do it gyro, but this is how I did it in init (I don't have persistence file in initrd like you do): "[ $TZ ] && hwclock -t" where hwclock is busybox applet similar to coreutils' utility of the same name.

This way, I only need to pass boot parameter TZ=EST-10 and all will be right. Of course, again, this is only necessary when running hwclock in localtime.

[peebee]

I've made a delta to produce LxPupSc-18.01+4-overlayfs.iso

See: http://murga-linux.com/puppy/viewtopic. ... 329#980329

[stemsee]

I tried this with artful pup. opened the initrd and put in the DISTRO_SPECS. added kernel arguments pdrv and psubdir and everything else was automatic.
Once at desktop loaded wine-2.2.sfs @200mb. the 32bit pae kernel recognised 7.9gb of 16gb.

[gyro]

I've made a delta to produce LxPupSc-18.01+4-overlayfs.iso

See: http://murga-linux.com/puppy/viewtopic. ... 329#980329

Thanks.
I assume you did not inclued a TIME_ZONE in the initrd.gz in this iso.
gyro

[peebee]

I've made a delta to produce LxPupSc-18.01+4-overlayfs.iso

See: http://murga-linux.com/puppy/viewtopic. ... 329#980329

Thanks.
I assume you did not inclued a TIME_ZONE in the initrd.gz in this iso.
gyro

Yes I did - 01Micko's suggestion as mentioned in the link....

[gyro]

Firstly, thanks folk for taking the time to test this.

Re the timezone thing:
1. Skewing only occurs if the hardware clock is set to local time.
2. If you set it to a fixed value you see the skew reported either:
When you mount a partition with the correct timezone that was umounted with the fixed timezone,
or
When you mount a partition with the fixed timezone that was umounted with the corrent timezone.
3. This init seeks to provide facilities so that the timezone can set consistently for all mounts.
4. The TIME_ZONE file idea makes most sense when you are modifying the initrd.gz before first boot, it does not make as much sense when this intrd.gz is already included in an iso.
5. Version 0.2 will include support for a "ptz" boot parameter. e.g. ptz=AEST-10
6. Version 0.2 will also support the possibility of the time-zone in init being set by some utility in the running system, possibly 'Quick Setup'. The skew issue could still occur on first boot.

gyro

[gyro]

I've made a delta to produce LxPupSc-18.01+4-overlayfs.iso

See: http://murga-linux.com/puppy/viewtopic. ... 329#980329

Thanks.
I assume you did not inclued a TIME_ZONE in the initrd.gz in this iso.
gyro

Yes I did - 01Micko's suggestion as mentioned in the link....

Sorry, but I don't think it's a good idea to use the timzone setting facilities in this manner. If you are going to set the timezone it should be to the correct one.
Which is one of the problems when trying to use version 0.1 in an iso.
Unfortunately version 0.1 effectively sets the timezone to UTC in the absense of a TIME_ZONE file.
Version 0.2 will set it to 'XXX12' (west of everwhere), if it is not set in a TIME_ZONE file or ptz boot parameter.
(I need to test the west v east thing again before releasing version 0.2)

gyro

[peebee]

If you are going to set the timezone it should be to the correct one.

There is no "correct" one for a general purpose .iso.....

[gyro]

If you are going to set the timezone it should be to the correct one.

There is no "correct" one for a general purpose .iso.....

Yes, that's what I was trying to say.
The 'init' needs to have a hardcoded default that ensures that no one sees the issue on first fsck on first boot.
Then provide mechanisms for the correct local timezone to be set in 'init'.
This is what I have coded into "overlay_init-0.2". (Just a bit more testing to do)

Then there is no point in having a TIME_ZONE file in the iso.

gyro

[gyro]

I've made a delta to produce LxPupSc-18.01+4-overlayfs.iso

See: http://murga-linux.com/puppy/viewtopic. ... 329#980329

I downloaded this and used it for a fresh manual frugal install.
It appeared to work fine.
I then extracted the TIME_ZONE file from the "initrd.gz" and realised that it appeared to work because it didn't work.
The problem is the TIME_ZONE file:

Code: Select all

<+14>-14|Pacific/Kiritimati

The "init" script assigns the characters before the '|' to the TZ variable. The problem is that the "hwclock" in "initrd.gz" cannot cope with this format, so does not set the expected timezone.
I changed the TIME_ZONE file to:

Code: Select all

LINT-14|Pacific/Kiritimati

and re-inserted it into "intrd.gz", then on first boot I got the superblock timezone warnings from fsck.

When I removed the TIME_ZONE file from "initrd.gz" with "initrdDfile", added "TZ=AEST-10" as a boot parameter, and did another first boot, I got no warnings.

The problem with the TIME_ZONE file is mine, "mk-timezone-file" simply accepted this incompatible format.
This is fixed in the next version which will replace:

Code: Select all

<+14>-14

with

Code: Select all

XXX-14

Appart from this TIME_ZONE thing, the iso worked as expected.

gyro

[gyro]

Release of overlay_init-0.2

Download from http://www.fishprogs.software/puppy/ini ... it-0.2.tar.

Changes:

1. New timezone handling including a new format for the "TIME_ZONE" file, example:

Code: Select all

P_TZONE='Australia/Queensland'
P_TZ='AEST-10'

So if you have an existing "TIME_ZONE" file you will need to run "mk-timezone-file" again.
The timezone in "init" can be set in any 1 of 3 ways:
Create a "TIME_ZONE" file with "mk-timezone-file" and add it to the "initrd.gz" with "initrd-pup-file".
Add "TZ=AEST-10" as a boot parameter, use "showTZ" to see an appropriate value for your timezone.
Run "set-init-timezone" after the timezone has been set in the running Puppy. The timezone in "init" will then be set for subsequent boots.
In the absence of any specified timezone, "init" uses "TZ=XXX12", the most westerly timezone.

2. All the timezone handling utilities have been fixed to handle the stange format of some TZ specs at the end of some zoneinfo files.

3. New Targeted update for "rc.update", after sfs stack changes.
When changes to the sfs stack are detected, the sfs is checked to see what updates are required, similar to current "sfs_load".
This is done in both the sfs manager utilities, and "rc.update" itself.
Then "rc.update" only performs the appropriate updates.
"BOOTCONFIG" is no longer used, instead "PREVUNIONRECORD" and "LASTUNIONRECORD" are files on their own.

4. Changed the "Archive" save mechanism to use "PUPMODE=37".

5. Renamed some CLI utilities:
bootspecs2tmpbs -> bootspecs-init
tmpbs2bootspecs -> bootspecs-save
tmpbs2variable -> bs-read-var
variable2tmpbs -> bs-write-var
tmpbsDvariable -> bs-delete-var
initrd2file -> initrd-get-file
file2initrd -> initrd-put-file
initrdDfile -> initrd-delete-file

6. Combined "lsoverlay" and "showoverlay", they now both provide the same output.

7. Removed support for "pimod" boot parameter, it's not really needed.

To try this version, use the same procedure as for overlay_init-0.1,
see http://www.murga-linux.com/puppy/viewto ... 6&start=96

Manual update of overlay_init-0.1 install, that might work:
1. Extract the tar file.
2. Replace the current "overlay_mods.sfs".
3. Extract the updated "init" from "initrd.overlay.gz" with "initrd2file" or "initrd-get-file".
4. Store this updated "init" in the currrent "initrd.gz" with "file2initrd" or "initrd-pup_file".
5. If your current "initrd.gz" contains a "TIME_ZONE" file you need to replace it with a new one.

gyro

[peebee]

New delta for LxPupSc made with overlay_init-0.2....

Timezone not altered during make process - boots with GMT-8....

[gyro]

New delta for LxPupSc made with overlay_init-0.2....

Timezone not altered during make process - boots with GMT-8....

Thanks, I'll give it a spin.

Later:
It works as expected.
I did a normal manual frugal install from the iso.
Booted, and as expected the timezone was GMT-8.
After doing "Quick Setup", I ran "mk-timezone-file", and added it into the "initrd.gz" with "initrd-put-file".
On first shutdown I chose 'Archive'.
On reboot all was as expected, I did not get a first boot, all the choices from the first "Quick Setup" were still in place.

gyro

[gyro]

New delta for LxPupSc made with overlay_init-0.2....

Don't bother trying to boot this using isoBooter, it won't work.
The 'init' script ignores the BOOT_SPECS file. The whole booting from an iso stuff in 'init', needs to be reworked.
gyro

[gyro]

New delta for LxPupSc made with overlay_init-0.2....

I just tried using this iso with the "UEFI Installer" in Tahrpup 6.0.6, and it failed on 2 accounts.
1. It didn't copy the BOOT_SPECS file from the iso.
2. It didn't provide a "pupsfs" (or "pdrv") boot parameter.
After manually fixing these 2 ommissions, it all worked fine.

This deficency in the installer is not surprising, since in the past neither of these were requirements.

So, this still ends up being a manual frugal install.

gyro

[gyro]

I've uploaded an iso delta file of the current version 0.2++, against "artfulpup-17.11+12.iso".

With some help from peebee, I've written a script that creates a patched iso that actually boots.
And now I've produced a delta from "artfulpup-17.11+12.iso" to "artful-overlay-17.11+12.iso".

This is not a release of a new version, rather a test of producing an iso and a delta.
I uploaded it mainly because it represents an iso that actually boots, and the delta is so small (1.8MiB).

Fixing the "init" and utilities to work correctly with actually booting an iso, is what I've been working on since the release of version 0.2.

For what it's worth, here it is.
Download from http://www.fishprogs.software/puppy/ini ... .iso.delta.

Note: My script uses an overlayfs stack to "merge" my mods with the parent iso.
I created a "mods" directory containing the patched files.
Mounted the parent iso on a "release" directory, and copied it's "isolinux.bin" to "mods".
Then mounted an overlay stack on the directory "stack", with "mods" as the read/write directory, and "release" as the lower read-only directory.
I then did a "mkisofs" using the "stack" directory.

gyro

[ally]

full ISO here: http://archive.org/download/Puppy_Linux ... plus12.iso

[gyro]

@ally,
Thanks.
gyro