Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sun 16 Jun 2019, 17:29
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
Minimal Init Overlay v13 (mio) - alternate "init" and utils
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [15 Posts]  
Author Message
gyro

Joined: 28 Oct 2008
Posts: 1623
Location: Brisbane, Australia

PostPosted: Fri 03 May 2019, 22:14    Post subject:  Minimal Init Overlay v13 (mio) - alternate "init" and utils
Subject description: Exploring some different ways for Puppy to work.
 

Edit:
Current release is now 'mio13', see http://www.murga-linux.com/puppy/viewtopic.php?p=1028295#1028295 for more information and downloads.

---------Original message below this line---------------

This is the first release in this "Minimal Init" project.
It is a successor to the "Overlay Init" project, but the focus has changed to the different facilities it implements, rather than overlayfs.
Although this release uses overlayfs for the stack, sometime in the future, there will be a "Minimal Init Aufs" (mia) relesase.

The "Minimal" does mean what you might expect, i.e. the 'mio_initrd.gz' is approximately half the size of the normal 'initrd.gz'.
It is about removing all the current stuff that it might be possible to do without, but it also tries to move complexity into utilities in the running system.
This 'init' "does what it is told", working out what needs to be done, is done elsewhere.

But this project also aspires to be very flexible in those things it does do.

As an example, consider "luks encryted files";
In a "normal" Puppy this is provided by a luks encrypted savefile, too bad if you want to use a savefolder.
Here, it is provided by the "PupVault - encrypted file store", which can be deployed anywhere, in a savefolder, in a savefile, or '/mnt/home', etc...
And the luks support is removed from 'initrd.gz'.

The focus for this project is Puppy frugal installs on ordinary "block" devices, both internal and usb connected.
In particular, installs on Linux and fat32 partitions.

For now, this project is a Puppy "add-on", i.e. you add "mio" to a "normal" frugal Puppy install,
which changes the way the Puppy works, but the original Puppy is still there.
If installed on a Linux partition, there are mechanisms provided to switch between the two.


Some Features:

Can run completely in RAM with persistence, while installed on a normal block device, (a bit like pupmode=77).

Supports "pfix=nosave" for all save mechanisms, (including pupmode=12).

SFS's can be located anywhere, and you can have as many as you like, both above and below the main puppy...sfs, with "MI SFS List manager".

Automatic saving without any dialog on first shutdown, if installed on Linux or fat32 partition.

The save mechanism and the location of the save directory can be changed at any time with "MI Save config".

Backup/Restore the save layer, while it is not in use, with "MI save backup".

Can specify commonly used "pfix=" boot parameters with "MI Boot config".

Can switch from "mio" to "normal" with "MI Boot config".

"FrugalPup/StickPup" to manage everything to do with frugal installs.
It can install any Puppy iso.
It can install a boot loader that will do a normal UEFI boot.
It can do a "Grub4dos Config" like thing for both UEFI and non-UEFI boots.

"PupVault" to support the storing of files encrypted with luks.

Default fsck of ext and fat32 partitions, every boot before first mount, for maximum data integrity.

"improved" 'init' support information in '/initrd' and '/initrd/tmp'.

No complicated boot parameters.

And several internal "improvements" to the 'init' script.


Downloads:
Most downloads for this project are available from http://www.mediafire.com/folder/q04fxq43exkfr/mio.


Downloadable documentation:
'mio_utils.txt': some usage information on the utilities included in "mio".
'README.txt': the README file for 'init' from 'mio_initrd.gz'


Installing mio v12:

The 3.8MiB download option:

This release is available as 'mio12_bionicpup32_8.0.tar' and 'mio12_bionicpup64_8.0.tar'.

Each '.tar' contains 2 files, an 'mio_initrd.gz' and an 'mio12...sfs'.
The idea is to setup a frugal Puppy install directory containg these 2 files,
and the files from a frugal install of the corresponding "normal" Puppy.
Then boot using the 'mio_initrd.gz' file instead of the "normal" release 'initrd.gz'.

This method requires a working Puppy, some space on a Linux partition, and a spare fat32 stick.

1. Create a new directory on the Linux partition.
2. Download and extract 'mio...tar' into this directory.
3. Use 'SFS-Load on-the-fly' to load the 'mio12...sfs' as an extra sfs in the current working Puppy.
4. Using the "Puppy" facility in "FrugalPup v12", install Puppy files from an '.iso' into this directory.
5. Insert the fat32 usb stick.
6. Using the "Boot" facility in "FrugalPup v12", setup the usb stick to boot the Puppy in the directory in the Linux partition.
7. Boot the usb stick, which boots the Puppy that has just been installed in the directory in the Linux partition,
this is a first boot with "mio" active.

Note1: It is important that the "mio" files are present in the install directory before "FrugalPup v12" is run,
i.e. step 2 before step 4.

Note2: If you can't boot from a usb stick, but have a CD burner,
you could use the "ISO" facility in "FrugalPup v12" to create a CD that performs the same task as the usb stick.

Of course the same result could be done manually fairly easily,
but just make sure you pay particular attention to the "Install Directory Boot parameters" section of the 'README.txt' file.
If the install directory is not specified, the "mio" 'init' will quickly abort the boot.

The 270.1MiB download 'zipboot' option:

An 'mio12_bionicpup32_8.0.zip' is available for download from http://www.mediafire.com/folder/7y3c7eafptzao/zipboot.

1. Setup an empty fat32 usb stick with a volume label of "USBPBOOT", (without the quotes).
In Puppy, you should be able to use the CLI utility 'fatlabel' to set the volume label.
2. Download 'mio12_bionicpup32_8.0.zip' and unzip it's contents into the root of the usb stick.
3. If your machine requires a non-uefi boot, use the utilities in either /utils/lin or /utils/win to write an mbr to the stick.
4. Boot the usb stick, this is a first boot with "mio" active.

This provides the opportunity to either, explore the capabilities of this "mio" fat32 based installation,
or to use the "FrugalPup v12" in this "mio" Puppy, to install 'mio12_bionicpup32_8.0.zip' to a Linux partition.

Of course if you have a Puppy with a working "FrugalPup v12", you can use it to install 'mio12_bionicpup32_8.0.zip' however you choose.


Note - bionicpup64:
After updating with bionicpup64 fixes in 'QuickPet', you may need to delete '/usr/local/frugalpup' and '/usr/local/mi-utils' from 'mi_bionicpup64save' using another Puppy.
These v11 files in the savefolder hide the v12 files in 'mio_bionicpup64_8.0.sfs'.

Note - the version number:
Version numbers are just an integer, and have no real significance, simply a newer release will always have a bigger version number.
This includes an updated "FrugalPup" and it's last release was v11.

gyro

Last edited by gyro on Wed 15 May 2019, 18:07; edited 1 time in total
Back to top
View user's profile Send private message 
wiak

Joined: 11 Dec 2007
Posts: 1398
Location: not Bulgaria

PostPosted: Sat 04 May 2019, 02:30    Post subject: Re: Minimal Init Overlay v12 (mio) - alternate "init" and utils
Subject description: Exploring some different ways for Puppy to work.
 

gyro wrote:
This is the first release in this "Minimal Init" project.
It is a successor to the "Overlay Init" project, but the focus has changed to the different facilities it implements, rather than overlayfs.
Although this release uses overlayfs for the stack, sometime in the future, there will be a "Minimal Init Aufs" (mia) relesase.


Always good to see this kind of initiative. The greater the community's understanding of the simplicity of using overlayfs and aufs to implement complex and flexible facilities the better since leads to better Puppy design implementation. Such work helps create a more active/productive distro-building community overall IMO than simply rehashing traditional Pup builds via one build system.

wiak

_________________
Tiny Linux Blog: http://www.tinylinux.info/
makepup: http://www.murga-linux.com/puppy/viewtopic.php?p=965541
tinycore/slitaz: http://www.murga-linux.com/puppy/viewtopic.php?p=990130#990130
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1623
Location: Brisbane, Australia

PostPosted: Fri 10 May 2019, 05:58    Post subject:  

I forgot to mention that it uses "zram" for '/tmp' not "tmpfs".
If you run in pupmode=12 and boot with "pfix=nocopy", you endup with a Puppy running with no "tmpfs".

"zram" is a RAM based, compressed, pseudo block device.
So the same amount of stuff in a "zram" device, is supposed to use less RAM than if it were stored in "tmpfs".

gyro
Back to top
View user's profile Send private message 
musher0

Joined: 04 Jan 2009
Posts: 14189
Location: Gatineau (Qc), Canada

PostPosted: Sun 12 May 2019, 14:17    Post subject:  

gyro wrote:
I forgot to mention that it uses "zram" for '/tmp' not "tmpfs".
If you run in pupmode=12 and boot with "pfix=nocopy", you endup with a Puppy running with no "tmpfs".

"zram" is a RAM based, compressed, pseudo block device.
So the same amount of stuff in a "zram" device, is supposed to use less RAM than if it were stored in "tmpfs".

gyro
In other words, a Puppy.sfs not transfered to RAM, yes?
Ok. I try your approach, but I remove the nocopy setting in pfix.

On the other hand, if Puppy is not transferred in RAM, the RAM is used only for
programs. So that may be better for low-RAM systems.

For the record, zram does offer a potential 50% increase of the RAM capacity.
I know this from testing it in a RAMdisk script I produced last year.

Just a few thoughts.

BFN.

_________________
musher0
~~~~~~~~~~
Je suis né pour aimer et non pas pour haïr. (Sophocle) /
I was born to love and not to hate. (Sophocles)
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1623
Location: Brisbane, Australia

PostPosted: Mon 13 May 2019, 02:39    Post subject:  

musher0 wrote:
In other words, a Puppy.sfs not transfered to RAM, yes?
Ok. I try your approach, but I remove the nocopy setting in pfix.
Yes.
musher0 wrote:
On the other hand, if Puppy is not transferred in RAM, the RAM is used only for
programs. So that may be better for low-RAM systems.
Yes, it also boots faster, it doesn't spend time copying to RAM.
musher0 wrote:
For the record, zram does offer a potential 50% increase of the RAM capacity.
I know this from testing it in a RAMdisk script I produced last year
That's why I started playing with zram.

gyeo
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1623
Location: Brisbane, Australia

PostPosted: Wed 15 May 2019, 18:02    Post subject: Minimal Init Overlay v13 (mio) - alternate "init" and utilit  

This is mostly a bug fix release with one notable exception.

1. New utility 'MI Update frugal' has been added.
This uses an appropriate downloaded '.tar' file to update the MI files in a Puppy frugal install directory.
If the install directory contains no existing MI files, the MI is added, i.e. "installed".

2. 'mio_utils.txt' has been updated to include 'MI Update frugal'.

3. New file 'mi-update-frugal_2.sfs' is available.
This is meant to be used as an "Extra sfs" in a "normal" Puppy,
it provides the utilities 'MI Update frugal' and 'MI Boot config'.

4. Added a catch-all to 'rc.shutdown' processing to umount any stray mounted overlayfs stacks.
(Overlayfs is a useful tool for "merging" multiple directories, mount them in a stack and then process the mountpoint.)

5. No "saving" is done if the device is "ro".
This could apply if you boot from an SD card, and then "lock" by moving the slider on the side of the card.

6. 'FrugalPup' has been fixed so that it adds "pfix=trim" to the boot entry if it finds an SSD as
either an install device, or a default save device, (not just an install device).

7. 'change_kernels' has been fixed so that it doesn't produce an update during the next boot,
i.e. a pink "next boot will be faster!" message.


Downloads available from http://www.mediafire.com/folder/q04fxq43exkfr/mio:
'mio_utils.txt'
'mio13_bionicpup32_8.0.tar'
'mio13_bionicpup64_8.0.tar'
'mi-update-frugal_2.sfs'
'README.txt'

Downloads available from http://www.mediafire.com/folder/7y3c7eafptzao/zipboot:
'mio12_bionicpup32_8.0.zip___mio13_bionicpup32_8.0.zip.delta'
'mio13_bionicpup32_8.0.zip'

Downloads available from http://www.mediafire.com/folder/rdyc5lgzpeij1/frugalpup:
'frugalpup-13.pet'
'frugalpup-13.sfs'


Installing mio v13:

The 3.8MiB + 8.0KiB download option:

Like the previous release, each '.tar' contains 2 files, an 'mio_initrd.gz' and an 'mio13...sfs'.
But now we can take advantage of the new 'MI Update frugal' utility.

This method requires a working Puppy, some space on a Linux partition, and a usb stick (optional).

1. Either, do a fresh frugal install of 'bionicpup32_8.0' or 'bionicpup64_8.0', or use an existing one.
2. Download the appropriate 'mio13...tar' file and copy it to this install directory.
3. In a "normal" Puppy, download 'mi-update-frugal_2.sfs' and load it with 'SFS-Load on-the-fly'.
3. From the menu, run "Setup->MI Update frugal", and select the 'bionicpup' install directory from step 2.
4 . When 'MI Update frugal' is "done", the install directory is ready to do a first boot with "mio" active.

But, the "mio" 'init' will quickly abort the boot, if the install directory is not specified in boot parameters.
(See the "Install Directory Boot parameters" section of the 'README.txt' file for more detail.)

Either
5a. Edit the relevent boot entry to ensure that it includes "pdrv=<partition>" and "psubdir=</relative/path/to/install/directory>" boot paramaters, and reboot.
or
5b. Using the "Boot" facility in "FrugalPup", setup a usb stick to boot the Puppy in the install directory, and boot the usb stick.

Of course the same result could fairly easily be done manually.

The 27.0MiB download 'zipboot' option:

If you already have a copy of 'mio12_bionicpup32_8.0.zip',
download 'mio12_bionicpup32_8.0.zip___mio13_bionicpup32_8.0.zip.delta'.
Use that to create an 'mio13_bionicpup32_8.0.zip', then do the following process from step 2.

The 270.1MiB download 'zipboot' option:

1. Download 'mio13_bionicpup32_8.0.zip'.

2. Setup an empty fat32 usb stick with a volume label of "USBPBOOT", (without the quotes).
In Puppy, you should be able to use the CLI utility 'fatlabel' to set the volume label.
3. Unzip the contents of 'mio13_bionicpup32_8.0.zip' into the root of the usb stick.
4. If your machine requires a non-uefi boot, use the utilities in either /utils/lin or /utils/win to write an mbr to the stick.
5. Boot the usb stick, this is a first boot with "mio" active.

This provides the opportunity to either, explore the capabilities of this "mio" fat32 based installation,
or to use the "FrugalPup v13" in this "mio" Puppy, to install 'mio13_bionicpup32_8.0.zip' to a Linux partition.

Of course if you have a Puppy with a working "FrugalPup v13", you can use it to install 'mio13_bionicpup32_8.0.zip' however you choose.


Note - bionicpup64:
After updating with bionicpup64 fixes in 'QuickPet', you may find that the MI utilities show as v11.
This is because bionicpup64 fixes include "FrugalPup v11".
This can be fixed by downloading 'frugalpup-13.pet' and installing it.

Note - bionicpup32:
It can still be updated from '.delta' files just like a "normal" bionicpup32.

gyro

Last edited by gyro on Sat 18 May 2019, 10:04; edited 3 times in total
Back to top
View user's profile Send private message 
wiak

Joined: 11 Dec 2007
Posts: 1398
Location: not Bulgaria

PostPosted: Thu 16 May 2019, 20:59    Post subject:  

Hi gyro,

Just tried your mio v.13 quickly on my existing bionicpup64 frugal install. Only booted but haven't checked your initramfs/init out but note you've made it much smaller than traditional Puppy init which is a good thing. One result of that cut was probably that my first attempt to boot mio bionicpup is that it failed owing to couldn't find pdrv since I hadn't specified that in my original bionicpup grub2 boot config. Presumably the larger traditional pup init searches for relevant psubdir and finds the partition without needing pdrv specified. But I agree with you that such added init complexity isn't a good thing when a simple boot parameter pdrv can be used to avoid need of extra init search code.

Also noted you prefer new boot param names psfspart and psfsdir instead of pdrv and psubdir presumably because parts of pup boot filed don't need to be kept in pdrv/psubdir so psfs___ more clearly specifies the relevant sfs storage location. I see you made that backwards compatible which is just as well since most are so used to using psubdir (but not so used to specifying pdrv I expect, so I'd suggest you really emphasise that psfspart requirement. Yes, its in the README but like most I read too briefly hence my recommending even more early emphasis since some may just give up if boot fails).

Congratulations however in the overall quality of your early documentation. It is generally very clear, detailed, and nicely written. My only small suggestion re that is that the steps for installation could do with being presented at the top of a separate post since these steps are nice and simple but buried inside longer text. But that'd a minor issue comment caused by my own bad habit to skim text and not notice some important details.

I'm too busy on my own dev workvat the moment to check out your init script, but I understand from your operation description what is basically in there at present in terms of how you are using/setting up your overlayfs layers.

Anyway, I feel that mio simplified init (more done instead in external-to-initramfs userspace utils) will be a nice improvement in future Puppy releases.

wiak

_________________
Tiny Linux Blog: http://www.tinylinux.info/
makepup: http://www.murga-linux.com/puppy/viewtopic.php?p=965541
tinycore/slitaz: http://www.murga-linux.com/puppy/viewtopic.php?p=990130#990130

Last edited by wiak on Fri 17 May 2019, 03:37; edited 1 time in total
Back to top
View user's profile Send private message 
musher0

Joined: 04 Jan 2009
Posts: 14189
Location: Gatineau (Qc), Canada

PostPosted: Thu 16 May 2019, 22:02    Post subject:  

Hi, gyro.

I know next to nothing about this initial step of booting a Puppy, but I have a hunch
that you and your utility could give a much needed helping hand to those of us ( me,
certainly! ) who are trying to build new Pups with jamesbond's woof-Next.

No obligation of course, but I think there would perhaps be great potential if the two
projects were in sync, with your "mio" alternative initrd and distro loader as the
"locomotive". ( Please excuse the comparison! )

TIA, whatever you decide.

_________________
musher0
~~~~~~~~~~
Je suis né pour aimer et non pas pour haïr. (Sophocle) /
I was born to love and not to hate. (Sophocles)
Back to top
View user's profile Send private message 
musher0

Joined: 04 Jan 2009
Posts: 14189
Location: Gatineau (Qc), Canada

PostPosted: Thu 16 May 2019, 22:14    Post subject:  

Oops. Double. Sorry.
_________________
musher0
~~~~~~~~~~
Je suis né pour aimer et non pas pour haïr. (Sophocle) /
I was born to love and not to hate. (Sophocles)
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1623
Location: Brisbane, Australia

PostPosted: Sat 18 May 2019, 09:53    Post subject:  

@wiak,

My mistake, I forgot that in using only "MI Update frugal" to do the upgrade, (unlike v12, which depened on "FrugalPup v12") the boot entry remained untouched.

I have made some quick modifications to the post, which hopefully makes things clearer.

In hindsight I probably should include "FrugalPup" in "mi-update-frugal" and have the instructions boot a usb stick.

gyro.
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1623
Location: Brisbane, Australia

PostPosted: Mon 20 May 2019, 02:57    Post subject:  

@musher0,
Thanks, but no thanks.

gyro
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1623
Location: Brisbane, Australia

PostPosted: Mon 20 May 2019, 03:00    Post subject: 'mi-update-frugal_3.sfs' - modified install instructions  

I have uploaded 'mi-update-frugal_3.sfs' to http://www.mediafire.com/folder/q04fxq43exkfr/mio.
It now contains 'FrugalPup v13'.

The modified install instructions follow:

Installing mio v13:

The 3.8MiB + 2.6MiB download option:

This method requires a working Puppy, some space on a Linux partition, and an empty fat32 stick.

1. Either, do a fresh frugal install of 'bionicpup32_8.0' or 'bionicpup64_8.0', or use an existing one.
2. Download the appropriate 'mio13...tar' file and copy it to this install directory.
3. In a "normal" Puppy, download 'mi-update-frugal_3.sfs' and load it with 'SFS-Load on-the-fly'.
3. From the menu, run "Setup->MI Update frugal", and select the 'bionicpup' install directory from step 2.
4. Wait until you get a "done" message from 'MI Update frugal'.
5. Insert the fat32 usb stick.
6. From the menu, run "Setup->FrugalPup - Puppy frugal installer", and select the "Boot" facility,
setup the usb stick to boot the 'bionicpup' install directory from step 2.
7. Boot the usb stick, which boots the Puppy that has just been installed in the directory in the Linux partition,
this is a first boot with "mio" active.

Of course the same result could be done manually fairly easily,
but just make sure you pay particular attention to the "Install Directory Boot parameters" section of the 'README.txt' file.
If the install directory is not specified, the "mio" 'init' will quickly abort the boot.

It's probably a good idea to use the usb stick, even if you intend to, or do already,
boot the selected install directory using a different boot device.
The generated boot-entry on the usb stick can be used as a template for your real boot-entry.

gyro
Back to top
View user's profile Send private message 
Rangan Masti

Joined: 01 Jan 2013
Posts: 38
Location: Germany, Berlin

PostPosted: Sun 26 May 2019, 14:01    Post subject:  

Hi, I am using your Mio on LxpupSc64 and testing it since a week.
Thank you and have a great day.
Rangan.
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1623
Location: Brisbane, Australia

PostPosted: Wed 29 May 2019, 05:16    Post subject:  

Rangan Masti wrote:
Hi, I am using your Mio on LxpupSc64 and testing it since a week.
Interesting, I have not tested "mio" on LxpupSc64.
Did you use 'mio13_bionicpup64_8.0.tar'?

gyro
Back to top
View user's profile Send private message 
Rangan Masti

Joined: 01 Jan 2013
Posts: 38
Location: Germany, Berlin

PostPosted: Wed 29 May 2019, 09:57    Post subject:    

Hi, yes I user your mio13_bionicpup64_8.0
with some changes manually done.
Thank you and have a great day.

Rangan.
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [15 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1650s ][ Queries: 11 (0.0136s) ][ GZIP on ]