Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Mon 20 Jan 2020, 16:52
All times are UTC - 4
 Forum index » Advanced Topics » Puppy Projects
FirstRib default WeeDog Linux build system
Moderators: Flash, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 17 of 49 [725 Posts]   Goto page: Previous 1, 2, 3, ..., 15, 16, 17, 18, 19, ..., 47, 48, 49 Next
Author Message
wiak

Joined: 11 Dec 2007
Posts: 1861
Location: not Bulgaria

PostPosted: Tue 06 Aug 2019, 16:38    Post subject:  

rufwoof wrote:
Seeing rockedge's posts got me wondering if perhaps two versions of the sstem were running concurrently, one via runit and another via init.


I'll look into it. I suspect, like I said, just startx not finding .xinitrc when avoiding login process, but I may be wrong. Starting x from rc.local is probably not a usual practice - like I suggested, modifying the agetty start line would be the way to go normally if you want root auto logged in.

The switch_root code simply runs /sbin/init, which in void filesystem structure is a symlink to /usr/bin/init, which itself is a symlink to EITHER (but not both) busybox init or if runit installed then the symlink is automatically changed to runit-init. So both cannot be happening at the same time. And once runit package installed that symlink always points to runit-init so busybox init and the associated rc.sysinit is simply not involved. Never say never... but I say never on this occasion Smile

I would need more details about gparted. Or does it simply not work in all configurations? I will try it sometime. Really, the system arrangement is so simple most issues should be easy to address. The most powerful feature, IMO, is simply being able to add any directory you like to a middle layer (hence the way NNupper_changes can be used, rather than having to make it into an sfs at first, but the code that sorts that out is simple too.

I've also used the power of that dir-into-layer-code facility to simply load an official Void Linux graphical-based root filesystem (that's the desktop image I posted some time back) and that worked fine too (that uses 'Slim' login manager I think). I have a traditional official full install of Void Linux itself on its own partition, so all I did was copy that into a directory in /mnt/bootpartition/firstrib and then replaced its modules and firmware with those of the newer kernel I was using in FirstRib, and the result booted perfectly with WeeDog initramfs04.gz. So with FirstRib you can work with official Void images, or build something much smaller or honed to whatever underlying root filesystem you wish. The result, though expected, was nevertheless almost amazing to me - it is not usual to be able to take what is actually a full install and change it so simply into a frugal install... exciting watching it boot up (since, despite a feeling it should work, still half-expecting a dreaded kernel crash or switch_root failure).

So that Void Linux full-install being so easily transformed into a WeeDog frugal install working suggests all fine in terms of full compatability (though there may be glitches I don't yet know about of course). Funny thing is, I hadn't even tried using runit prior to that full to frugal boot 'experiment'... I just couldn't see any reason why it wouldn't work (/sbin/init ending up symlinking to runit-init), and, as it turns out, it did work...

wiak

_________________
New Puppy/Dog forum: https://puppylinux.rockedge.org
Tiny Linux Blog: https://www.tinylinux.info/
Check Firmware: http://murga-linux.com/puppy/viewtopic.php?p=1022797
Back to top
View user's profile Send private message 
wiak

Joined: 11 Dec 2007
Posts: 1861
Location: not Bulgaria

PostPosted: Wed 07 Aug 2019, 17:01    Post subject:  

wiak wrote:
... I am currently working on providing a [EDIT] 'changes' in RAM only mode, which is no-persistence mode (but can, if wished, usefully use previous upper_changes directory, by simply renaming that to NNupper_changes, for example 90upper_changes as one of the read-only middle layers. Note that I am not myself interested in a copy2ram mode (sfs copying to RAM), which simply uses too much RAM with little end-benefit in my opinion; Linux does a pretty good job of caching anyway...

At the same time, I'm also implementing a read-only mode, in which the merged overlay result is entirely read-only (so cannot be written to) - that mode is good for making remasters.

And also a mode that allows the upper_changes directory to be stored anywhere (rather than default /mnt/bootpartition/bootdir).

The additional code for these particular alternatives is very simple and if no new mode is specified on grub linux/kernel line then default current /mnt/bootpartition/bootdir/upper_changes will continue to be used.


I have the above working in initramfs/init (just need to bolt it into the build initramfs script now).

Simple it was in terms of code needed (around 20 new lines of code in one block plus some overall code reorganisation), but came with xino issue. Not a lot of docs on that since all happening at kernel level so complete understanding would require studying kernel overlayfs C code (a difficult task indeed). I found working solutions but on try and see basis of course.

EDIT: Regarding xino (which can effect matters on overlayfs arrangements, where different filesystems being used): https://www.gnu.org/software/libc/manual/html_node/Attribute-Meanings.html
Quote:
When you read the attributes of a file, they come back in a structure called struct stat.
... amongst various attributes, stat contains:

ino_t st_ino: The file serial number, which distinguishes this file from all other files on the same device.

dev_t st_dev: Identifies the device containing the file. The st_ino and st_dev, taken together, uniquely identify the file. The st_dev value is not necessarily consistent across reboots or system crashes, however.

I've found that having overlay mount option xino translation table on, mucked things up when upper_changes directory placed on different filesystem to underlying firstrib_rootfs, so needed mount option xino=off.

EDIT: As far as copy ALL to RAM feature, which is not the same thing as changes only to RAM since copy all to RAM would include any NNfilesystem (sfs or uncompressed), I'm still thinking of whether to implement that in default initramfs/init or as a plugin. Each new addition does bloat the build script more than I like, but to make it easier to read I continue for the main part to avoid functions so buildscript can be read in nice linear fashion without having to jump back and forwards in the script to see what each function does - the only exception being the _rdsh plug in (or debug shell function), since that is repeatedly used in the code.

A copy ALL to RAM mode, as I understand it (though personally almost never use it because it is slow and my usual dev system has only 2GB RAM sadly - yes I should buy an old extra RAM stick for it...), is just a matter of mounting a tmpfs (or using an existing one) and if copy2ram requested, cp all the NNfilesystems to that tmpfs and then mount them from there rather than from the original media - uses a fair amount of RAM (especially when the NNfilesystem is an uncompressed directory) but I guess a lot of systems having plenty RAM to spare nowadays and on smaller RAM systems user can just arrange things so very few sfs files and only very small uncompressed filesystems used (or don't use copy2ram at all since optional) - anyway copy2ram not done yet.

wiak

_________________
New Puppy/Dog forum: https://puppylinux.rockedge.org
Tiny Linux Blog: https://www.tinylinux.info/
Check Firmware: http://murga-linux.com/puppy/viewtopic.php?p=1022797

Last edited by wiak on Thu 08 Aug 2019, 03:06; edited 16 times in total
Back to top
View user's profile Send private message 
rockedge


Joined: 11 Apr 2012
Posts: 1450
Location: Connecticut, United States

PostPosted: Wed 07 Aug 2019, 17:38    Post subject:  

I can post different htop readouts from this version to demonstrate ->

Back to top
View user's profile Send private message Visit poster's website 
wiak

Joined: 11 Dec 2007
Posts: 1861
Location: not Bulgaria

PostPosted: Wed 07 Aug 2019, 18:44    Post subject:  

Just a comment, now that the FirstRib WeeDog system is becoming more 'mature'.

There are basically two ways of documenting enhanced builds that work with FirstRib build system:

1. A wrapper script that runs FirstRib scripts and does extra stuff around that result (so it is an external methodology). That is similar to what Fredx181 does with his mklive-stretch and similar builds using debootstrap internally and then doing a chroot inside the wrapper to add new applications or whatever.

2. Using builld_firstrib_rootfs firstrib00.plug plugin facility (and the build rootfs script also allows for alternative plugin names). That is an internal methodology. The difference is that it leverages the fact that build_firstrib_rootfs script already uses a chroot so no external chroot wrapper required. Whatever code gets added to the plugin (e.g. xbps-install some-new-package... etc etc etc) gets done inside the chroot build of the main build_firstrib_rootfs script itself. Where the same 'recipe' can be providided by a plugin rather than a wrapper, then that would be more efficient or at least neater. Having said that, either can be used (FirstRib flexibility) and sometimes, for other reasons, a combination of using both a wrapper script and a firstrib plugin may prove useful.

And the build_firstrib_initram script also has its rdsh.plug plugin facility, which effectively allows a user to plugin their own init for the initramfs to use (or some extra init code for insertion into the default init build). To some/large extent, that removes some of the possible development burden from myself, since a user/developer can modify the resulting system for their own wishes/requirements without needing the default released build scripts themselves altered (and a library of user-plugins could be usefully created for use by others).

And Void Linux repos/xbps/runit system removes an even greater burden from the maintenance needs of FirstRib Weedog since Void provides the rolling distro mechanism out-of-the-box. So, as long as Void has a long lifetime (and Void Linux upstream auto-compiling-build system should now ensure that) then so too does WeeDog - should be able to rely on it for years... unlike systems that continually need new builds every year or two in order to keep up with the times. Result is less time wasted trying to keep system up to date, or endlessly reconfiguring it or replacing it: gives more time to just sit back, drink coffee, and enjoy the system. And, with suitable wrapper/plugin combinations, rebuilds become just a one-click operation anyway (and often only a relatively few minutes to build to completion).

Last, but not least, the plugin/wrapper facilities allow a user/developer the satisfaction of building their own complex-as-they-wish specialised system, which can then be published as either a 'recipe' or, via a simple script, an iso or similar for others to use.

Actually, the very last, perhaps the raison d'être for FirstRib's design, is that the simplicity of the scripts hopefully makes them educational - re: how such systems work... (including for myself since I forget how to do things after a few months so can just re-read the scripts to remind me).

wiak

_________________
New Puppy/Dog forum: https://puppylinux.rockedge.org
Tiny Linux Blog: https://www.tinylinux.info/
Check Firmware: http://murga-linux.com/puppy/viewtopic.php?p=1022797
Back to top
View user's profile Send private message 
rockedge


Joined: 11 Apr 2012
Posts: 1450
Location: Connecticut, United States

PostPosted: Wed 07 Aug 2019, 20:55    Post subject:  

first example is the initial boot with /etc/rc.local starting a script for network connection and startx



http://rockedge.org/images2/2019-08-07-184049_1680x1050_scrot.png


this second screenshot shows the system after exiting X and logging in at the void-login: prompt and using startx from the shell ->



http://rockedge.org/images2/2019-08-07-183751_1680x1050_scrot.png
Back to top
View user's profile Send private message Visit poster's website 
wiak

Joined: 11 Dec 2007
Posts: 1861
Location: not Bulgaria

PostPosted: Wed 07 Aug 2019, 23:08    Post subject:  

rockedge wrote:
first example is the initial boot with /etc/rc.local starting a script for network connection and startx

...


this second screenshot shows the system after exiting X and logging in at the void-login: prompt and using startx from the shell ->


Seems to be running efficiently, but interesting how the jwm Rox pinboards are different, or did you set it up that way somehow? Certainly doesn't look like anything to worry about - might even be a useful anomaly.... Since using same dirs to call xinitrc I can only assume login process calls up some configuration difference - would have to trace/check through that. EDIT: seems to simply be runit itself:

EDIT:
rc.local wrote:

# This is run by runit in stage 2 before the services are executed
# (see /etc/runit/2).


You can see when rc.local gets used in /etc/runit/2 shell script; so when using that to startx, you are bypassing "runsvdir -P /run/runit/runsvdir/current", which means starting X without agetty's being run and all the services in /etc/runit/runsvdir/default. I'm not sure how that effects your Rox pinboard though.

wiak

Void Linux /etc/runit/2 code:

Code:
#!/bin/sh
# vim: set ts=4 sw=4 et:

PATH=/usr/bin:/usr/sbin

runlevel=default
for arg in $(cat /proc/cmdline); do
    if [ -d /etc/runit/runsvdir/"$arg" ]; then
        echo "Runlevel detected: '$arg' (via kernel cmdline)"
        runlevel="$arg"
    fi
done

[ -x /etc/rc.local ] && /etc/rc.local

runsvchdir "${runlevel}"
mkdir -p /run/runit/runsvdir
ln -s /etc/runit/runsvdir/current /run/runit/runsvdir/current

exec env - PATH=$PATH \
    runsvdir -P /run/runit/runsvdir/current 'log: ............


Perhaps you've seen this reddit void forum post, or maybe of interest regarding starting up alternative desktop managers:

https://www.reddit.com/r/voidlinux/comments/apej53/problems_with_startx_and_startxfce4/

wiak

_________________
New Puppy/Dog forum: https://puppylinux.rockedge.org
Tiny Linux Blog: https://www.tinylinux.info/
Check Firmware: http://murga-linux.com/puppy/viewtopic.php?p=1022797
Back to top
View user's profile Send private message 
wiak

Joined: 11 Dec 2007
Posts: 1861
Location: not Bulgaria

PostPosted: Thu 08 Aug 2019, 07:07    Post subject:  

Progress update for build_firstrib_initramfs04 Revision 0.0.5

I have now implemented code to handle xino (on or off) as well as allowing for user-forced xino=on or xino=off via the grub kernel line. All overlay/aufs constructions have a problem with these inode effects (different inodes for files on layer being copied up or down to another layer). Generally, we want xino=on since that causes overlay translation operations that generally make all work correctly (i.e. same as if all files on same filesystem). However, when the filesystems are different, switching xino off is often necessary to get things basically working. I say 'basically working' because situations could arise (when removing or copying files) when inode change effects could mix things up (hopefully rarely); anyway, I've provided that grub linux/kernel xino=? argument so that user can force xino should that prove necessary (and provided sensible default values depending on changes mode being used). Same issues apply to systems that use aufs by the way.

Aside from handling xino, the three new changes modes (in addition to changes stored in /mnt/bootpartition/bootdir/upper_changes) have now been implemented and tested per my earlier post:

http://www.murga-linux.com/puppy/viewtopic.php?p=1033898#1033898

I therefore expect to post Revision 0.0.5 with these additions sometime tomorrow (it's late night here now).

In tests, I'm finding the 'rollback' feature very useful (i.e. renaming current upper_changes directory to say 50upper_changes (so it automatically becomes one of the read-only middle layers on next boot, even though it is an uncompressed directory - but could also be used in a squashedfs form) and the upper_changes that occurs next time round can then be renamed to 51upper_changes, and so on (I expect there are penalties of some sort or other if you involve too many layers though...). Using read-only NNupper_changes like that can also then be used with the new mode: changes=RAM (which is not the same as copy2ram or pfix=RAM; rather changes=RAM just means no persistence (since any changes are simply occuring in RAM in that mode).

Note that you can, however, manually make a copy of the /mnt/layers/RAM directory (which contains the changes in tmpfs) to a mounted persistent directory and then rename it to, for example, 52upper_changes if you decide you actually want to keep it...

Per earlier post description you will now also be able to store subdirectory upper_changes in a different filesystem partition/directory altogether from the one being booted from. For that to work I found I needed to make xino=off, though there may be occasions, with different filesystem types where xino=on will still work (which would be preferred) in which case you can force that on grub linux/kernel line.

I'll be looking into a full copy2ram mode for the version after that (either as a plugin or a new build script depending on complexity, though the required process isn't too difficult to implement in FirstRib because of the way the NNlayers are coded so might not be overly-bloating in practice).

wiak

EDIT: Have finished putting new rev 0.0.5 init into build_firstrib_initramfs04_005 script ready for final testing/tweaks. Going out now but shall do final testing and likely upload about 12 hours from now.

_________________
New Puppy/Dog forum: https://puppylinux.rockedge.org
Tiny Linux Blog: https://www.tinylinux.info/
Check Firmware: http://murga-linux.com/puppy/viewtopic.php?p=1022797

Last edited by wiak on Thu 08 Aug 2019, 18:53; edited 1 time in total
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3711

PostPosted: Thu 08 Aug 2019, 16:44    Post subject:  

With firstrib (Void kernel) booted, I installed/added enough to compile the kernel (gcc make bc bison flex perl elfutils-devel elfutils libressl-devel) and compiled the 4.19.44 kernel (same as what I'm using in Fatdog) with the localyesconfig ... so all firmware/modules are built into the kernel. Expanded the kernel from around 7MB to 13MB, but its booting/running fine (with no firmware or modules, i.e. just what was built into the kernel), chrome playing youtubes etc. all OK. Opening the main sfs and removing firmware/modules from that knocks it down around 300MB (gzip'd size reduction). But that is bespoke to my laptop (you have to have the usb and wifi connected when you build with localyesconfig).

As a basic kernel + busybox setup its around 14MB total - with wifi connect etc. capabilities (but just for my laptop/hardware).

_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
wiak

Joined: 11 Dec 2007
Posts: 1861
Location: not Bulgaria

PostPosted: Thu 08 Aug 2019, 17:45    Post subject:  

rufwoof wrote:
compiled the 4.19.44 kernel (same as what I'm using in Fatdog) with the localyesconfig ... so all firmware/modules are built into the kernel. Expanded the kernel from around 7MB to 13MB


That's a nice idea for a local dedicated system. In my house, the whole family (four of us) use the same laptops, which I picked cheap (two without harddrives) several years ago. Such a kernel compile very useful in such a case (the kids two computers run from 16GB SD cards).

wiak

_________________
New Puppy/Dog forum: https://puppylinux.rockedge.org
Tiny Linux Blog: https://www.tinylinux.info/
Check Firmware: http://murga-linux.com/puppy/viewtopic.php?p=1022797
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3711

PostPosted: Thu 08 Aug 2019, 20:21    Post subject:  

Rolling releases are nice is some respects, get the latest security updates for instance, but periodically you also get the latest bugs that were overlooked, sometimes critical bugs. Fatdog is somewhat like Void, Fatdog being based on Linux From Scratch, but with Fatdog its fixed, for historically around a year between releases, so that can be more refined/tweaked to be more stable. With Void to install another package you pretty much have to update the whole, which introduces the risk of breakage. One way might be to set up your own repo, a local build server, and stick with a single Void source snapshot instance,somewhat similar to Fatdog. Even then however Fatdog has the edge as more eyes have looked at more things all pulling and working together well. But with that you tend to run with older (tried and tested) versions of programs that relatively infrequently get updated (yearly) and maybe not even then.

Void is a bit of fun for me, but overall I prefer the Fatdog model as my main boot choice, it just works, works well and is unchanging. usb booting and not saving at the end of a session pretty much guarantees that if it booted yesterday, it will boot exactly the same again tomorrow. Reliable and consistent. The Fatdog team do expend a massive amount of effort to get to each release however. What might be a simpler alternative choice could be to use Void as the base/source, rather than sourcing more directly from each individual programs source. Each Fatdog program includes the build instructions internally, source location, compile/build method etc. I had intended to look into sourcing Fatdog build scripts using a snapshot of Void source, but in my latest build I forgot to include aufs support (doh!), so I've now put that idea onto the backburner. I haven't actually looked yet, but it sounds like Void uses a similar concept to Fatdog i.e. Void's runit modularisation.

_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
wiak

Joined: 11 Dec 2007
Posts: 1861
Location: not Bulgaria

PostPosted: Fri 09 Aug 2019, 02:18    Post subject:  

rufwoof wrote:
Void is a bit of fun for me, but overall I prefer the Fatdog model as my main boot choice, it just works, works well and is unchanging. usb booting and not saving at the end of a session pretty much guarantees that if it booted yesterday, it will boot exactly the same again tomorrow. Reliable and consistent.


No doubt about how good FatDog is - I've used it and was impressed (more by quite a lot than by Puppy itself - though that was great in its day). WeeDog isn't exactly Void Linux of course - Void proper is a full install distro - it's live cd is only for installation purposes so doesn't have frugal install modes and so on. FirstRib build initramfs fills in that gap and FirstRib build rootfs allows user to build up either a full Void Linux root filesystem or something much less or could be made something entirely different altogether (tho I'm all for Void runit and so on). By renaming a built up WeeDog's upper_changes directory into a middle overlayfs layer (by simply renaming it with a two-digit number (NNupper_changes) and then using new non-persistent mode (being provided in about to be uploaded build initramfs Revision 005) the system will always boot exactly the same each time. But will also be able to save the RAM contents if changes wanting to be kept for next time (via creating yet another, different, NNupper_changes rollback folder. I haven't as yet however created any 'merge' upper_changes utility/functionality, and may or may not ever get round to that).

But yes, WeeDog (for me) is also just for personal fun (and its FirstRib build system has only been under development for just over two months... so comparisons are a bit apple/orangelike, though I have been working on it relatively intensely for that short duration). I never had idea to build it up into all-singing/dancing polished distro (never been my interest) though it could be done and maybe someone else will end up doing that. Good thing is the build scripts are simple and open to view and, hopefully, in a form that is easy to modify for whatever purpose anyone might want.

wiak

_________________
New Puppy/Dog forum: https://puppylinux.rockedge.org
Tiny Linux Blog: https://www.tinylinux.info/
Check Firmware: http://murga-linux.com/puppy/viewtopic.php?p=1022797
Back to top
View user's profile Send private message 
wiak

Joined: 11 Dec 2007
Posts: 1861
Location: not Bulgaria

PostPosted: Fri 09 Aug 2019, 08:21    Post subject:  

build_firstrib_initramfs04 Revision 0.0.5 uploaded to thread first post:

http://www.murga-linux.com/puppy/viewtopic.php?t=116212

I have extracted the usage/docs information from the script because it is becoming harder to maintain (and needs some work). For now, the main somewhat rough usage information can be found in build_firstram_initramfs04_s005_README, which is also provided for download from thread first post.

****NEW (in Revision 0.0.5):

Code:
On grub linux/kernel line you can now specify changes save location:
There are four alternative "changes=" modes: empty arg, changes=RAM, changes=readonly, changes="path2dir"
1. No changes argument on grub kernel line: Use upper_changes in /mnt/bootpartition/bootdir
2. RAM: All changes go to RAM only (/mnt/layers/RAM/upper_changes). i.e. non-persistent
3. readonly: overlay filesystem is rendered read only so it cannot be written to at all
4. path2dir: store upper_changes in specified path/directory at upper_changes subdirectory


Code:
Note: if any of rdsh0 and/or rdsh1 rdsh2 rdsh3 are specified on grub linux/kernel
line the init looks for file /mnt/bootpartition/bootdirectory/rdshN.plug and
sources that if not empty. Otherwise it runs busybox job control shell at
rdshN code position in initramfs/init script.

Example: grub4dos that specifies storing upper_changes directory on a different partition:

   title FirstRib (Void Linux Flavour)
   root (hd0,0)
   kernel /firstrib/vmlinuzX.XX usbwait=12 bootfrom=/mnt/sda1/myfirstrib changes=/mnt/sda2/firstrib2
   initrd /firstrib/initramfs04.gz
   
Can also use:

changes=RAM (for no persistence).
changes=readonly (for a readonly system, so no writes allowed).
Leaving changes out altogether results in /mnt/bootpartition/bootdir/upper_changes being used.


There is lot to test. I've tested as best I can for now, but more testing is required.

wiak

_________________
New Puppy/Dog forum: https://puppylinux.rockedge.org
Tiny Linux Blog: https://www.tinylinux.info/
Check Firmware: http://murga-linux.com/puppy/viewtopic.php?p=1022797
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3711

PostPosted: Sat 10 Aug 2019, 07:09    Post subject:  

Hi wiak
wiak wrote:
WeeDog isn't exactly Void Linux of course - Void proper is a full install distro - it's live cd is only for installation purposes so doesn't have frugal install modes and so on. FirstRib build initramfs fills in that gap and FirstRib build rootfs allows user to build up either a full Void Linux root filesystem or something much less or could be made something entirely different altogether (tho I'm all for Void runit and so on). By renaming a built up WeeDog's upper_changes directory into a middle overlayfs layer (by simply renaming it with a two-digit number (NNupper_changes) and then using new non-persistent mode (being provided in about to be uploaded build initramfs Revision 005) the system will always boot exactly the same each time. But will also be able to save the RAM contents if changes wanting to be kept for next time (via creating yet another, different, NNupper_changes rollback folder. I haven't as yet however created any 'merge' upper_changes utility/functionality, and may or may not ever get round to that).

My one click build wrapper script now builds a development void (make, gcc ..etc). Pretty much a full (non gui) void, that boots from usb and all runs in ram Smile. I've been compiling kernel versions with that, but so far haven't got Void sources to build (my ignorance/learning) in order to build all/any sources instead of xbps-install'ing pre-build binaries. Void's recipes are similar to Fatdog's from what I've casually inspected.

One of the changes I made was to use chroot inside initrd instead of switch_root, which means that at any time I can re-enter the initrd layer. The main system doesn't see any of the layering, just the merged folder (of upper_changes ...etc.). So back in initrd I could copy/change the upper_changes i.e. perhaps unmount the existing overlay and mount a new version of that before chroot'ing back into the main (Void) level.

For where you might not want to be able to chroot out of the main session back into initrd, then that could be invoked with the drop chroot capability. But that would also mean that you wouldn't be able to use chroot at all within the main system

I'm at a stage now where I might swap over from using ext3 to using btrfs, as that way multiple drives could be merged into a single btrfs volume and use that volume as the chroot target (with stacked/multiple usb's in mind). With btrfs you can create or rearrange snapshots i.e. is pretty much a copy-on-write and rsync type manager.

As the initialisation is pretty much a lock-in while :; do setsid cttyhack /bin/sh; done type loop I'm using a short C code snippet to break out of chroot back into initrd (otherwise 'exit' just keeps you within the chroot).

Code:
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>

int main(int argc, char** argv) {
    int dir_fd, x;
    setuid(0);
    mkdir(".delme", 0755);
    dir_fd = open(".", O_RDONLY);
    chroot(".delme");
    fchdir(dir_fd);
    close(dir_fd);
    for(x = 0; x < 1000; x++) chdir("..");
    chroot(".");
    return execl("/bin/busybox", "sh",argv[1], NULL);
}

I have had to extend the one click build process to pause to ask for a Void root password as part of its run, as otherwise with a more full Void being built you end up at the login prompt where the default root/void-linux userid/password doesn't work, at least not for me. I did try setting up auto-login (under /etc/sv/...) but I couldn't get that working.

The end of my version of your build...ramfs... script looks something like (I've removed much of the xbps-install program list to shorten down that line)
Code:
# bind mount host virtual filesystes required for chroot into firstrib_rootfs to work on host system
mkdir -p firstrib_rootfs/run/udev
mount --bind /proc firstrib_rootfs/proc && mount --bind /sys firstrib_rootfs/sys && mount --bind /dev firstrib_rootfs/dev && mount -t devpts devpts firstrib_rootfs/dev/pts && mount --bind /tmp firstrib_rootfs/tmp && mount --bind /run/udev firstrib_rootfs/run/udev && cp /etc/resolv.conf firstrib_rootfs/etc/resolv.conf

echo "#!/bin/sh" >firstrib_rootfs/doit.sh
echo "xbps-install -y linux4.19 base-system ... etc ... " >>firstrib_rootfs/doit.sh
echo "rm /var/cache/xbps/*" >>firstrib_rootfs/doit.sh
echo "echo Set root password" >>firstrib_rootfs/doit.sh
echo "touch /etc/shadow" >>firstrib_rootfs/doit.sh
echo "chmod 000 /etc/shadow" >>firstrib_rootfs/doit.sh
echo "passwd" >>firstrib_rootfs/doit.sh
echo "rm /doit.sh" >>firstrib_rootfs/doit.sh # Don't leave as a remnant
echo "exit" >>firstrib_rootfs/doit.sh
chmod +x firstrib_rootfs/doit.sh

# chroot into and run the above script
chroot firstrib_rootfs /doit.sh

_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
wiak

Joined: 11 Dec 2007
Posts: 1861
Location: not Bulgaria

PostPosted: Sat 10 Aug 2019, 08:29    Post subject:  

rufwoof wrote:

One of the changes I made was to use chroot inside initrd instead of switch_root, which means that at any time I can re-enter the initrd layer. The main system doesn't see any of the layering, just the merged folder (of upper_changes ...etc.).


build_firstrib_initramfs version 01 up to revision 0.0.8 all used chroot rather than switch_root (indeed much of the development was done with these chroot rather than switch_root inside initramfs versions), much as you describe. For example:

https://github.com/firstrib/firstrib/blob/master/build_firstrib_initramfs01_ver006.sh

Actually, I've been a bit re-miss and not uploading to the github site since then. I think the last version01 build initramfs scripts reached revision 0.0.8pre (still using chroot) - all versions were uploaded to the forum but I've removed them as things progressed to avoid confusion. I'll look that last chroot initramfs version out and push it up to the github site now in case someone else finds chroot method useful. EDIT: here it 008pre chroot version:

https://github.com/firstrib/firstrib/blob/master/build_firstrib_initramfs01_ver008pre.sh

Personally, I want to stick with switch_root version development now.

rufwoof wrote:
I did try setting up auto-login (under /etc/sv/...) but I couldn't get that working.


As far as I'm aware you won't be able to use runit in chroot version? or am I mistaken - as far as I remember runit must run as the very first process ID.

By the way, the version I'm working on now includes option to all run in RAM (with ability to alternatively only run selected parts in RAM, such as main rootfs or whatever sfs or other NNdirs). It is only half a dozen lines of extra code than current uploaded version so I decided to put the extra code in rather than just as a plugin (though I can't imagine me running my own system all in RAM since my machine is already a bit RAM-deprived...). I'll upload it in a day or two once tested but seems to be fine.

wiak

_________________
New Puppy/Dog forum: https://puppylinux.rockedge.org
Tiny Linux Blog: https://www.tinylinux.info/
Check Firmware: http://murga-linux.com/puppy/viewtopic.php?p=1022797
Back to top
View user's profile Send private message 
wiak

Joined: 11 Dec 2007
Posts: 1861
Location: not Bulgaria

PostPosted: Sat 10 Aug 2019, 10:13    Post subject:  

This project has been taking up a lot more of my time than I intended when I started it (and probably way more than it was worth). I originally only meant to create the build_firstrib_rootfs script for use via chroot on Linux hosts. The first build_firstrib_initramfs script was simply to illustrate it could be done and added to, but that first one couldn't connect to Internet so really useless in practice. Rather than leaving it at that I therefore made these earlier build initramfs script versions that employed chroot inside the initramfs (along with full rootfs, so a pretty big initramfs).

Basically, I wanted to provide an alternative to any woof-CE type build system. My idea being to use busybox/with-native-package-manager to provide simple to understand build scripts that would be both instructive and easy to modify and quick to use (and hopefully relatively flexible and powerful).

Anyway, I couldn't resist moving on from that to the current switch_root version that would switch to extrernal root filesystem, and I've added quite a bit of functionality and flexibility to that since; the latest almost trivial addition I completed in tests today being optional ability to copy all (or selected components) to RAM prior to running.

Nevertheless, additions come at a cost in size and complexity. The init inside the initramfs (and thus part of the build script) is now up to around 175 lines of code, including comments, and I don't want it to become much bigger or more complex than that (aside from possible plugin additions). A design aim of FirstRib is to make it easy to understand and modify for other purposes, hence my desire to keep the code short and simple. Despite its relatively small size (Puppy initramfs/init, for example, is over 1400 lines of code!), FirstRib initramfs/init does provide quite a lot of functionality and flexibility so I'm not overly upset that it's size has crept up a wee bit...

However, this new revision I'm soon to upload will probably be the last, for a while anyway (aside from bug-fix releases). That's my current intention anyway. I feel that FirstRib WeeDog does the job I hoped it would both in terms of being simple to use and modify and easy to build up into a full desktop system. There are a few facilities its short initramfs/init doesn't provide, but also some highly flexible options that others don't have. Anyway, does enough for my own current needs. Not many downloads/feedback from the forum community overall, but thanks certainly to rockedge for his consistent feedback throughout, to rufwoof for his recent comments and thoughts, and to fredx181 for his earlier tests and encouraging comments.

EDIT: I do intend to work on documentation and WeeDog/FirstRib website and possible wiki but I'll post here once I do get round to any of that...

wiak

_________________
New Puppy/Dog forum: https://puppylinux.rockedge.org
Tiny Linux Blog: https://www.tinylinux.info/
Check Firmware: http://murga-linux.com/puppy/viewtopic.php?p=1022797
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 17 of 49 [725 Posts]   Goto page: Previous 1, 2, 3, ..., 15, 16, 17, 18, 19, ..., 47, 48, 49 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Puppy Projects
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1137s ][ Queries: 12 (0.0249s) ][ GZIP on ]