RC7 (STABLE) WeeDogLinux Arch 64 now released

[rufwoof]

As you should realise there are three different useful build_firstrib_initramfs script models now:

Model 01: which used chroot inside humongous initramfs to access the embedded firstrib_rootfs, which all runs in RAM. The last that model was worked on was Revision 008pre (I think). I will update that with the new stuff in model 04 (for changes location selection: readonly, RAM, or some other partition/dir than bootpartition) - I will similarly update model 02. @rufwoof: good to hear that runit can also work inside chroot-model-01 arrangements (sv didn't work for me when I tried it and though the PID of runit was the issue, but I didn't investigate further since was busy on switch_root implementation. In this model 01 testing I was also exiting out of the chroot back to a cttyhack-type shell in the initramfs and then restarting the chroot when required).

I've been letting that boot into void (chroot into 'merged' running Void kernel/installation) and then exit-chroot out of that to make or restore a tar of the upper-changes folder content (changes).

cd /
tar cf uc.tar /CHR/upper-changes
or
cd /
tar xf uc.tar

A form of save/restore. And chroot /CHR/merged /bin/bash .. back into void. I'm storing uc.tar on hdd which I mount within Void mkdir /mnt/hdd;mount /dev/sda1 /mnt/hdd ... and then access that within initrd once I've exit-chroot
cp /CHR/merged/mnt/hdd/uc.tar /.
Would be better however to just directly use the tar file

cd /
tar xf /CHR/merged/mnt/hdd/uc.tar

Note that /CHR is just my own choice of folder within initrd where lower/upper_changes/merged ...etc. are created/mounted/used.

My changes are generally little, so the tar file is small and quick to create/restore. Instead of hdd the saves could be stored elsewhere, on usb, even remote (retrieve/store using scp or whatever).

That's storing/restoring from/to a live running system. i.e. when I chroot back into void the changes immediately are visible. So far I've not encountered any problems from that, in some cases even having exit-chroot from within X with chrome running and after restoring and chroot back into Void the browser is still running OK on its prior content. If the browser isn't running then when started it usually prompts for 'restore session' to get back to where it was at the time of making the 'save'.

Desktop environment is pretty light installation wise, but highly functional. I like tilda (terminal) as I set that to F1 to show/hide and have it full screen when showing (i.e. I'm inclined to <F1> mtp <tab> <enter> ... to launch mtpaint than I am to use the touchpad to locate/click on mtpaint). mc as file manager and text editor (alongside busybox vi), alsa/pulseaudio/pavucontrol for sound, chromium browser (that I use to listen/watch multi-media, office docs (googledocs), pdf viewer ...etc.), xcalc for calculator (requires xbitmaps to be installed otherwise xcalc crashes and also crashes xorg!), jwm window manager, ccrypt for encrypting my ssh keys, mtpaint for image editing, and gcc/make/squashfs so I can run wiak's scripts and compile kernels if I so desire. void base also includes ssh so I can ssh into the server I use for email, irc, bbs's ...etc.

xbps-install -y linux4.19 base-system squashfs-tools xorg xinit xbitmaps alsa-utils pulseaudio pavucontrol jwm chromium tilda mc gcc make mtpaint ccrypt

Not bothering to compile my own kernel versions now, just using what Void provide (quicker/easier updates), and with the above the initrd weighs in at around 560MB (using xz high compression). Nice to be able to have the kernel and entire system fresh/new to the latest version in around a 20 minute script run time. I'm tending to kick that off in the background on a daily basis so it builds whilst doing other things. I'm also pulling down Steven Blacks host list as part of start up (as good as ad-block). Haven't bothered setting up auto suspend on closing the laptop lid, instead I run that manually by running echo -n mem >/sys/power/state ... inside a suspend.sh script. As that way I can have the system suspended (low power consumption) when the lid is up, or more typically not having it suspend when the lid is closed and I have something running such that I don't want that suspending just because the lid is closed.

The AMD/Radoen 4GB Acer Aspire ES 15 that I'm using does encounter slow HDD spin-up delays, that pretty much make it useless for the likes of Windows. However for booting from usb, running in ram with usb disconnected once booted and only using the HDD for storage its a great little system. I could even disconnect/remove the HDD and just store data/saves on my ssh server. I don't think that the extra battery charge/life doing that would add would be that much different however and having some laptop storage space can at times be handy, or even setting it up as swap.

[rockedge]

I just built a WeeDog model 4.005 on a DELL PowerEdge R210 II blade server
4 core 4 thread 8 gigs of RAM

base-minimal base-devel xorg jwm rox xterm bash geany mc mtpaint adwaita-icon-theme python cmake git ffmpeg ffmpeg-devel hiawatha mariadb php-cgi

now starting to add more PHP modules and PERL modules plus libraries needed for an eventual build of ZoneMinder from source

have of course network connection on eth0 and eth1..no wlan no audio...this is a server....

I have a model 2 and model 4.004 that both run very nicely...and as noted before, cpu's run cool and the machine is fast and responsive.

I have pmcputemp from Puppy linux running on both a 32 bit and 64 bit machines but on the R210 it never changes the temp output image.

[rockedge]

once I have the initial build of weedog I chroot the root_fs and change the password for root, exit and umount......then build the initramfs which is really small and finishes quickly. Then boot and add all the components via xbps-install

then I rename /upper_changes to something like /50upper_changes.....reboot
and go from there, so if I break it while adding the neural network stuff and opencv - darknet-yolo I can roll back to the original setup

[rufwoof]

For a 1366x768 laptop, just over 27 lines of console font is nice IMO

Code: Select all

xbps-install terminus-font

edit/add-to /etc/rc.conf

Code: Select all

FONT="ter-i28b"

i.e. bold 28 size (768 / 28 = 27.4 lines)

[wiak]

@rufwoof and rockedge:

You guys are doing great and many thanks for the information and details/ideas you are posting. They are certainly very helpful for me, but I'm sure also giving important ideas/techniques/inspirations to others who may try WeeDog builds of their own.

Unfortunately, I'm currently sick just now, a bit feverish with it so my coding of build initramfs04 Revision 006 has slowed down to stop for a day (or two). Should hopefully be recovered in a few days time and the only extra I'm working on in 006 is copy2ram option so usb sticks can be unplugged with these model 04 switch_root versions too (I have basically completed the coding - it's only about 12 extra lines of code - but needing to tidy it all up and check it out a bit yet).

Thereafter, I'm also wanting to experiment with (plugin/optional maybe) saving changes in tar files, which, as I mentioned, tinycore linux does in its own quite interesting way - and I have also been very impressed by how small and fast to save such compressed tar saves often tend to be once main system already built. I might actually study exactly how tinycore linux does it, because it allows selection of which parts of filesystem to bother saving (filetool.list) as well as having an additional exclude file (xfiletool.list). Using compressed tar like that does seem to generally result in smaller save-files/save-times than merging filesystems/dealing-with-whiteouts etc. Other mechanisms used in TinyCore Linux, however, are probably not so useful or relevant because that system uses symlinks to bolt everything together rather than layered filesystems.

http://tinycorelinux.net/concepts.html

Well, what I'm really wanting to do is play with the system for a while before adding anything at all further! I'm resisting doing that till I have Revision 006 out of thet way (or I'll lose focus on that). Generally speaking, the builds seem to work sufficiently well now (and/or are simple-enough-in-form/easy enough to understand/modify, as is, by users), so I plan not change much at all now (if it works... don't break it...).

wiak

[wiak]

Progress report:

The init for build_firstrib_initramfs04 Revision 0.0.6 has now been completed and passed all my tests thus far. Only major change in this new revision of the build script is support for copy2ram grub kernel line option, which causes WeeDog to copy all NNfilename.sfs files, all NNuncompressed_dirs to a RAM tmpfs prior to them being mounted from there (rdsh plugins also copied to RAM at same time). Following some related code changes in the initramfs/init, it is then possible to unmount and eject the boot device if one of the following additional conditions are met:

grub kernel line also has:

1. changes=RAM specified (upper_changes then stored only in RAM: non-persistent)

or

2. changes=readonly specified (also only using RAM and writes not allowed)

or

3. changes=<path to changes directory> specified, where changes_partition is different from initial bootpartition (since bootpartition then not being used it can be unmounted).

For all of the above possibilities, it is then left up to the user to do the umount /mnt/bootpartion if they so wish. However, for each of the above cases, WeeDog prints a message to tell the user they can unmount bootdevice if they so wish.

I still have to assemble the tested initramfs/init revision 006 into the buildscript, and double check all is well. Late now, so build_firstrib_initramfs04_s006.sh should be available for download sometime tomorrow, all going well.

wiak

[wiak]

build_firstrib_initramfs04 Revision 1.0.0 uploaded to thread first post: Bumped to Revision 1.0.0 since feature freeze planned for this one for the time being (aside from maintenance/bug-fixes).

http://www.murga-linux.com/puppy/viewtopic.php?t=116212

build_firstram_initramfs04_s100_README, which is also provided for download from thread first post.

****NEW (in Revision 1.0.0):

Code: Select all

grub kernel-line copy2ram option, which causes all NNsfs, NNdirs, and rdshN.plug files to be copied to RAM tmpfs and mounts the NNsfs and/or NNdirs from there.

One result of that is that when either of the following grub kernel-line option combinations is used, the bootdevice (e.g. usb stick) can be umounted and ejected:

1. changes=RAM at same time as copy2ram
   (upper_changes then stored only in RAM: non-persistent)
2. changes=readonly at same time as copy2ram
   (also only using RAM and writes not allowed) 
3. changes=<path to changes directory> specified, where
   changes_partition is different from initial bootpartition
   (since bootdevice then not being used it can be unmounted)

I mainly tested this in combination with two previous upper_changes sessions, which I renamed to 50upper_changes and 51upper_changes prior to rebooting with changes=RAM copy2ram on grub kernel line. Note that if you are booting from usb using these options, after booting you should be able to remove it but only after relevant umount command.

There remains lot to test. I've tested as best I can for now, but, yet again, more testing is required. Though most of the code remains the same as before, I did quite a lot of re-organization to facilitate putting in the copy2ram code; hopefully it will all work, but the alterations were significant so I can't be sure until better tested.

wiak

[rockedge]

I will give the latest a try out from a usb stick and from a HDD connected by usb SATA adaptor

[wiak]

Whilst I take time out for testing and for improving the documentation including WeeDog website, in the first post of thread,

http://www.murga-linux.com/puppy/viewto ... ?p=1028956

I've included

QUICK BUILD/TEST INSTRUCTIONS

in addition to a firstrib00.plug that installs the needed Void kernel components for those wishing to use Void Linux kernel/modules etc. It's contents are as follows:

Code: Select all

xbps-install -y ncurses-base linux4.19 linux-firmware-network wifi-firmware shadow
pwconv # set up passwd system
grpconv
echo -e "root\nroot" | passwd >/dev/null 2>&1 # Quietly set default root passwd to "root"

# you can add as many valid commandlines as you want in here

Note that the plugin also sets up default root passwd to simply "root", which you can change later of course.

I've tested above in usb stick install, ran it with grub options changes=RAM and copy2ram and was able to then umount my usb stick (on my system that was: umount /mnt/sdb1) and then unplug the usb stick.

You can later install bash, and say, base-minimal; e.g. after booted, or via mount_chrootXX.sh (or simply by adding to above firstrib00.plug prior to running ./build_firstrib_rootfsXXX.sh).

I'm looking forwards to someone producing a firstrib00.plug (or othername.plug) that contains packages and config code to build a nice polished X desktop...

I would suggest to anyone who has been reading this thread not to be put off by the technical nature of some of the discussions here. The system is extremely simply to build and use (really just the matter of running the two small build scripts, per the quick instructions in the first post of this thread) and most anyone on this forum should find it very simple to do and you may well even find working with WeeDog/Void useful and help you better understand/quickly-learn Linux distro/system operation more generally. For full X-desktop-build help it becomes simply a matter of asking/checking info and questions on this thread once you have WeeDog booting successfully to the commandline.

I'm sure the current testers could confirm that Void Linux xbps use provides WeeDog with an extremely powerful package manager with excellent and continually up-to-date package repositories, and WeeDog FirstRib initramfs provides some flexible frugal install options including easy rollback and use of either sfs files or uncompressed directories (including copy2ram capability) in but a couple of short and simple easy to read and modify scripts.

EDIT: If you are late to the party and want to search this thread, then MochiMoppel's post regarding viewing thread as one html page might prove useful to you:

http://www.murga-linux.com/puppy/viewtopic.php?t=111347

wiak

[rufwoof]

Reported my test of the more recent scripts here http://murga-linux.com/puppy/viewtopic. ... 19#1034419

[wiak]

I did do one run where .plug included installation of xorg, chromium ...etc. straight off, but that resulted in a .sfs file of nearly 1GB that wouldn't boot.

But this is odd. I have just attempted to repeat same installation (though I ran out of space during creation of 01firstrib_rootfs.sfs) and can confirm switch_root causing kernel panic (I used uncompressed 01firstrib_rootfs since I didn't have the sfs). I am looking into the issue at this exact moment.

wiak

[wiak]

@rufwoof: If I'm correct that test above that failed with large 01firstrib_rootfs.sfs was on a usb drive? Could you please confirm that?

The successful chromium test you did was on hard drive from what you write. I have found bug on similar test, but on usb stick boot only (all booted fine when same files on my hard drive instead) where my simple routine for removing modules to save memory has proved to be a bit too aggressive (I need to tell it not to remove module ehci_pci, which usb stick needs but is not being marked as in active use somehow - hence able to be removed). At the moment, simple workaround for me will be to have simple list of modules I definitely don't want moved (currently being that ehci_pic module, but there may be one or two other modules related to SD card booting I have to definitely force keep).

EDIT: testing my workaround/fix now.

wiak

[rockedge]

I am getting a kernel panic and have not yet built a successful system using model s100.

working on using the Void kernel with this build because up to now I have been using the Bionic kernels

[wiak]

I am getting a kernel panic and have not yet built a successful system using model s100.

working on using the Void kernel with this build because up to now I have been using the Bionic kernels

Is this with a usb connected drive of some sort? If so, which sort. Problem is probably the one I am addressing with bug-fix this very moment.

wiak

[rufwoof]

@rufwoof: If I'm correct that test above that failed with large 01firstrib_rootfs.sfs was on a usb drive? Could you please confirm that?

No. My hdd is pretty much empty so I've been using that as the test-bed (installed grub4dos to boot - where my otherwise usb based regular desktop boot usb isn't connected (BIOS set to boot usb before hdd)).

[wiak]

@rufwoof: If I'm correct that test above that failed with large 01firstrib_rootfs.sfs was on a usb drive? Could you please confirm that?

No. My hdd is pretty much empty so I've been using that as the test-bed (installed grub4dos to boot - where my otherwise usb based regular desktop boot usb isn't connected (BIOS set to boot usb before hdd)).

@rockedge and rufwoof:

Could you try with Revision s101, which I've just uploaded. It fixes the kernel panic issue on my machine - but that panic was only happening for usb boot on my machine - booting from my ata harddrive was fine. My test was with your (rufwoof) big chromium build all built at once into big 01firstrib_rootfs.

Even if that doesn't work on your machine, I've now included a modules_remove.plug plugin that can be altered to address alternative issues of this sort. Late here now though, so I have to go, but please send in feedback to see is all okay now, and if not, try and find out what driver/module your boot media needs and we can fix it from that info tomorrow.

EDIT: Even if issue on your machine is different than it was on mine, you may be able to sort it out with modules_remove.plug inbetween times anyway if you check how the relevant code used (extract below) works (note how I now, in Revision s101, prevent modules name-matching ehci, xhci and sdhci being removed - same could be done for any module regular expression match):

Code: Select all

if [ -s "\${mountfrom}"/modules_remove.plug ]; then  # source modules_remove plugin
	. "\${mountfrom}"/modules_remove.plug
else
	# Remove unused modules to save memory
	modprobe -r \`lsmod | cut -d' ' -f1 | grep -Ev 'ehci|xhci|sdhci'\` 2>/dev/null  # keep ehci,xhci,sdhci
fi

To be honest, rufwoof, it doesn't make sense to me that smaller build booted on your machine, but bigger build didn't (unless you were also using copy2ram with the smaller build, since that would not run into any modules removed issue) - if it were a module issue, would occur for small or big firstrib_rootfs booting. On my machine, module ehci_pci was being removed on s100 revision - not allowing that to be removed fixed the usb boot for me (as I said boot from hd with your big firstrib00.plug build worked fine anyway).

Note that revision s101 is hot off the press and only given one brief test by myself thus far. But I'm releasing ultra-early since the quick test worked as my theory imagined it would... it may indeed need some further tweaks for some systems. I'm pretty sure it will be a similar module removed issue though.

EDIT2: If worst comes to the worst you can just comment out that whole block of code though and your system should boot since all modules will remain loaded. But uses more RAM that way. Actually Porteus boot in DebianDog does that; there are far more media driver modules loaded by DD Porteus boot than need to be (as lsmod shows). Puppy kernel, on the otherhand, has majority of these media drivers built permanently into the kernel (hence such issues don't occur) - official Void Linux kernel is pretty light in terms of few drivers directly built into it - so a bit harder to handle, but potentially lighter in use.

Alternatively, once plugin code fixed will be able to just use an empty plugin (or simply a # comment line in that with no actual code otherwise), then modules will not be removed at all so at least booting should fthen always work.

EDIT3: Sorry, in my haste I forgot I had moved "mount from" mountpoint after the overlay merge, so modules_remove plugin won't work yet, though the else code above should work for USB devices anyway. I'll fix the plugin code later today.

wiak

[rockedge]

yes I will test s101 revision....I can't get past the kernel panic with the s100. I'm going with the Void kernel on this series of tests

[wiak]

yes I will test s101 revision....I can't get past the kernel panic with the s100. I'm going with the Void kernel on this series of tests

If you run into further problems, I'll need more info regarding media you are booting from to help work out what is going on. Also, please remember to start from blank directory in major bug tests just in case something in folder (and old upper_changes or whatever contains something causing any issues).

Despite bugs sometimes not seeming logical, there are sometimes race conditions involved that throw all logic out the window (though sleep insertions often then 'fix' the issue), or sometimes it can just be a matter of order of loading/unloading to fix. The biggest problem is when all works fine on the developer's own hardware - but not on someone elses - I can only test my own system; then system/devices-uses information becomes paramount in the bug search (followed by copious googling and probably lots of luck).

wiak

[rockedge]

I start fresh every time I build one to avoid problems and mix ups

I am booting from a HDD connected via USB
just got back home so I'll start the tests soon...

where is s101 located for download?
I cloned the project from github but I don't think I see s101

[wiak]

where is s101 located for download?

It is already available in first post of thread rockedge. I uploaded it last night after a quick test of my own. I haven't uploaded to github, though I should, but I was waiting for tests (I'm not using git during actual development at the moment).

http://www.murga-linux.com/puppy/viewto ... 56#1028956

I imagine it may work for you since you are indeed connecting via usb.

wiak