Rationalisation of init

News, happenings
Message
Author
gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

Re: Fallback when specification fails

#31 Post by gyro »

technosaurus wrote:
gyro wrote:One of the aims of the rewrite is to get the sfs files loaded so that the "initmodules" facility can "insmod" any keyboard required modules before the keyboard is needed.
If you need to load kernel modules during init, the kernel is configured wrong. Just change tne CONFIG_XXXX=M to =Y for everything referenced in init - there should be no modules in the initramfs/initrd.
Correct, except for, where is this mythical list of all the modules that are needed?
My keyboard works with 666philb's kernels, but not with 01micko's. (not a big deal for me, since I then use "initmodules", with a 01micko kernel.)
666philb used to have a problem with a particular USB port, that required a module, (fixed in his kernels now.)
The end result is a kernel that conatins everything that looks slightly like it might be needed.
But stiil, at any time someone can start using new hardware that is only supported with a new kernel module, thus making all existing kernels obselete.
And until the new kernel's are available, the user has the "no keyboard during init" problem.

Edit:
Making the "huge-kernel" modules available during "init", we can do.
The remaining problem is; how do we sort out what is the minimal set of modules that need to be loaded to enable the keyboard?
The only current solution is "intmodules".

gyro
Last edited by gyro on Sat 25 Jun 2016, 06:05, edited 1 time in total.

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#32 Post by gyro »

jlst wrote:gyro, watching the logs I see vercmp complains about a wrong syntax... i made sure the busybox vercmp works the same as the standalone app.

This happens when one of the operands is empty, so I wonder if there is something to delete there...
I haven't got to the upgrading code yet.

I'm currently working with my new minimilist init script to sortout all the new stuff.
Once that is working I will start porting this code into the current init. (First action, delete obsolete stuff.)
And/Or port the current stuff that is missing in my new init, from the current init to my new init.

gyro

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#33 Post by gyro »

jlst wrote:That leads us to kernel-kit and the huge kernels already built. When compiling a kernel, one needs to be sure that all (or almost all) modules that have to do with keyboard input are builtin.. if a huge kernel does not comply with this, then it should be deleted immediately...
Sounds nice, but at any time, new hardware could invalidate all previous kernels.
My keyboard worked in "init" with old kernels, because there was no module peculiar to it. But then latter kernels introduced a new module to provide full support for my keyboard. It then stopped working during "init". (Fixed in 666philb's kernels.)

gyro

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#34 Post by gyro »

I have found a difficulty with searching for pup...sfs instead of vmlinuz, PDEV1 won't be set if pup...sfs is included in the initrd.gz.
A "humongous initrd" puppy will probably still boot, but PUPSTATE will have a couple of empty variables, unless the user adds a "pupsfs=xxx" boot parameter.

gyro
Last edited by gyro on Sat 25 Jun 2016, 05:55, edited 1 time in total.

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#35 Post by gyro »

jlst wrote:i think we should use the full blkid in the initrd, it's just more robust at everything, but a bit slower too
My testing has found no problem with the blkid in the tahrpup 6.0.5 initrd.gz.
My test init script relies on blkid to decode Label's into partition names, and so far this has been reliable in both ata and usb boots.
I have not discovered any need to replace it.

gyro

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#36 Post by technosaurus »

gyro wrote:My test init script relies on blkid to decode Label's into partition names, and so far this has been reliable in both ata and usb boots.
Out of curiousity, have you tested it with toybox blkid? I wrote and optimized that one a while back (with help from Rob) to be able to detect all filesystems UUIDs and lables in a single pass, so it should be faster/lighter than the busybox and GNU versions. I'm interested to see some feedback on it... BTW, it also has the guess-fstype functionality builtin (used in many puppy inits) I think Rob omitted my hacky NTFS label code because ... well NTFS is really a PITA that requires a lot of workarounds for "correct" code (UTF-16 labels, large spans between like data, weird "UUID"s)

Now that I think about it, with my microsaurus init.c (absolute minimal combo of init and xinit) and my toybox blkid.ccode, I could probably do the whole thing in C

The microsaurus code was something along the lines of this code:

Code: Select all

/*  Copyright 2011 -2016 Brad Conroy.
	Released to the Public Domain.
	Redistributable under any OSI approved license.
	Redistributable under any license meeting the OSI definition of open source.
*/

/* config.h begin */
#define CFG_SH ENABLED // {EN/DIS}ABLE using shell as init fallback
#define SHELL "/bin/sh"

#define ENV "PATH=/bin", "HOME=/", "PS1=# ", "SHELL=" SHELL

#define CFG_X  ENABLED // {EN/DIS}ABLE X server
#define X_CMD "Xvesa","-screen","1280x800x24","-nolisten","tcp","-tst"
#define X_ENV "TERM=rxvt", "DISPLAY=:0"

#define CFG_WM ENABLED // {EN/DIS}ABLE window manager
#define WM_CMD "jwm" , "-display", ":0"

#define CFG_DE DISABLED // {EN/DIS}ABLE desktop environment (icons)
#define DE_CMD "rox","-p","default"
#define DE_ENV "APP_DIR=/usr/share/ROX-Filer", "CHOICESPATH=/usr/share"
/* config.h end */

#define ENABLED(...) __VA_ARGS__
#define DISABLED(...)

#include <unistd.h> /* for exec */
#include <sys/mount.h> /* for mount */
#include <stdlib.h> /* for putenv */

static void forkit(char * const *argv,char * const * envp){
  long x = fork();
  //if (x<0) die(1); //TODO but this really shouldn't happen
  if (x==0) execve(argv[0], argv, envp);
  //TODO error if exec fails - unlikely unless command is bad
}


CFG_WM(static char * const wm[]={ WM_CMD,NULL}; )
CFG_X( static char * const  X[] ={ X_CMD, NULL}; )
CFG_DE(static char * const de[]={ DE_CMD,NULL}; )
CFG_SH(static char * const sh[]={ SHELL, NULL}; )
static char * const env[] = { ENV, CFG_X(X_ENV,) CFG_DE(DE_ENV,) NULL };

int main(int argc, char** argv, char **envp) {
	CFG_X(forkit(X,env);) //should this optionally be /bin/sh ??
	//Xvesa doesn't need anything mounted, so mount while X starts up
	mount("/proc", "/proc", "proc", MS_MGC_VAL, NULL);
	mount("/sys", "/sys", "sys", MS_MGC_VAL, NULL);
	mount("/dev", "/dev", "devtmpfs", MS_MGC_VAL, NULL);
	mount( "/dev/pts", "/dev/pts", "devpts", MS_MGC_VAL, NULL );
	//TODO wait for /tmp/.X11-unix/X0 socket
	CFG_DE(forkit(de,env);)
	CFG_WM(forkit(wm,env);) //should wm be optional too?
/*  The rest of "init", can be setup using jwm's <StartupCommand> etc...*/
	CFG_SH(execve(sh[0],sh,env);)
	/* TODO reap zombies, handle signals and clean shutdown method */
	while (1) sleep(1);
}
Note: Microsaurus had multicall binary with Xvesa, jwm, rxvt and dash included in the kernel builtin initramfs and the kernel size was under 1MB (2MB with gtk1 apps like rox, dillo, aumix, an editor, gtkdialog1 and a few other tools). The additional required kernel options for general purpos use would add an additional 1-3MB (wireless modules can be rather large). Using shared libraries like X11, c, gtk2 and its dependencies (for current ROX-Filer) would add a couple MB also.
Last edited by technosaurus on Mon 27 Jun 2016, 22:18, edited 1 time in total.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#37 Post by gyro »

technosaurus wrote:Out of curiousity, have you tested it with toybox blkid?
No.
Since I'm having no problem with the blkid I have, I haven't bothered trying others.

Perhaps this is a "busybox" v "toybox" issue, which is outside the scope of my current project.

gyro

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#38 Post by gyro »

I have uploaded a new minimalist test "init" script to the github.com/puppylinux-woof-CE/initrd_progs repository.
It can be downloaded using this command:

Code: Select all

svn export https://github.com/puppylinux-woof-CE/initrd_progs/trunk/0initrd/init
This is the first version of this "init", it can actually boot to a running system.

But it's a work in progess, there are still many missing facilities:
Support for pupmodes other than 5 and 12.
Support for "initmodules".
Support for multisession cd's.
Support for encrypted savefiles.
Support for savefile resizing.
Support for updrading savefile/savefolder.
Support for keymaps.
Support for non-english languages.
May need more savefile/savefolder searching for cd boots.

It does not support full installs, and since it resides in "initrd.gz" it may never do so.
(Unless someonelse changes this.)

Some issues with what it does do:
In pupmode=12, /tmp is in the savefile/savefolder, not tmpfs.
The pupmode management is not good.
Humongus initrd folk, may want to specify a "pdev1=" boot parameter to get something more meaningful into PUPSTATE->PDEV1.

It does implement the new algorithm for puppy file searching; much simpler that the old one.
It builds the aufs stack as it goes, first a pupmode=5 stack, it changes this to a pupmode=12 stack if a savefile/savefolder is found.
It only waits for usb storage if it is needed.
It fully supports the recent changes to specifying puppy files using Label or UUID in boot parameters.

My immediate plans:
1. fix the /tmp issue in pupmode=12
2. Devise a pupmode management scheme.
3. Implement "initmodules" support.
4. Sortout savefile/savefolder searching for cd boots.
5. Sortout what's really necessary for upgrading, because it's messy.

I will be pushing changes directly to initrd_progs on git-hub as soon as I have any patch that's worth backing-up.
Of course, being on git-hub, others are welcome to make changes also.

gyro

learnhow2code

#39 Post by learnhow2code »

hi gyro, sorry to ask something that you might have answered several times:

whats this init for? i know what an init is, what i mean is which distros do you currently work on/with? tahr was mentioned, did you have a hand in maintaining/editing its init? what about slacko? any others? kind of a trivia question but im actually interested, so its a sincere trivia question. good luck with your init, it sounds like an interesting project.

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#40 Post by gyro »

hi learnhow2code,

This project is about updating the "init" script in woof-ce. So improvements produced by this project will be included in all puppies generated using woof-ce, in the future.

The "init" script is inside the "initrd.gz" and is the code that runs at the beginning of the boot in frugal installs. It's major purpose is to setup the aufs stack that is at the heart of a puppy frugal install.

gyro

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#41 Post by gyro »

The "iinit" in "initrd_progs" has been patched to fix the /tmp issue in pupmode=12 and to add pupmode=13 support.
gyro

learnhow2code

#42 Post by learnhow2code »

gyro wrote:This project is about updating the "init" script in woof-ce. So improvements produced by this project will be included in all puppies generated using woof-ce, in the future.
fantastic! and i thought this was perhaps for one or two versions of puppy, but youre closer to "upstream" for a number of them, eh? thanks for the reply. i will have to look at the entire thread now, as it just got more interesting.

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#43 Post by gyro »

The "iinit" in "initrd_progs" has been updated.
It now makes a reasonable fist of pupmode=5, 12, 13, including "pfix=ram".
It also has support for pupmode=6,7, but this is untested.
"humongus initrd" support is also present but untested.

Still significant facilities not implemented.
e.g. savefile support does not include encryption or resize.
cd booting support is still only rudimentary.
"initmodules" not implemented.

Someone once asked why the "init" script has become so complicated; Well I think it's because it supports so many different scenarios.
If I were doing this just for me, there's an awful lot of stuff I would omit.

gyro

jlst

#44 Post by jlst »

gyro i added some changes that you might want to port to the new init

In ea7734f there are additions to check_status() and code to avoid a kernel panic when things go wrong

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#45 Post by gyro »

jlst,
New init doesn't have a check_status() yet, but does have a fatal_error() to support multiple possible fatal error messages.
The new check_status() should be able to take over the function of fatal_error().
gyro

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#46 Post by gyro »

A question:
Given "huge" kernels, is the zdrv critical to a boot?
By critical, I mean don't continue on with the boot without it.

My current version considers puppy...sfs as critical, everything else is optional.

gyro
Last edited by gyro on Tue 12 Jul 2016, 18:01, edited 1 time in total.

starhawk
Posts: 4906
Joined: Mon 22 Nov 2010, 06:04
Location: Everybody knows this is nowhere...

#47 Post by starhawk »

zdrv is your drivers, gyro. Not just for your network card... ;)

Sailor Enceladus
Posts: 1543
Joined: Mon 22 Feb 2016, 19:43

#48 Post by Sailor Enceladus »

When I boot Slacko 6.3 without the zdrv it makes it to desktop, but my mouse/trackpad doesn't work, even with using the dead mouse trick, and neither my wireless or wired ethernet show up. I did manage to hit Alt-F1 to get to Pmount to mount a drive that had the zdrv in it and put it back in with terminal commands though. Interestingly, "depmod" runs when I remove the zdrv but not when I keep it, and the pcmcia part says 1 2 3 4 5 6 7 8 9 10 before continuing. Maybe this info will be useful, or not...

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#49 Post by gyro »

I have accidently booted without a zdrv, and it did boot to a desktop, but of course without any kernel modules, lots of hardware simply did not work.

I guess my question really is:
If the init script detects that there is no zdrv, should it abandon the boot or continue?

My current thinking is that it should continue, since it will most likely still boot to a desktop (although it will most likely be a crippled one).

Whereas, if the main puppy...sfs is missing the boot is abandoned.

gyro

learnhow2code

#50 Post by learnhow2code »

gyro wrote:I guess my question really is:
If the init script detects that there is no zdrv, should it abandon the boot or continue?
puppalyzer can help you answer this question.

here is a list of several puppy isos (including modern ones) and whether they contain a zdrv or not:


contains a zdrv sfs file / name of iso

NO / puppy-4.2.1-k2.6.25.16-seamonkey.iso
YES / tahr-6.0-CE_PAE.iso
NO / slacko-5.6-4G-NON-PAE.iso
NO / wary-5.5.iso
YES / Legacy OS 2.1
LTS.iso?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Flegacyoslinux%2Ffiles%2FLegacy%2520OS%25202.1%2520LTS%2520Extra%2520Packages%2F&ts=1467656308&use_mirror=heanet
YES / puppy-3.01-seamonkey.iso
NO / precise-5.7.1-retro.iso
NO / squeeze-5.X.15-SCSI.iso
YES / puppy-2.17.1-nolzma-seamonkey-fulldrivers.iso
NO / pup-431.iso
YES / tahr-6.0.5_PAE.iso
YES / tahr-6.0-CE_noPAE.iso
YES / tahr64-6.0.5.iso
YES / slacko64-6.3.2-uefi.iso
NO / slacko-5.6-PAE.iso
NO / slacko-5.4-firefox-4g.iso
YES / slacko-6.3.2-uefi.iso
NO / slacko-5.7.0-PAE.iso
NO / Lxpup_13.01.iso?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fcheckmatelxde%2Ffiles%2FLxpup-
13.04%2F&ts=1467655822&use_mirror=tenet
NO / racy-5.5.iso
NO / slacko-5.7-NO-pae.iso
YES / yara.iso
NO / slacko-5.4-opera-4g.iso
NO / lupu-520.iso
NO / slacko-5.4-opera-PAE.iso

Post Reply