Woof-CE could tidy up automatically after building a Puppy

What features/apps/bugfixes needed in a future Puppy
Post Reply
Message
Author
stemsee

Woof-CE could tidy up automatically after building a Puppy

#1 Post by stemsee »

Woof-CE kernel-kit builds a headers package, then either a kernel and modules package, or kernel with modules package. The aufs-utils are bundled with the modules. It seems to me that the headers shouldn't be packaged as a separate sfs, but they should be bundled with the devx, and also the kernel sources.sfs. This is one of the things that get me with puppy - the true size of a useful puppy is disguised, the size is hidden in the related packages, devx, headers, sources, and apps.sfs, That is why I built EmSee-linux 2-3GB pups, because it includes the undisguised size of a useful pup. Now, I build several pups and kernels per month, there is a lot of manual work involved even when using kernel kit. And there are lots of packages and folders here and there. After compiling four kernels, 2x 64bit and 2x 32 bit, aufs-utils don't get compiled because build script doesn't forcefully overwrite previous one when the folder is left in existence. Also in order to use your kernel and modules package (which gets converted to sfs for use with...?) in woof-ce, you need to manually convert to a pet and copy or move it to the local repositories. So you have a sources.sfs and sources dir = 1.3 GB sitting on the disc doing nothing (x4). No wonder I get mucking fuddled! I am not complaining i am just moaning. There needs to be a tidying up function at the end of the script to sort all of this out, to move all packeted stuff out and delete the debris. I don't know where else to say this so I said it here. I do know that all of this will be unified in woof-next at some point, but in the mean time it won't require much typing to make the changes. Unfortunately git doesn't work well on my system, or I don't know how to push to testing from my laptop, it is not as though I didn't try. I did!

User avatar
01micko
Posts: 8741
Joined: Sat 11 Oct 2008, 13:39
Location: qld
Contact:

#2 Post by 01micko »

Puppy Linux Blog - contact me for access

stemsee

#3 Post by stemsee »

'./build.sh clean' will do nicely thank you! I like Draconian measures!! ;-)

Can I suggest moving headers /usr/include to kernel modules for safekeeping, as default. As woof-ng kernel combo is proving to have a much wider use than just for puppy. I envision booting any squashfs live package with it, and then one would, I assume, need native development libraries, rather than puppy devx.sfs, which means having headers with kernel combo in nearly all circumstances is preferable, Also when building a woof-ng distro there is no option to import the headers package, except they are copied to chroot or root-fs-skeleton (or whatever it's called now.) adding a degree of extra difficulty for novices. But for devs it will be easy to move the headers out of modules.sfs and implant wherever!

User avatar
01micko
Posts: 8741
Joined: Sat 11 Oct 2008, 13:39
Location: qld
Contact:

#4 Post by 01micko »

Headers belong with development files, not the kernel source.

Did you know many programs will not compile without linux headers.

And why subject users to the bloat of the kernel source when they don't need it. It is only needed for compiling modules.
Puppy Linux Blog - contact me for access

stemsee

#5 Post by stemsee »

I didn't know the technical issues, just thinking of the immediate convenience when looking at them in /dist/packages folder! So are they not available to the system from kernel-modules.sfs?

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#6 Post by SFR »

01micko wrote:Headers belong with development files, not the kernel source.
Just to be sure: let's say I compile a different kernel and replace zdrv.sfs.
Then it would be a wise move to also replace the headers in devx.sfs, in order to have a 100% proper, complete enviroment?

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

User avatar
01micko
Posts: 8741
Joined: Sat 11 Oct 2008, 13:39
Location: qld
Contact:

#7 Post by 01micko »

SFR wrote:Just to be sure: let's say I compile a different kernel and replace zdrv.sfs.
Then it would be a wise move to also replace the headers in devx.sfs, in order to have a 100% proper, complete enviroment?

Greetings!
Yes, that is best practice. However the headers don't change a lot and if kernel versions are reasonably close it often doesn't matter. Lazy man's way would be simply install the headers pet which will be on the top layer covering the old devx version. Depends on whether you want to make an iso to ship it as a puplet or just tinkering around for personal use.

One example where they were needed (and not kernel source) was when I compiled abiword for 64 bit. I forgot to install headers at the woof level and consequently ./configure didn't even complete. I installed a wrong version (just because it was handy) and abiword compiled fine. Of course most userspace progs are not kernel version dependent.

An instance of where headers did matter (version) was building AMD video driver. As you know they lag 4 or 5 versions of the kernel behind, so it wouldn't compile. Kernel devs changed the location of version.h ( I forget where for now ) but I made a symlink to where AMD wanted it and the driver compiled. Old headers would not suffice even though version.h would have been in the correct place.

So, it is the responsibility of the distro builder/maintainer to ensure correct version headers are installed when shipping and the user if they change kernels.
Puppy Linux Blog - contact me for access

stemsee

#8 Post by stemsee »

I for my part see no better way to deal with headers than to put them in with modules.sfs, possibly compressed, WHEN merely providing a kernel combo update, but in the case of a new distro to package them in devx! Can anyone foresee any critical conflicts! Which layer takes precedence?

Edit: or bundled as a pet as 01micko suggested. So that would need to be added to build.sh

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#9 Post by jamesbond »

There are two kinds of the kernel headers.

1. libc kernel headers (also known as "userspace kernel headers" - yes, an oxymoron I know). This is what you get with "make headers_install". These are used to build applications and libc.
2. the real kernel headers (the one used for building kernel modules) - this is a subset of kernel source.

---

(1) does not have to match the version of the running kernel. See: https://sourceware.org/glibc/wiki/FAQ#W ... be_used.3F. So it doesn't belong in modules.sfs, it should live in devx (because it doesn't depend on kernel version anyway and it is mostly needed for compiling apps).

Puppy never has (2), instead it always packs the *entire* kernel source. And packing the 100MMB of kernel source to modules.sfs doesn't make a lot of sense.

You can of course makes (2) yourself but there isn't an easy way to do so; and when you do, it makes sense to put it as part of modules.sfs to make it easier for other people to compile new kernel modules - but again the question from Mick is - how many people actually *builds* kernel modules?
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

stemsee

#10 Post by stemsee »

@jamesbond

Thanks for sharing knowledge! I now know three times more about headers today as I did yesterday!

"Who can count what is lacking?"

Post Reply