Running with compressed kernelmodules

[PANZERKOPF]

. I am also not able to compile findutils ( 4.1.20 , 4.4.0 and 4.5 9 ) statically , even after googling around one or two hours.

Did you try to set LDFLAGS variable?
LDFLAGS="-static"

[Karl Godt]

LDFLAGS="-static"

yes :

> ./configure --prefix=/usr
> mmake LDFAGS=-static
> make --prefix=$LFS/usr install does the trick

from
http://www.linuxfromscratch.org/piperma ... 06299.html
but :
static find became 808kb and `strip find` 519kb against the 55/60kb inside the initrd ( same version 4.1.20 ) from http://ftp.gnu.org/gnu/findutils/ the largest difference between a static and a shared binary that I have seen so far ( usually 2 to 4 times larger ) . How to disable features ? The features.h at the common /usr/include seems to be ok and --disable-group --disable-ctime --disable-empty --disable-10-or-more-find-options didn't get me further : no halt on an error and no reduction of size

[PANZERKOPF]

LDFLAGS="-static"

no halt on an error and no reduction of size

I never make It but maybe a find utility can be compiled statically with
uclibc or dietlibc? These libraries are smaller (and possibly faster) than libc.
Also CFLAGS="-Os" reduces a size.

[technosaurus]

glibc static compiles are always pretty large (compare the glibc implementation of ldconfig and sln at over 500k to the uclibc versions that are under 50k)

Rob Landley has a nice uClibc{++} toolchain at http://www.landley.net/aboriginal/downloads/binaries/

I don't have my notes handy at the moment but IIRC you'll need
Do not use a devx.sfs in combination, but have it mounted to get binaries
CFLAGS=" -muclibc -static -I/usr/include/c -I/usr/include/c++ -Os ..." <<--may not be exact
you will may want to build ncurses for busybox build (its easier than finding all the files)

If you want to compile more than just basic apps I recommend getting additional patches from openwrt (I was even able to build geany and scintilla with those)

note1: btw the really large linux from scratch file there can be renamed to *.sfs and mounted in order to do an automated LFS bootstrap (which includes apps mentioned above)

note2: may put alpine linux on a spare partition to keep an updated uclibc system (another good source of uclibc{++} patches)

[Karl Godt]

Code: Select all

./configure LDFLAGS=-static CFLAGS=-Os

find-4.1.20 : 681kb , strip 434kb
find-4.4.0 : 1085kb , strip 758kb

Best founds for "Complete list of compiler options" :
http://msdn.microsoft.com/en-us/library ... 71%29.aspx
http://en.gentoo-wiki.com/wiki/CFLAGS

/Os Favors small code

-Os
-Os optimizes for size. -Os enables all -O2 optimizations that do not usually increase code size and performs further optimizations designed to reduce code size.
-Os is very useful for large applications, like Firefox, as it will reduce load time, memory usage, cache misses, disk usage etc. Code compiled with -Os can be faster than -O2 or -O3 because of this. It's also recommended for older computers with a low amount of RAM, disk space or cache on the CPU. But beware that -Os is not as well tested as -O2 and might trigger compiler bugs.

downloaded uclibc-0.9.30.2-src and this has got a menuconfig that is also quite large ..... Locale doesn't automatically download : wget 404 ....

What would I need to pass to ./configure that make uses the uclibc ?

[Karl Godt]

ERRATA : -Os + strip didn't made find 434kb , it became 511kb ... so it seems that -Os doesn't do much that strip can't do ......

Anyhow : I've got now some uclibc files at
make install PREFIX=/opt/ULIBC/001
make install_runtime PREFIX=/opt/ULIBC/001
make install_headers PREFIX=/opt/ULIBC/001
make install_utils PREFIX=/opt/ULIBC/001
make install_hostutils PREFIX=/opt/ULIBC/001

/opt/ULIBC/001/usr/i386-linux-uclibc

and found out that there are already some few files at

/usr/dietlibc at my fullinstall + devx

[Karl Godt]

I recommend getting additional patches from openwrt

http://downloads.openwrt.org/sources/

They have complete source there and I had to use save link as on .tar.gz at my firefox to prevent it beeing rendered inside the browser ... something I had occassionally with .pets ( ibiblio I think )

may put alpine linux on a spare partition

http://alpinelinux.org/wiki/Main_Page

.iso : 193MiB and where are the large files at openwrt ?

[Karl Godt]

About man /opt/diet/man/man1/diet.1 :

diet(1) diet(1)
NAME
diet - mangle gcc command line arguments
SYNOPSIS
diet [-v] [-Os] [normal gcc command line]
DESCRIPTION
diet is a small wrapper around gcc. It will try to look at the command
line you specify and try to mangle it so that gcc will compile against
the diet libc header files and link against the diet libc itself.

diet tries to be smart for cross compiling. If the first argument is
not gcc but sparc-linux-gcc, diet will guess that you want to cross
compile for sparc and use bin-sparc/dietlibc.a instead of the
dietlibc.a for your default architecture.

The -v option will make diet print the modified gcc command line before
executing it.

When passed the -Os option before the gcc argument, diet will mangle
the gcc options to include the best known for-size optimization set-
tings for the platform.
FILES
~/.diet/compiler may contain compiler options as you would specify them
on the command line, i.e. separated by spaces. Those will then be used
instead of the built-in defaults for diet -Os.
AUTHOR
Felix von Leitner <felix-dietlibc@fefe.de>
April 2001 diet(1)

it seems that to use it I would have to find+replace "gcc" with "diet" at all makefiles insinde the source ?

[technosaurus]

dietlibc is difficult to build more complex applications with but there is a FAQ here:
http://www.fefe.de/dietlibc/FAQ.txt

"diet" is just a wrapper
before you start hacking away at the source, try this though:
CC="diet gcc -nostdinc" CFLAGS=" -D_BSD_SOURCE ... "

you may get warnings of missing include files (blah blah linux/*.h or similar) you _can_ copy them from /usr/include to /usr/diet/include got get it to at least _try_ to build

besides using the -Os flag, the best size saving C{XX}FLAGS are -ffunction-sections and -fdata-sections when combined with the LDFLAGS -Wl,--gc-sections (as of gcc-4.5.2 the -flto related flags essentially replaces these) (--as-needed is a good ldflag too)

a little bit more details in the pet packaging 100/101 section of my signature

[Karl Godt]

About dietlibc : No success to compile `find` with

Code: Select all

./configure LDFLAGS="-static -Wl,--gc-sections,--as-needed" CFLAGS="-ffunction-sections -fdata-sections -Os" CC="diet gcc -nostdinc" CFLAGS="-D _BSD_SOURCE"

find-4.1.20 :

getline.h:35: error: conflicting types for 'gnu_getline'
/usr/dietlibc/include/stdio.h:191: error: previous declaration of 'gnu_getline' was here
getline.h:38: error: conflicting types for 'getdelim'
/usr/dietlibc/include/stdio.h:192: error: previous declaration of 'getdelim' was here
getline.c:139: error: conflicting types for 'gnu_getline'
/usr/dietlibc/include/stdio.h:191: error: previous declaration of 'gnu_getline' was here
getline.c:145: error: conflicting types for 'getdelim'
/usr/dietlibc/include/stdio.h:192: error: previous declaration of 'getdelim' was here

after /* uncommenting */ these two it stopped at two other files .

find-4.4.0 :

./stdio.h:395: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'rpl_getline'
/usr/dietlibc/include/sys/time.h:10: error: expected specifier-qualifier-list before 'time_t'
/usr/dietlibc/include/signal.h:265: error: expected specifier-qualifier-list before 'pid_t'
/usr/dietlibc/include/sys/fsuid.h:9: error: expected ')' before 'uid'
/usr/dietlibc/include/unistd.h:36: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'lseek'
/usr/dietlibc/include/time.h:20: error: expected ')' before 'clock_id'
/usr/dietlibc/include/wchar.h:34: error: expected ')' before '*' token

[technosaurus]

yes you have come across another of the many programs that is riddled with "GNU=isms" It would probably need getline instead of gnu_getline (may need to change the constructs a bit) ... comment out getdelim in the find code so that it uses the diet version and define the types (pid_t and time_t) with their corresponding types such as:
#define pid_t int //maybe unsigned int?

As you can see, it takes some time and work to get things to work with dietlibc (including making use of libowfat sometimes)

see my efforts here:
http://code.google.com/p/diet-binaries/downloads/list

[amigo]

Karl, you need to get your CFLAGS all together like this:
CFLAGS="-ffunction-sections -fdata-sections -Os -D _BSD_SOURCE"
Otherwise only the last part is being set.

[Karl Godt]

Finally I got me a find-4.1.20 of 92kb (/1.024 /1.024 = 87kb) :

Code: Select all

./configure LDFLAGS="-static -Wl,--gc-sections,--as-needed" CFLAGS="-ffunction-sections -fdata-sections -Os -D _BSD_SOURCE" CC="diet gcc -nostdinc"

- which behaves like before -

l10nflist.c:(.text._nl_make_l10nflist+0x1b0): undefined reference to `stpcpy'
l10nflist.c:(.text._nl_make_l10nflist+0x1cc): undefined reference to `stpcpy'
../intl/libintl.a(l10nflist.o):l10nflist.c:(.text._nl_make_l10nflist+0x1f8): more undefined references to `stpcpy' follow
collect2: ld returned 1 exit status
make[3]: *** [xargs] Error 1
make[3]: Leaving directory `/COMPILE/FINDUTILS/findutils-4.1.20/xargs'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/COMPILE/FINDUTILS/findutils-4.1.20/xargs'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/COMPILE/FINDUTILS/findutils-4.1.20'
make: *** [all] Error 2
23:16 findutils-4.1.20 # strip /COMPILE/FINDUTILS/findutils-4.1.20/find/find
23:25 findutils-4.1.20 # ls -s /COMPILE/FINDUTILS/findutils-4.1.20/find/find
92 /COMPILE/FINDUTILS/findutils-4.1.20/find/find
23:26 findutils-4.1.20 # file /COMPILE/FINDUTILS/findutils-4.1.20/find/find
/COMPILE/FINDUTILS/findutils-4.1.20/find/find: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, stripped
23:31 findutils-4.1.20 #

with outcommented lines for "stpcpy" at getline.h+.c , l10nflist.c , dcigettext.c and for "mbrtowc" at quotearg.c
This find is working though, but locate and xargs did not compile (as quoted above).
And find is still larger than the original initrd.gz-find.

Further it turned out that busybox more and less do not work in the rdsh like they use to at the prompt or in X normally : they rush to the very end of the file. e3 is already a very sophisticated and good editor but e3vi ( trying to exit e3vi with Ctrl-z) gave me a hang and the kernel did not reboot though i passed a panic=40 param to the grub commandline.
The busybox modinfo also needs two tweaks in the init-script :

Code: Select all

[ "`modinfo aufs | grep vermagic 2>/dev/null`" = "" ]
[ "`modinfo ppa | grep vermagic 2>/dev/null`" != "" ]

because busybox modinfo does not produce an error exit code if the module is not found .
Also the commands reboot,poweroff and halt do not work simply typed at the commandline. I'tll try again with umount -ar , fusermount -z -u , umount-FULL -i -n -l , kill 0 -3 -9 as it is mentioned at rc.shutdown.
waitmax and fsck are the only binaries that are not used by the scripts (e2fsck bianry is used) and bb-fsck hasn't got ext4 support.

Btw : Which version control system and licence to choose at code.google ? I have a few on skydrive at live/hotmail but to provide source for eg. kernels 50mb are not enough ( splitted into drivers and else ) and the navigation there is quite uncomfortable.

[PANZERKOPF]

And find is still larger than the original initrd.gz-find.

Because initrd's find is compressed with UPX.
e2fsck, resize2fs,fsck also are compressed with UPX.
You can see an UPX signature at offset 078h-07Ah.

[technosaurus]

for stpcpy try adding -lcompat to the end of your LDFLAGS and or this in the includes of the referenced .c file
#include <string.h>

for mbrtowc try adding this to the list of includes
#include "dietlocale.h" //maybe
#include <wchar.h> //for sure
#include <errno.h> //maybe

maybe try this busybox patch pack that I cobbled together to use the old full e2fsprogs from various sources using chicken wire and duct tape.

Notes:
It has some compile warnings, but worked ok
Needs updated to new busybox constructs and ext4 support
I didn't add in the full usage text

[Karl Godt]

maybe try this busybox patch pack that I put together to use the old full e2fsprogs

Already wondered how to enable them at the menuconfig .
Think I've found it :

@@ -469,6 +469,10 @@
coreutils/ \
coreutils/libcoreutils/ \
debianutils/ \
+ e2fsprogs/e2p/ \
+ e2fsprogs/ext2fs/ \
+ e2fsprogs/blkid/ \
+ e2fsprogs/uuid/ \
e2fsprogs/ \
editors/ \
findutils/

will take me a while to test it .. there are already 17 _install inside the source dir ... I think I've to delete halv of them ... even some DOTconfigs .... I am getting confused

Because initrd's find is compressed with UPX.

Next interesting news :
http://upx.sourceforge.net/

upx EXECUTABLE seems to be working pretty good
87kb > 46kb

for stpcpy try adding -lcompat to the end of your LDFLAGS and or this in the includes of the referenced .c file

OK : I'll start that again in 10 hours ... need some sleep now .

[Karl Godt]

Code: Select all

./configure LDFLAGS="-static -Wl,--gc-sections,--as-needed -lcompat" CFLAGS="-ffunction-sections -fdata-sections -Os -D _BSD_SOURCE" CC="diet gcc -nostdinc"

and adding #include <string.h> did not help . <string.h> had been only [missing] at getline.c+.h . This time I commented

Code: Select all

#ifdef _GNU_SOURCE
int vasprintf(char **strp, const char *fmt, va_list ap);
// for FIDUTILS // ssize_t getline(char **lineptr, size_t *n, FILE *stream);
// for FIDUTILS // ssize_t getdelim(char **lineptr, size_t *n, int delim, FILE *stream);
#endif

at dietlibc/include/stdio.h and commented the static definitions for stpcpy : Find did compile same size 122/87/46kb (staticdietlibc/strip/upx -7) , xargs and locate not

l10nflist.c:(.text._nl_make_l10nflist+0x1cc): undefined reference to `stpcpy'
../intl/libintl.a(l10nflist.o):l10nflist.c:(.text._nl_make_l10nflist+0x1f8): more undefined references to `stpcpy' follow
collect2: ld returned 1 exit status
make[3]: *** [xargs] Error 1
make[3]: Leaving directory `/COMPILE/FINDUTILS/findutils-4.1.20/xargs'
make[2]: *** [all-recursive] Error 1

I made a "fancy-full-blown--long-options-busybox" of statically 844kb and with lzma compressed drivers the initrd.gz became 1363kb . But busybox less and more seem to need nvidiafb or intel-agp support to work like they normally do even at the prompt. ed is described at the menuconfig from year 1970 and "really evil" and somehow it's true : working, but uncomfortable.
bb modprobe seems to ignore module.alias even if support for modules.alias is enabled ( perhaps just for depmod )

Code: Select all

MODALIASES="`cat /sys/bus/pci/devices/*/modalias`" #important, save to variable before loop.
for ONEMODALIAS in $MODALIASES
do
  modprobe -v $ONEMODALIAS #-v means verbose.
  [ $? -eq 0 ] && echo -n "." > /dev/console
done

for ONEHID in `grep -o 'MODALIAS=hid:[^ ]*' /tmp/uevents.log | cut -f 2 -d '=' | tr '\n' ' '`
 do
  modprobe $ONEHID
 done

modprobe: module pci:v000010DEd00000182sv0000107Dsd00002922bc03sc00i00 not found in modules.dep
modprobe: module hid:b0003v0000413Cp00002010 not found in modules.dep

One good new : busybox eject is working with creating a symlink to /dev/sr0 from /dev/cdrom .
Booting time : 10 seconds for loading drivers.ko.lzma , +18 seconds for searching puppy files (500GB/12-83/4-82 partitions) , +45 seconds after [Enter] to load the pupsave-2fs and drivers.ko for reaching jwm .

And this is busybox-linker.sh :

Code: Select all

#!/bin/bash
cd `pwd`
./busybox > bbcommands
cat bbcommands | sed '/^$/d' > bbcommands2
BEGIN=`cat -n bbcommands2 | grep Currently | tr -s " " | tr "\t" " " | cut -f 2 -d " "`
LINES=`cat -n bbcommands2 | wc -l` 
COMMANDBLOCKLINES=`expr $LINES - $BEGIN`
COMMANDBLOCK=`cat bbcommands2 | tail -n $COMMANDBLOCKLINES | tr ',' ' ' | tr -s " " | tr " " "\n" | tr "\t" " " | tr -d " " | sed '/^$/d'`
for ONELINE in $COMMANDBLOCK ; do
ln -s busybox $ONELINE 
done
rm bbcommands
rm bbcommands2

and bb-unlinker.sh :

Code: Select all

#!/bin/bash
cd `pwd`
file * | grep symbolic > links 
cat links | grep busybox > busyboxlinks
cat busyboxlinks | cut -f 1 -d ':' >busyboxlinks2
BBLINKS=`cat busyboxlinks2`
for ONELINE in "$BBLINKS" ;do
rm $ONELINE
done
rm links
rm busyboxlinks
rm busyboxlinks2

[technosaurus]

I made a "fancy-full-blown--long-options-busybox" of statically 844kb and with lzma compressed drivers the initrd.gz became 1363kb . But busybox less and more seem to need nvidiafb or intel-agp support to work like they normally do even at the prompt. ed is described at the menuconfig from year 1970 and "really evil" and somehow it's true : working, but uncomfortable.
bb modprobe seems to ignore module.alias even if support for modules.alias is enabled ( perhaps just for depmod )

Even if you do allyesconfig, you still need to manually enable full modutils - otherwise it uses the small version which operates differently and thus uses bb.* prefix

Re: look of more/less - I thought Barry's kernels were compiled with vesafb only ... I need to check that, b/c it had prevented me from testing some stuff before.

by the way there is a new feature that wasn't documented very well:

Code: Select all

busybox --list

I use this to place all of the links in /usr/local/bin so that if the full version exists, it gets used instead(due to the $PATH precedence)

Code: Select all

for x in `busybox --list`; do ln -s busybox $x;done

[Karl Godt]

busybox --list

Really works at 1.17.1 compiled statically and dynamicallly , and it seems with and without --long-options enabled .
The puppy bb-1.15.0 doesn't have it .

bb `umount -l` works nicely at the rdsh to unmount everything one after another for doing the filesystemcheck .
bb `reboot -f` worked , too.
bb `lspci -k` could replace `elspci -l` .

[PANZERKOPF]

Re: look of more/less - I thought Barry's kernels were compiled with vesafb only

Barry's kernels were compiled with vesafb but without console framebuffer. When we boot using vga=??? we get a black screen ( we should blindly type "modprobe fbcon" to get a working console.)
Barry's config:
CONFIG_FRAMEBUFFER_CONSOLE=m
Should be:
CONFIG_FRAMEBUFFER_CONSOLE=y