Compiling with musl (an alternate libc)

For discussions about programming, programming questions/advice, and projects that don't really have anything to do with Puppy.
Message
Author
Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#46 Post by Ibidem »

Currently 0.9.3 has been released; there are several bugfixes for ARM as well as general bugfixes, a static MIPS32 (o32 abi) port, getaddrinfo, ioperm, and iopl have been added (so it's a lot closer to building X), along with a number of syscalls and a few things that improve glibc compatability.

libc.so is the musl loader (ld-musl-$ARCH.so.1). It can now (since 0.9.2, I think) directly execute binaries. If you go
ln -s $syslibdir/libc.so ldd
it acts as ldd (with one exception:
libc.so is not reported, AFAICT)

Rich is testing ASM replacements for some string functions (memcpy is the current project; the version he has is ~2x as fast as the C version).

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#47 Post by Ibidem »

FYI, 0.9.4 may be approaching.
MIPS and ARM bugfixes, several optimisations, blowfish crypt, and a few BSD extensions such as fgetln are among the changes.
Rich is asking whether any other features are desired for the release.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#48 Post by technosaurus »

If anyone is following the musl mail list, one of the requests for help was sha1,md5
See:
https://github.com/libtom/libtomcrypt/t ... src/hashes
If you'd like to help, or at least inform them ... It seems to be more robust than the proposed alternative and unencumbered with license restrictions
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#49 Post by Ibidem »

technosaurus wrote:If anyone is following the musl mail list, one of the requests for help was sha1,md5
See:
https://github.com/libtom/libtomcrypt/t ... src/hashes
If you'd like to help, or at least inform them ... It seems to be more robust than the proposed alternative and unencumbered with license restrictions
Thanks for the info!

By the way, there's a change that just landed that you may be interested in: musl just went from straight ISO C99 by default to _XOPEN_SOURCE=700 _BSD_SOURCE by default. This means that as long as you don't pass -std=c{89,99,11}, a large portion of the functionality will be available OOB.
Also, musl is getting pretty close to C11 support (not threads yet, and there are a few remaining issues, but a large part of it is implemented).
Gregor has ~half of pkgsrc (including Xfce, Xorg, and Mesa) building, so it's fairly compatible now. A large part of the remainder is just dependency waits.

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#50 Post by Ibidem »

musl 0.9.6 has been released.

There have been a number of ~trivial bugfixes and optimizations; -O3 is no longer forced (it had been the default for shared libraries because it actually resulted in smaller binaries);
arm (and mips
technosaurus wrote:If anyone is following the musl mail list, one of the requests for help was sha1,md5
See:
https://github.com/libtom/libtomcrypt/t ... src/hashes
If you'd like to help, or at least inform them ... It seems to be more robust than the proposed alternative and unencumbered with license restrictions
md5 has been implemented, based on freebsd's implementation, which is based on libtomcrypt.
sha1 is not planned (as far as I can tell, no libc supports sha1 crypt).

User avatar
Karl Godt
Posts: 4199
Joined: Sun 20 Jun 2010, 13:52
Location: Kiel,Germany

#51 Post by Karl Godt »

Thanks, want to inform that i was able to build the Puppy guess_fstype with these parameters in the Makefile :

Code: Select all

CC = /usr/local/musl/bin/musl-gcc
CFLAGS = -W -Wall -Os -I/usr/include
LDFLAGS = -static
EXE = guess_fstype

default: $(EXE) tidy

$(EXE):
	@echo "Single compile for guess_fstype ..."
	@$(CC) -o $@ guess_fstype.c main.c $(CFLAGS) $(LDFLAGS) -s
	@ls -sl $@

tidy:
	@rm -f *.o *~ *.bak

clean: tidy
	@rm -f $(EXE)
that works with kernel that have CONFIG_DEBUG_BLOCK_EXT_DEVT=y , which diet was not able using the -nostdinc flag
statically with a size of ls -s 24 while simple gcc LDFLAGS=-static builded too with a bin of ls -s 388 .

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#52 Post by Ibidem »

musl 0.9.8 is out, and currently under discussion is the roadmap for 0.9.9 / 1.0 (as well as promotional materials and testing). If you've got any issues, it would be nice to hear about them soon.

0.9.8 features support for amd64, x86, arm (big and little endian), mips32 (hardfloat, big/little endian), powerpc (static only), and microblaze.
There are several bugfixes (math, asm, corner cases, ...). 0.9.7 had more bugfixes, though. Also, C11 CMPLX macros are now supported, and a few new features have come in.

Rob Landley is starting to switch Aboriginal over to musl, and OpenWRT has just added musl support recently.

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#53 Post by Ibidem »

jamesbond wrote:Impressive :D

This is my attempt at building dnsmasq as 32-bit executable under 64-bit environment.

....
4. Went smoothly until it tried to compile log.c, then failed - cannot find _PATH_LOG. This is defined in bits/syslog-path.h, but /opt/musl/include/bits doesn't have this. I just add _PATH_LOG to dnsmasq's config.h and compilation continued and finished.
5. But it failed at link step - saying that it skipped libc.a when searching -lc. This would not happened if I compiled for 64-bits, so I added LDFLAGS=-m32 to the make file, which now looks like this:

Code: Select all

make CC=musl-gcc  CFLAGS"=-Os -static -fno-stack-protector -D_GNU_SOURCE -m32 -I /mnt/sda2/tmp/linux-3.2.2/usr/include/" LDFLAGS=-m32
6. All went well - except that ld cannot find capset/capget.

Code: Select all

dnsmasq.c:(.text+0x1172): undefined reference to `capget'
dnsmasq.c:(.text+0x14d4): undefined reference to `capset'
dnsmasq.c:(.text+0x156a): undefined reference to `capset'
....
Questions:
1. I'm sure the above is not the only one, so the general question is: as I understand it musl strives to be POSIX-compliant (not Linux compliant), however it also strives to be a glibc replacement. musl already implements _GNU_SOURCE, what about compatibilities at this level? Unfortunately, compatibility support == bloat :?

2. Assuming:
- not trying to cross-compile (not as above)
- not trying to migrate the whole system
- building a static binary
How does libs compiled with musl co-operate with libs compiled with glibc? Let's say I already have a libxxxx compiled with glibc (let say openssl's libssl). Will this work with musl - i.e. if I use musl-gcc and include does -lssl (pointing to the appropriate -L) - will this work? Probably not, I suppose.
If not, then I need to compile openssl again and install it on, say, /opt/musl/openssl, and make sure that all my compiles use that version of libgtk+, no?
Well, I revisited this, and dnsmasq now compiles OOB (make PKG_CONFIG=true CC=musl-gcc) :)

and _if_ a library does not import glibc-only symbols, it should work with musl.
For those who are curious how compatible musl is, it's currently building:
-Wine
-Over 60% of pkgsrc
-SDL, Qt 4, Xfce and GIMP (with patches)
-FLTK and dillo (no/minimal patches)

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#54 Post by Ibidem »

musl 0.9.9 is out, with numerous bugfixes, and much-improved compatability.

Also, for reference--dnsmasq: ~180k static/stripped here.

goingnuts
Posts: 932
Joined: Sun 07 Dec 2008, 13:33
Contact:

#55 Post by goingnuts »

atm I have compiled the below packages static with success using musl 0.9.9
Attachments
snap0001.png
(39.37 KiB) Downloaded 474 times
Last edited by goingnuts on Tue 12 Mar 2013, 17:18, edited 7 times in total.

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#56 Post by Ibidem »

goingnuts wrote:atm I have compiled the below packages with success using musl 0.9.9
e2fsprogs-1.40.2
elspci-1.0
fileutils-4.1
findutils-4.1.20
freetype-2.4.10
fuse-2.6.0
gdk-pixbuf-0.21.0
giflib-4.1.4
glib-1.2.10
gtk+-1.2.10
guess_fstype_withext4_test1
icewm-1.2.37
imlib-1.9.15
jpeg-6b
jwm-686
libpng-1.2.46
ntfs-3g-2009.3.8
tiff-3.7.0
tinyxlib-0.01
util-linux-2.13-pre7
zlib-1.2.5
Looks like a good start. FYI, though, the branch I pointed you at earlier in the other thread has some changes you may want for building tinyX. The main issue is getting vm86 integrated into the build system properly.
However, I can probably just use Technosaurus's approach and drop that dependency.

You may want to use libpng 1.2.50 for security reasons (there's a bad bug that's fixed in 1.2.49, and a trivial issue fixed in 1.2.50).

Busybox works, but don't enable regexes in busybox vi.

goingnuts
Posts: 932
Joined: Sun 07 Dec 2008, 13:33
Contact:

#57 Post by goingnuts »

I will update the list in my previous post as new things come up (if its ok for you...). I have worked further on tinyXlib, slowly working towards X11R-6.7.0 source files. I will upgrade to libpng 1.2.50 - thx.

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#58 Post by Ibidem »

goingnuts wrote:I will update the list in my previous post as new things come up (if its ok for you...). I have worked further on tinyXlib, slowly working towards X11R-6.7.0 source files. I will upgrade to libpng 1.2.50 - thx.
That's fine, nice to see progress.

Is this going to go in pupngo? It looks like it should be enough.

I'm contemplating mentioning puplets that use musl to some extent on the musl wiki's list of distros; currently, I think that means Fatdog64.

goingnuts
Posts: 932
Joined: Sun 07 Dec 2008, 13:33
Contact:

#59 Post by goingnuts »

Ibidem wrote:Is this going to go in pupngo? It looks like it should be enough.
A musl-dog that would be :)

goingnuts
Posts: 932
Joined: Sun 07 Dec 2008, 13:33
Contact:

#60 Post by goingnuts »

I am having problems getting util-linux to build (tried 2.13-pre7 & util-linux-2.22) both with their own problems. I have also tried to search for util-linux-musl-patches but not found something usable...Any hints?
I only need 4 of the aps from it (rev/mount/umount/losetup) but they are quite important as they go in initrd...

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#61 Post by Ibidem »

goingnuts wrote:I am having problems getting util-linux to build (tried 2.13-pre7 & util-linux-2.22) both with their own problems. I have also tried to search for util-linux-musl-patches but not found something usable...Any hints?
I only need 4 of the aps from it (rev/mount/umount/losetup) but they are quite important as they go in initrd...
Is there a reason you can't use busybox for these?

All the projects I know of so far have used busybox, but...perhaps starch would have that...Ah yes:
https://github.com/StarchLinux/starch-p ... x/PKGBUILD
Not quite enough info to build here; you need to build PAM or disable su, login, and sulogin. Also it seems you might have better luck with 2.21.5.

OK...just going at it my own way (2.22.2)...
./configure --without-udev --without-slang --without-selinux --without-audit --disable-login --disable-su --disable-sulogin --disable-schedutils --disable-nls
CFLAGS="-D_GNU_SOURCE -Os"
I inserted this in include/cpuset.h below the first include:

Code: Select all

#include <sys/types.h>
#ifndef __cpu_mask
#define __NCPUBITS (8 * sizeof(__cpu_mask))
typedef unsigned long int __cpu_mask;
typedef struct
{
  __cpu_mask __bits[ 1024 / __NCPUBITS ];
} cpu_set_t;
#endif
Now it fails in a few places due to a missing <sys/ttydefaults.h> (CEOL et al.); cfdisk is using __sighandler_t instead of sighandler_t; those are most of the problems...

goingnuts
Posts: 932
Joined: Sun 07 Dec 2008, 13:33
Contact:

#62 Post by goingnuts »

Busybox bins will do in initrd but u/mount-FULL is needed in some scripts later (functions4puppy4, rc.shutdown a.o.) so I include them in initrd and copy before switchroot.

Your hints made me able to compile util-linux-2.22...but the mount-bin segfaults.
Below end of running strace on the mount:
stat64("/run", 0xbff4ede8) = -1 ENOENT (No such file or directory)
lstat64("/dev/.mount/utab", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
open("/dev/.mount/utab", O_RDWR|O_CREAT|O_LARGEFILE, 0644) = 3
close(3) = 0
lstat64("/etc/mtab", {st_mode=S_IFLNK|0777, st_size=12, ...}) = 0
open("/proc/self/mountinfo", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
open("/proc/mounts", O_RDONLY|O_LARGEFILE) = 3
readv(3, [{"", 0}, {"rootfs / rootfs rw 0 0\n/dev/sdc5"..., 1024}], 2) = 876
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++
Well - I will see if I can get the old util-linux-2.13-pre7 working...

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#63 Post by Ibidem »

goingnuts wrote:Busybox bins will do in initrd but u/mount-FULL is needed in some scripts later (functions4puppy4, rc.shutdown a.o.) so I include them in initrd and copy before switchroot.

Your hints made me able to compile util-linux-2.22...but the mount-bin segfaults.
Below end of running strace on the mount:
stat64("/run", 0xbff4ede8) = -1 ENOENT (No such file or directory)
lstat64("/dev/.mount/utab", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
open("/dev/.mount/utab", O_RDWR|O_CREAT|O_LARGEFILE, 0644) = 3
close(3) = 0
lstat64("/etc/mtab", {st_mode=S_IFLNK|0777, st_size=12, ...}) = 0
open("/proc/self/mountinfo", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
open("/proc/mounts", O_RDONLY|O_LARGEFILE) = 3
readv(3, [{"", 0}, {"rootfs / rootfs rw 0 0\n/dev/sdc5"..., 1024}], 2) = 876
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++
Well - I will see if I can get the old util-linux-2.13-pre7 working...
If you see this issue again, would you mind posting what the output from gdb "where" or "bt full" is?

goingnuts
Posts: 932
Joined: Sun 07 Dec 2008, 13:33
Contact:

#64 Post by goingnuts »

Attached output of both...thx.
I compiled util-linux-2.22 with
CFLAGS="-g -D_BSD_SOURCE -D_GNU_SOURCE -D__KERNEL_STRICT_NAMES" LDFLAGS="-static -Xcompiler -static" \
./configure --prefix=/usr --without-udev --without-slang --without-selinux --without-audit --disable-login --disable-su --disable-sulogin --disable-schedutils --disable-nls --disable-shared --disable-fsck --without-ncurses --disable-utmpdump
and had to remove fsck.minix$(EXEEXT) and hexdump$(EXEEXT) from Makefile after configure to get on building - but I do not think that has anything to do with it. Attached applied patch as well...
Attachments
util-linux-2.22.diff.gz
fake .gz
(1.4 KiB) Downloaded 397 times
mount_dbg.txt.gz
fake .gz - output of gdb mount
(7.26 KiB) Downloaded 382 times

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#65 Post by jamesbond »

Ibidem,

I used to be able to cross-compile 32-bit static binaries with my 64-bit compiler - ie, under Fatdog64's 64-bit gcc, I built 32-bit musl static libs and use that to compile a program (documented in the first page of this thread).

I just tried recent version of musl (9 mar 2013) and I can't do that anymore, using exactly the same config.mak. Is this capability removed?

The error was this: it keeps trying to load crtbegin.o. I fixed this by editing musl-gcc specs file, but then it failed to find "__dividi3" symbol. In case you're wondering, the program I tried to build is guess_fstype.

cheers!
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

Post Reply