Need examples of static compiling of various apps.

For discussions about programming, programming questions/advice, and projects that don't really have anything to do with Puppy.
Message
Author
User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#41 Post by sunburnt »

Thanks penguinpowerppp; More of the picture is taking shape.

So glibc is not so good and libc is good. Are musl and uclibc better than libc?
libc seems like a good one as it`s main stream, and I think none are compatible.
Another words, apps. compiled with one are not going to work on any others.

Clarify: "Add -L/path to your CFLAGS", Correct use is: "configure -L/path"?

Please clarify the difference between -L CFLAGS and -I includes (RPATH?).

### -L sets /PathToLib , -I copies the lib. into the build and sets /PathToLib ?

None of this has to do with static libs. (*.a) which are compiled into the build?

Thanks for -nostdlib and -nostdinc , no one has said anything about them.
Both keep system libs. out of the build (good), -nostdinc seems best to use.
In using these I must have all the deps. compiled before building the app.
So I can keep a dir. of compiled libs. and use " -nostdinc -I " to point to it?

Thank again... Terry
Last edited by sunburnt on Sat 26 May 2012, 19:30, edited 1 time in total.

jpeps
Posts: 3179
Joined: Sat 31 May 2008, 19:00

#42 Post by jpeps »

sunburnt wrote:Thanks penguinpowerppp; More of the picture is taking shape.
reference to technosaurus' wife?

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#43 Post by technosaurus »

glibc is short for gnu libc - it is a libc, but so many things get inlined that a static compile adds half a Mb for a simple hello world. eglibc is slightly better depending on the config options and musl, uclibc and diet were designed with static compiling in mind; whereas Ulrich Drepper formerly (mis-)guided the gnu libc to being only effective as a shared library and only on x86 platforms

-I is for the locations of specific .h files (with -nostdinc it will only look there)
-L is for the locations of .a and .so files (with -nostdlib it will only look there)
-D is used to define a symbol (for example adding -DNODROIDFONT saves mupdf from building in an unnecessary 3Mb font)
you can use as many of these as you want

to add it to CFLAGS/LDFLAGS it would be

Code: Select all

CFLAGS="-Os -nostdinc -I/usr/musl/include -DNODROIDFONT" \
LDFLAGS="-nostdlib -L/usr/musl/lib" \
./configure --prefix=/usr/musl ...
I have never found a good use for rpath except for during testing
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

jpeps
Posts: 3179
Joined: Sat 31 May 2008, 19:00

#44 Post by jpeps »

"I do not know where to find the historic references, but yes, static linking is dead on GNU systems. (I believe it died during the transition from libc4/libc5 to libc6/glibc 2.x.)"


http://stackoverflow.com/questions/3430 ... ng-is-dead

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#45 Post by sunburnt »

Yeah, everyone has their own ideas about how things should be... (glibc)

Thanks again technosaurus; I wanted full examples and I`m getting scraps.
It`s hard to put it all together when you`ve never seen a complete example.

# Lots more Qs to clarify further if I could ( numbered As for each ):

0 ) Is it correct that none of these Cs will run apps. made on the other ones?
___ This could be a really serious problem for portable apps. ( hope not...)

1 ) "-nostdinc -I..." used only in CFLAGS and "-nostdlib -L..." in LDFLAGS ?
___ This seems to be a very critical piece of info...

2 ) To use -nostdinc or -nostdlib means all "ldd" lib. deps. are needed?

3 ) Used alone, -L locations are searched before the system`s lib. paths are?

4 ) -D(symbol) , but how to know what potentially unneeded items there are?

5 ) -static is used with make? Need a clear picture of what it does exactly.

6 ) Using downloaded .so files. is a gamble. Is there a good way to do it?

7 ) The advantage to .a files is they compile with and into the app.?
___ Possible at the same time to compile .so files with and into the app.?

8 ) Method to make complete tree files for app. cataloging and statistics?
___ There`s no way to do this for uncompiled apps.? Need working app.?

### This is for "relatively" self contained Squash file apps.
# Target is: ./configure --prefix=/SqApp/mnt/( AppPackageNameNoExt.)

# Things like X, GTK+, other O.S. deps. and other deps. should be shared.
# Different versions of GTK+ and other deps could give problems though...

The more complicated apps. give me lib. problems. Hope this helps!!!

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#46 Post by Ibidem »

sunburnt wrote: 0 ) Is it correct that none of these Cs will run apps. made on the other ones?
___ This could be a really serious problem for portable apps. ( hope not...)
Basically correct. Also largely irrelevant.
Almost every desktop linux (but not DeLi, pupngo, alpine, sabotage, a few other rarely used distros, or android) uses glibc2/libc6 or eglibc (a fork of glibc that's fully compatible)

musl is currently (0.9.1) working on LSB ABI support, which means a degree of interoperability with glibc.

uclibc doesn't even retain compatibility across versions (binaries built with 0.9.29 won't run on 0.9.30).

HOWEVER, if you compile statically (-static or -Bstatic), you can build one program and run it on just about any machine of the same architecture. You do not need ANY C library installed to run a static binary.
1 ) "-nostdinc -I..." used only in CFLAGS and "-nostdlib -L..." in LDFLAGS ?
___ This seems to be a very critical piece of info...
I would use -nostdinc -I... in CFLAGS & CPPFLAGS; -nostdlib -L... would go in CFLAGS & LDFLAGS
This is because I've seen several subtly broken build systems that used gcc $CFLAGS for linking, or ran a preprocessor that ignored CFLAGS, or ignored CPPFLAGS altogether...
2 ) To use -nostdinc or -nostdlib means all "ldd" lib. deps. are needed?
Yes, including -lc
3 ) Used alone, -L locations are searched before the system`s lib. paths are?
Yes, as long as they come before the libraries that are searched for.
IIRC, gcc .... -L./ -labc would find ./libabc.a, while -labc -L./ would not.
If you had ./libm.so, the first pattern would link against it, while the second pattern would use /lib/libm.so
4 ) -D(symbol) , but how to know what potentially unneeded items there are?
In advance? Find someone else's build scripts or read the source code.
It's simplest to try building, then look up whatever stops the build.
5 ) -static is used with make? Need a clear picture of what it does exactly.
When using make or any normal build system,

Code: Select all

export CFLAGS="-static $CFLAGS"
before configuring or building.
CFLAGS is extra options (flags) for the C compiler to use.
With a simple make-based build system,
make CFLAGS=-static
should be enough.

I'll try giving a simple example:
If compiling with gcc,

Code: Select all

gcc -Os -static -D_XOPEN_SOURCE minimp3.c  -lm -o minimp3
will/"should" compile minimp3 fully static.
In other words, only .a libraries are searched for symbols/functions *; any code that the program needs will be linked into the binary. When I say linked into, I mean that the object files become part of the statically linked program/app/binary.
Dynamically linked (using shared libraries/ .so files) programs just request at runtime that the .so file be put in RAM with its code available for anything to call. Sometimes "linked into" is used to refer to this, but it doesn't quite fit what's happening.

(*it may fallback to shared libraries, according to technosaurus)
6 ) Using downloaded .so files. is a gamble. Is there a good way to do it?
7 ) The advantage to .a files is they compile with and into the app.?
___ Possible at the same time to compile .so files with and into the app.?
Not clear what you mean compile with and into.
Do you mean "Will running

Code: Select all

make 
produce the .so files and an app linked against them at the same time " ?

If so: yes

Long answer: a .so file contains code that is not included in the binary (app) linked against it. So no, it is not linked into the app.
8 ) Method to make complete tree files for app. cataloging and statistics?
___ There`s no way to do this for uncompiled apps.? Need working app.?

### This is for "relatively" self contained Squash file apps.
# Target is: ./configure --prefix=/SqApp/mnt/( AppPackageNameNoExt.)
I'm guessing this would be like

Code: Select all

make install DESTDIR=../pkg-temp
?
(if I understand correctly)
DESTDIR makes it install as if DESTDIR were /

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#47 Post by sunburnt »

Wow! Thanks Ibidem; Succinct answers, what a great tutorial.

1) So this answers the static Q, it includes the C lib. in with the binary.

4) Yep, that`s the conclusion I`ve come to. Let the errors fly, it`s the easy way.

6) I`ve found that Ubuntu Lucid libs. work well in Puppy Lucid. Same base.

7) Correct, the ".so" libs. and app. binaries are built together. And dyn. linked.

I want tree files as manifest files to track files and deps. in a build.
____ I found ldd-recursive.pl and it s great, I wrote one also but this is better.

So gathering source libs. into a dir. as they`re needed will make a large enough
lib. base that eventually only the apps. + other deps. will need to be downloaded.
Time to clear a little hd space and gather more apps.

Again many thanks... Terry B.
Last edited by sunburnt on Tue 05 Jun 2012, 20:58, edited 1 time in total.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#48 Post by technosaurus »

when doing a static build it does not include the entire library (though it may as well be with gnu libc) only the required symbols (basically the needed functions and data) - but sometimes the compiler has a difficult time telling which are really needed. Fortunately there _are_ a couple of helper flags for this.
Recent versions of gcc use -flto (and others releated to it), but it is not supported in puppy; however you can use -ffunction-sections -fdata-sections in CFLAGS and -Wl,--gc-sections in LDFLAGS so long as the app does not use plugins (if a function is only used in a plugin or only called from an XML UI file then the linker can't see it and will discard the symbol)

ldd is "recursive" too, so I am not sure how a perl script helps except for different formatting

use ldd <binary> to know _all_ dependencies - this is what static libs you need for building that binary.

use objdump -x <binary> | grep NEEDED to know only the direct dependencies - this is what minimum shared libs you will need
If you tell gcc to link to any additional libraries and do not include the --as-needed LDFLAG, then a recompile of a library or an indirect dependency such as libxcb can and often will break the app - relying on packages' makefiles and configure scripts will cause this issue 99% of the time and sometimes --as-needed can cause an issue (particularly in apps that can use plugins) If --as-needed won't work there is a hack you can get away with most of the time. Delete the binary and run a verbose make again - then copy the gcc command (or deduce it from the crappy libtool command) and just remove the unneeded libs
For gtk1 apps, you can remove -lglib -lgdk etc.. because just doing -lgtk will automatically take care of all of its dependencies and have the side effect of being significantly smaller due to not storing the locations of symbols in the indirect dependencies which has the additional side effect that when those symbols change, it doesn't break the binary (see numerous issues with xcb and the broken behavior of linking directly to it instead of just -lX11 because of pkgconfig shortcomings).

another note on this is that quite often you will see one library linked multiple times (especially X11) - this is often unnecessary and can reduce binary size by removing additional instances and changing the order that the -l<libs> appear (hint the proper order is related to the dependency chain).

there is one other thing you may want to know about when trying to package a standalone app - the other files that it uses such as config files and resources like fonts, images or UI files. This can be accomplished by running lsof (not included in puppy, but configurable in recent versions of busybox)

Some tips specifically for building static _binaries_:
see --gc-sections above (savings between 20 and 80%)
objects in .a archives that will be used for static linking _binaries_ (a.k.a. static libs) should not be built with -fpic or -fPIC (savings of up to 5%)
The symbols will not need to be visible at all, since the user will only directly be calling main, so you can use -fvisibility=hidden in your CFLAGS. This also reduces the size of the elf headers, GOT, etc... as much as 20%
Note: if they will be used for building into libraries, ignore all but the --gc-sections part (very uncommon, - I tried it with GTK2, but its build system fought me too much)
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#49 Post by sunburnt »

Wow again! Thanks technosaurus; This really gives learned detail to it all.
I`ll have to reread it as I go through the steps myself.

I looked at the 2 outputs from ldd and ldd-recursive.pl
It looks like you`re right that ldd does give a binary`s deps. and the dep`s. deps.
I thought it just output the binary`s deps. only, leaving out a bunch of lib. deps.

ldd-recursive.pl outputs the tree with lots of duplicate libs.
A line count for the 2 outputs:

Code: Select all

sh-4.1# ldd /mnt/sdb3/ubuntu_apps/lbreakout2_2.6.1/usr/bin/lbreakout2 |wc -l
19
sh-4.1# ldd-recursive.pl /mnt/sdb3/ubuntu_apps/lbreakout2_2.6.1/usr/bin/lbreakout2 |wc -l
840
As I said, most of the 840 dep. lines are dups., but it outputs a complete dep. tree.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#50 Post by technosaurus »

To answer a previous question, it is perfectly legitimate to mix static and shared libs. Check the Gparted thread that ttuuxxx and I had a couple years back, all of the gtkmm related libs are built-in statically (barry does this too, but his static libs aren't optimized for it). Usually this is done on direct dependencies only, but occasionally an indirect dependency sneaks in statically because the .so symlink is unintentionally missing from the devx.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#51 Post by sunburnt »

Amazingly I`m actually starting to understand ( I think...) most of the conversations.

We always tend to complicate things when we`re new to them.
I understand this and attempt to look at it simply, but it usually doesn`t help.

Experience is the best teacher ( but a rough one...), I need to get my game on.
Then I`ll have a whole new list of Qs, but they`ll come from actual problems.

Thanks technosaurus, Ibidem, jpeps, penguinpowerppp, and all who helped.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#52 Post by sunburnt »

### Guys, Qs...


### Q #1

Upon running an app. it errors:

The package file:LLK_DATA_PATH/pak/Classical does not exists.

# There is this file: share/llk_linux/Classical.pak

It seems this is what it`s looking for, but the /path/file is wrong.

# How do I set LLK_DATA_PATH, or supply the missing file?


### Q #2:

I`ve gotten this error from a few apps. I`ve tried:

/mnt/sdb3/ubuntu_apps/usr/games/pinball: /usr/lib/libjpeg.so.62: no version information available (required by /root/my-applications/lib/libSDL_image-1.2.so.0)

Lib`s. there, but it doesn`t like it for some reason.

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#53 Post by Ibidem »

sunburnt wrote:### Guys, Qs...


### Q #1

Upon running an app. it errors:

The package file:LLK_DATA_PATH/pak/Classical does not exists.

# There is this file: share/llk_linux/Classical.pak

It seems this is what it`s looking for, but the /path/file is wrong.

# How do I set LLK_DATA_PATH, or supply the missing file?


### Q #2:

I`ve gotten this error from a few apps. I`ve tried:

/mnt/sdb3/ubuntu_apps/usr/games/pinball: /usr/lib/libjpeg.so.62: no version information available (required by /root/my-applications/lib/libSDL_image-1.2.so.0)

Lib`s. there, but it doesn`t like it for some reason.
1: Check with strace (strace ... 2>&1 |grep /pak/)
If it's a variable, try export ...

2:No idea...except that you should see how LD_LIBRARY_PATH=/mnt/... works

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#54 Post by jamesbond »

sunburnt wrote: ### Q #2:

I`ve gotten this error from a few apps. I`ve tried:

/mnt/sdb3/ubuntu_apps/usr/games/pinball: /usr/lib/libjpeg.so.62: no version information available (required by /root/my-applications/lib/libSDL_image-1.2.so.0)

Lib`s. there, but it doesn`t like it for some reason.
This should not stop the app from running. The message is just a warning that means libjpeg in your system is built differently from the one used to compile pinball. See here and here.
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

muggins
Posts: 6724
Joined: Fri 20 Jan 2006, 10:44
Location: hobart

#55 Post by muggins »

Re: Q1, if you download the compiled llk_linux-2.0.tar.gz it will give that error as the binary has been compiled to look for the wav & .pak files in /usr/local/llk_linux/pak directory.

If I run turma on the extracted tarball directory, it shows the binary has only one reference to /usr/local/llk_linux.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#56 Post by sunburnt »

Thanks again Ibidem; I know Tiny Core`s SCM files use LD_... to fix paths.

Thanks jamesbond; The links you gave verified that it`s not critical.
It`s the only error reported in some of the apps. I`ve made.
Obviously the best solution is to find the correct lib.

And thanks muggins; I`ll look into fixing that path, but what about the file?

It doesn`t seem to be a config. file that`s generated.
And I don`t think I missed downloading a Data pak. file for it.

# Maybe a link: local/llk_linux/pak/Classical => share/llk_linux/Classical.pak ?

I couldn`t figure out what path it wanted, how did you find it?. With Turma?

I need to make a SqApp of Turma, do you have a .pet or SFS file?

muggins
Posts: 6724
Joined: Fri 20 Jan 2006, 10:44
Location: hobart

#57 Post by muggins »

Yes, I used turma. I'm also not sure where the path is being set but, if you ran ./configure --prefix=/usr then, before running make, you could try running turma to see if there are any instances of /usr/local and, if there are, change them to /usr.

gtk1-based turma is in the puppy ibiblio repository for the pre-gtk2 base pups. Or, if you want to try gtk2-based turma2, just do a forum search of the additional software section.

Post Reply