Need examples of static compiling of various apps.

[sunburnt]

Agreed, doing programming of any type, or compiling anything is an exercise
in thankless and endless updating, sometimes needed and many times not.

The idea is for app. pkgs. that are widely usable and will keep working longer.
I made a static Skype SFS many years ago, and it still works in many cases.
The hope is to leap frog generations of apps. and libs., thus reducing effort.

I see little difference between app. versions, browsers are the exception.
Wait until they do the inevitable, they make the old versions unusable ( M$ ).

[jpeps]

In general, it's probably easiest to just choose the distro that already includes most everything you need. I wonder what percentage of users have ever compiled anything.

[sunburnt]

Very few I`m sure, I`ve avoided it for years now as I`ve had such bad luck.

I`d hope some well traveled app. compilers would help list media and games.
What are the common deps. for the popular media apps., and for the games?
Also X, xVesa, xOrg, GTK+, pango, cairo, and so many more needed items.

Tiny Core`s stripped, a good starting point, but it needs lots more added to it.
Debian or Ubuntu are probably the best choices for building the app. pkgs.
But would the apps. run properly on Tiny Core? Static apps. should, but...

[jpeps]

Have you checked out CDE? Automatic packaging of Code,Data,Environment. I used it a while ago to send exactimage to Dingo, and he was able to run it his own distro.
http://www.pgbovine.net/cde.html

[sunburnt]

Thanks jpeps; I downloaded it, not sure what to try it on.
The big problem I see is the "having to use everything in the app."...

# I compiled Fox Kit, I realized I should have put it in: /SqApp/mnt/Xfe/usr
But the only *.a libs. are: /lib/libCHART-1.6.a and: /lib/libFOX-1.6.a
Do I need to recompile it, or as the libs. are to be included, does it matter?
I think the path would not matter if just the code in the .a files is being used.

As was said, I should remove all Fox`s *.so libs. and links from the build dir.

# I was surprised to see the kit has calculator, file browser, and editor apps.!

Note: I need a faster PC, Fox Tool Kit took forever to compile at 20 MB total.

[jpeps]

Thanks jpeps; I downloaded it, not sure what to try it on.
The big problem I see is the "having to use everything in the app."...

Here's an example of my calc script, which uses tcl-tk. You can run calc.cde inside the cde-package without installing tcl-tk or anything else. Everything is there, but you could also remove anything you wanted to make the package smaller. I could also add other scripts that use tcl-tk within the same package, and they would just use the same libs. Note my original script is present, which can be altered to suit (files are in the cde-root dir).

http://jpeters.net/apps/cde-package.tar.gz

[sunburnt]

Thanks, I`ll take a look at it, examples are the best learning tool.

In thinking about it, you need a working app., so a big distro. with everything?
You could try all the apps. and use the ones you want with out all the building.

Plus he talks about compressing it, and then uncompressing it to use it.
But if it were in a Squash file then all the effort isn`t needed, just mount it.

### Please look at the new forum thread I started in "Compiling":
http://www.murga-linux.com/puppy/viewto ... 595#629595

[jpeps]

The other advantage is that you can move the package anywhere (pen drive, etc), since nothing gets installed.

[sunburnt]

More thoughts about static compiling...

# 1) How to point to libs. to be used in the build?
............. If I build libs. for the app. pkg., I don`t want them in /lib or /usr/lib

# 2) Tried pointing to a lib.: LD_LIBRARY_PATH=/Path/Lib:$LD_LIBRARY_PATH
............. LD_LIBRARY_PATH is the first thing parsed for finding libs. Didn`t work.

# 3) In a static build, how to control what`s to be added and what`s not?
............. If it`s totally static, the kernel would be in the pkg. too. Add / Deny lists?

[penguinpowerppp]

Add -L/path to your CFLAGS, where path is the directory with your lib
if you don't want it to even look in /lib or /usr/lib use -nostdlib
similarly use -nostdinc and -I/path to do the same for includes

There is no way to link in the kernel all you can do is interact via syscalls
Which is what libc does but glibc is really bad for static links...musl or uclibc are better and dietlibc has other issues

[sunburnt]

Thanks penguinpowerppp; More of the picture is taking shape.

So glibc is not so good and libc is good. Are musl and uclibc better than libc?
libc seems like a good one as it`s main stream, and I think none are compatible.
Another words, apps. compiled with one are not going to work on any others.

Clarify: "Add -L/path to your CFLAGS", Correct use is: "configure -L/path"?

Please clarify the difference between -L CFLAGS and -I includes (RPATH?).

### -L sets /PathToLib , -I copies the lib. into the build and sets /PathToLib ?

None of this has to do with static libs. (*.a) which are compiled into the build?

Thanks for -nostdlib and -nostdinc , no one has said anything about them.
Both keep system libs. out of the build (good), -nostdinc seems best to use.
In using these I must have all the deps. compiled before building the app.
So I can keep a dir. of compiled libs. and use " -nostdinc -I " to point to it?

Thank again... Terry

[jpeps]

Thanks penguinpowerppp; More of the picture is taking shape.

reference to technosaurus' wife?

[technosaurus]

glibc is short for gnu libc - it is a libc, but so many things get inlined that a static compile adds half a Mb for a simple hello world. eglibc is slightly better depending on the config options and musl, uclibc and diet were designed with static compiling in mind; whereas Ulrich Drepper formerly (mis-)guided the gnu libc to being only effective as a shared library and only on x86 platforms

-I is for the locations of specific .h files (with -nostdinc it will only look there)
-L is for the locations of .a and .so files (with -nostdlib it will only look there)
-D is used to define a symbol (for example adding -DNODROIDFONT saves mupdf from building in an unnecessary 3Mb font)
you can use as many of these as you want

to add it to CFLAGS/LDFLAGS it would be

Code: Select all

CFLAGS="-Os -nostdinc -I/usr/musl/include -DNODROIDFONT" \
LDFLAGS="-nostdlib -L/usr/musl/lib" \
./configure --prefix=/usr/musl ...

I have never found a good use for rpath except for during testing

[jpeps]

"I do not know where to find the historic references, but yes, static linking is dead on GNU systems. (I believe it died during the transition from libc4/libc5 to libc6/glibc 2.x.)"


http://stackoverflow.com/questions/3430 ... ng-is-dead

[sunburnt]

Yeah, everyone has their own ideas about how things should be... (glibc)

Thanks again technosaurus; I wanted full examples and I`m getting scraps.
It`s hard to put it all together when you`ve never seen a complete example.

# Lots more Qs to clarify further if I could ( numbered As for each ):

0 ) Is it correct that none of these Cs will run apps. made on the other ones?
___ This could be a really serious problem for portable apps. ( hope not...)

1 ) "-nostdinc -I..." used only in CFLAGS and "-nostdlib -L..." in LDFLAGS ?
___ This seems to be a very critical piece of info...

2 ) To use -nostdinc or -nostdlib means all "ldd" lib. deps. are needed?

3 ) Used alone, -L locations are searched before the system`s lib. paths are?

4 ) -D(symbol) , but how to know what potentially unneeded items there are?

5 ) -static is used with make? Need a clear picture of what it does exactly.

6 ) Using downloaded .so files. is a gamble. Is there a good way to do it?

7 ) The advantage to .a files is they compile with and into the app.?
___ Possible at the same time to compile .so files with and into the app.?

8 ) Method to make complete tree files for app. cataloging and statistics?
___ There`s no way to do this for uncompiled apps.? Need working app.?

### This is for "relatively" self contained Squash file apps.
# Target is: ./configure --prefix=/SqApp/mnt/( AppPackageNameNoExt.)

# Things like X, GTK+, other O.S. deps. and other deps. should be shared.
# Different versions of GTK+ and other deps could give problems though...

The more complicated apps. give me lib. problems. Hope this helps!!!

[Ibidem]

0 ) Is it correct that none of these Cs will run apps. made on the other ones?
___ This could be a really serious problem for portable apps. ( hope not...)

Basically correct. Also largely irrelevant.
Almost every desktop linux (but not DeLi, pupngo, alpine, sabotage, a few other rarely used distros, or android) uses glibc2/libc6 or eglibc (a fork of glibc that's fully compatible)

musl is currently (0.9.1) working on LSB ABI support, which means a degree of interoperability with glibc.

uclibc doesn't even retain compatibility across versions (binaries built with 0.9.29 won't run on 0.9.30).

HOWEVER, if you compile statically (-static or -Bstatic), you can build one program and run it on just about any machine of the same architecture. You do not need ANY C library installed to run a static binary.

1 ) "-nostdinc -I..." used only in CFLAGS and "-nostdlib -L..." in LDFLAGS ?
___ This seems to be a very critical piece of info...

I would use -nostdinc -I... in CFLAGS & CPPFLAGS; -nostdlib -L... would go in CFLAGS & LDFLAGS
This is because I've seen several subtly broken build systems that used gcc $CFLAGS for linking, or ran a preprocessor that ignored CFLAGS, or ignored CPPFLAGS altogether...

2 ) To use -nostdinc or -nostdlib means all "ldd" lib. deps. are needed?

Yes, including -lc

3 ) Used alone, -L locations are searched before the system`s lib. paths are?

Yes, as long as they come before the libraries that are searched for.
IIRC, gcc .... -L./ -labc would find ./libabc.a, while -labc -L./ would not.
If you had ./libm.so, the first pattern would link against it, while the second pattern would use /lib/libm.so

4 ) -D(symbol) , but how to know what potentially unneeded items there are?

In advance? Find someone else's build scripts or read the source code.
It's simplest to try building, then look up whatever stops the build.

5 ) -static is used with make? Need a clear picture of what it does exactly.

When using make or any normal build system,

Code: Select all

export CFLAGS="-static $CFLAGS"

before configuring or building.
CFLAGS is extra options (flags) for the C compiler to use.
With a simple make-based build system,
make CFLAGS=-static
should be enough.

I'll try giving a simple example:
If compiling with gcc,

Code: Select all

gcc -Os -static -D_XOPEN_SOURCE minimp3.c  -lm -o minimp3

will/"should" compile minimp3 fully static.
In other words, only .a libraries are searched for symbols/functions *; any code that the program needs will be linked into the binary. When I say linked into, I mean that the object files become part of the statically linked program/app/binary.
Dynamically linked (using shared libraries/ .so files) programs just request at runtime that the .so file be put in RAM with its code available for anything to call. Sometimes "linked into" is used to refer to this, but it doesn't quite fit what's happening.

(*it may fallback to shared libraries, according to technosaurus)

6 ) Using downloaded .so files. is a gamble. Is there a good way to do it?

7 ) The advantage to .a files is they compile with and into the app.?
___ Possible at the same time to compile .so files with and into the app.?

Not clear what you mean compile with and into.
Do you mean "Will running

Code: Select all

make 

produce the .so files and an app linked against them at the same time " ?

If so: yes

Long answer: a .so file contains code that is not included in the binary (app) linked against it. So no, it is not linked into the app.

8 ) Method to make complete tree files for app. cataloging and statistics?
___ There`s no way to do this for uncompiled apps.? Need working app.?

### This is for "relatively" self contained Squash file apps.
# Target is: ./configure --prefix=/SqApp/mnt/( AppPackageNameNoExt.)

I'm guessing this would be like

Code: Select all

make install DESTDIR=../pkg-temp

?
(if I understand correctly)
DESTDIR makes it install as if DESTDIR were /

[sunburnt]

Wow! Thanks Ibidem; Succinct answers, what a great tutorial.

1) So this answers the static Q, it includes the C lib. in with the binary.

4) Yep, that`s the conclusion I`ve come to. Let the errors fly, it`s the easy way.

6) I`ve found that Ubuntu Lucid libs. work well in Puppy Lucid. Same base.

7) Correct, the ".so" libs. and app. binaries are built together. And dyn. linked.

I want tree files as manifest files to track files and deps. in a build.
____ I found ldd-recursive.pl and it s great, I wrote one also but this is better.

So gathering source libs. into a dir. as they`re needed will make a large enough
lib. base that eventually only the apps. + other deps. will need to be downloaded.
Time to clear a little hd space and gather more apps.

Again many thanks... Terry B.

[technosaurus]

when doing a static build it does not include the entire library (though it may as well be with gnu libc) only the required symbols (basically the needed functions and data) - but sometimes the compiler has a difficult time telling which are really needed. Fortunately there _are_ a couple of helper flags for this.
Recent versions of gcc use -flto (and others releated to it), but it is not supported in puppy; however you can use -ffunction-sections -fdata-sections in CFLAGS and -Wl,--gc-sections in LDFLAGS so long as the app does not use plugins (if a function is only used in a plugin or only called from an XML UI file then the linker can't see it and will discard the symbol)

ldd is "recursive" too, so I am not sure how a perl script helps except for different formatting

use ldd <binary> to know _all_ dependencies - this is what static libs you need for building that binary.

use objdump -x <binary> | grep NEEDED to know only the direct dependencies - this is what minimum shared libs you will need
If you tell gcc to link to any additional libraries and do not include the --as-needed LDFLAG, then a recompile of a library or an indirect dependency such as libxcb can and often will break the app - relying on packages' makefiles and configure scripts will cause this issue 99% of the time and sometimes --as-needed can cause an issue (particularly in apps that can use plugins) If --as-needed won't work there is a hack you can get away with most of the time. Delete the binary and run a verbose make again - then copy the gcc command (or deduce it from the crappy libtool command) and just remove the unneeded libs
For gtk1 apps, you can remove -lglib -lgdk etc.. because just doing -lgtk will automatically take care of all of its dependencies and have the side effect of being significantly smaller due to not storing the locations of symbols in the indirect dependencies which has the additional side effect that when those symbols change, it doesn't break the binary (see numerous issues with xcb and the broken behavior of linking directly to it instead of just -lX11 because of pkgconfig shortcomings).

another note on this is that quite often you will see one library linked multiple times (especially X11) - this is often unnecessary and can reduce binary size by removing additional instances and changing the order that the -l<libs> appear (hint the proper order is related to the dependency chain).

there is one other thing you may want to know about when trying to package a standalone app - the other files that it uses such as config files and resources like fonts, images or UI files. This can be accomplished by running lsof (not included in puppy, but configurable in recent versions of busybox)

Some tips specifically for building static _binaries_:
see --gc-sections above (savings between 20 and 80%)
objects in .a archives that will be used for static linking _binaries_ (a.k.a. static libs) should not be built with -fpic or -fPIC (savings of up to 5%)
The symbols will not need to be visible at all, since the user will only directly be calling main, so you can use -fvisibility=hidden in your CFLAGS. This also reduces the size of the elf headers, GOT, etc... as much as 20%
Note: if they will be used for building into libraries, ignore all but the --gc-sections part (very uncommon, - I tried it with GTK2, but its build system fought me too much)

[sunburnt]

Wow again! Thanks technosaurus; This really gives learned detail to it all.
I`ll have to reread it as I go through the steps myself.

I looked at the 2 outputs from ldd and ldd-recursive.pl
It looks like you`re right that ldd does give a binary`s deps. and the dep`s. deps.
I thought it just output the binary`s deps. only, leaving out a bunch of lib. deps.

ldd-recursive.pl outputs the tree with lots of duplicate libs.
A line count for the 2 outputs:

Code: Select all

sh-4.1# ldd /mnt/sdb3/ubuntu_apps/lbreakout2_2.6.1/usr/bin/lbreakout2 |wc -l
19
sh-4.1# ldd-recursive.pl /mnt/sdb3/ubuntu_apps/lbreakout2_2.6.1/usr/bin/lbreakout2 |wc -l
840

As I said, most of the 840 dep. lines are dups., but it outputs a complete dep. tree.

[technosaurus]

To answer a previous question, it is perfectly legitimate to mix static and shared libs. Check the Gparted thread that ttuuxxx and I had a couple years back, all of the gtkmm related libs are built-in statically (barry does this too, but his static libs aren't optimized for it). Usually this is done on direct dependencies only, but occasionally an indirect dependency sneaks in statically because the .so symlink is unintentionally missing from the devx.