Options for fixing paths in apps.?

For discussions about programming, programming questions/advice, and projects that don't really have anything to do with Puppy.
Message
Author
User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

Options for fixing paths in apps.?

#1 Post by sunburnt »

I`m experimenting with using squash files mounted only, no union.
I know ChoicePup did this, but I don`t see how this problem was solved.

Executable and library files that`re are found by their paths are easy to fix.
Why not have other paths to other types of files: config., data, etc.? (kernel)

Example: I have an app., xmahjoong, it won`t run because it can`t find the tiles.
Obviously the path to the tile files is coded absolutely.

If app. code had only relative paths to it`s own files this wouldn`t be a problem.
Then the app. could be "installed" anywhere, even in a mounted squash file.

Something to do with how the app. is compiled effects how paths are handled?
Would a static compile put the tile files inside the executable and fix this?

1) I can`t think of an easy fix for this except with lots of links. ( Sucks...)
2) Other than that, it`s possible that recompiling may fix it.
3) Or worse case scenario... rewrite the apps. code with relative paths.

Any ideas to solve this? Maybe something I haven`t thought of here?

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#2 Post by amigo »

Links or a slight mod to the sources are your best bet. Some apps code will let you use relative paths, but others may not easily do so.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#3 Post by sunburnt »

Somehow I knew it would be you to answer amigo...

1) So in the case of xmahjoong, the tile files and etc. can`t be compiled into it?
2) Does a static compile mean only libraries are joined?
3) Or can what`s included in the executable be controlled?

I`ve read much on static compiling, but nothing has explained this very well.

Links are such a mess and still have to be tracked just like the files themselves.
And modding source code is a long treacherous task over thousands of apps...

# You said: "Some apps code will let you use relative paths"
4) You mean: "Some apps. will work no matter where they`re installed." Right?

5) Which begs the Q... Why don`t all apps. use relative paths for all file access?

jpeps
Posts: 3179
Joined: Sat 31 May 2008, 19:00

#4 Post by jpeps »

sunburnt wrote:Somehow I knew it would be you to answer amigo...

1) So in the case of xmahjoong, the tile files and etc. can`t be compiled into it?
2) Does a static compile mean only libraries are joined?
3) Or can what`s included in the executable be controlled?

I`ve read much on static compiling, but nothing has explained this very well.

Links are such a mess and still have to be tracked just like the files themselves.
And modding source code is a long treacherous task over thousands of apps...

# You said: "Some apps code will let you use relative paths"
4) You mean: "Some apps. will work no matter where they`re installed." Right?

5) Which begs the Q... Why don`t all apps. use relative paths for all file access?
I think static build generally refers to having all the llbs included. The point is that it's not relying on them being present in the distro. For example, it was difficult getting Skype to work until they released a static build. With dynamic builds, links are often necessary anyway when it calls for specific versions of a lib...so it's standard procedure.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#5 Post by amigo »

Ach, you caught me red-faced there, so to speak! LOL

It is common to include pixmaps right in the binary -Xdialog is a good example. But, that is usually for XPM's. I've just had a quick look cause I've had xmahjongg in my collection for ages. It's using those yucky *.gif's and it's all written in that yucky C++. But, you can set the location to install them by setting pkgdatadir or datadir in configure options.Something like this should work:

./configure --prefix=/usr --datadir=/usr/share/xmahjongg

Or, cd into the unpackaged sources and do:
grep -nrH tiles * |less
That should give a clue to this:
grep -nrH pkgdatadir * |less
Which gives a clue to this:
grep -nrH PKGDATADIR * |less

There, you'll see that configure substitues the given or default value of pkgdatadir into the:
#define PKGDATADIR
(in the config.h.in) configure creates config.h from this. If you edit
the config.h.in and change line 73:
#undef PKGDATADIR
to read:
#define relative/path/here

If somehow that is getting clobbered when configure is run (no matter if/what setting pkgdatadir gets), then, before compiling, simply edit the finished config.h file to read like above, compile, strip, compress, check perms, check ownerships, check this, check that -oh wait, I have src2pkg to do all that for me...

(Edit: Actually grepping for PKGDATADIR should also show you that you could edit src/main.cc {line 526} to read like this:
const char *config_dir = "relative/or/absolute/path";
to make all those files go where you like.)

Oh man, I feel like a hero... I did not plan on being nice today, but sometimes nature has her way with me.
Have fun!

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#6 Post by sunburnt »

Thanks jpeps; I made a sfs file from a static build of Skype years ago.

WOW !!!!! ... I`m overwhelmed amigo... ( I copied your post to my tutors dir.)
Now I`m going back and rereading all the compiling tutors again!

I hear you about xmahjongg looking yucky, it`s just the test bed for my idea.
I take it you`re a standard C programmer? None of this fancy object crap?

# More damn Qs.......... Can this be put into a generic script for all apps.?
# Or make a script to run on one or more of these files to edit it as needed?
# Or are the config. files for compiling apps. so different from app. to app.?

# Need to automate this to make a big collection of apps. that`re relocatable.

Your last paragraph seems to indicate that this may be possible:
edit src/main.cc {line 526} to read like this:
const char *config_dir = "relative/or/absolute/path";
In xmahjongg`s case "relative/path" would be "../share/xmahjongg"?

I`m a beginner at this, I have no clue as to what`s going on ( Duh...).
Believe me... I have read, but concept wise it`s not helping.

Years ago I asked my friend a M$ Access and SQL dev. what a table was.
He chuckled and explained a table as records and fields... I knew instantly.

Thank you very much for indulging me amigo... Terry B.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#7 Post by amigo »

Remember that grep command and use it every time you want to know where something is located or getting called from.

# More damn Qs.......... Can this be put into a generic script for all apps.?
# Or make a script to run on one or more of these files to edit it as needed?
I have a simple and pretty good way of dealing that -I create things as rox AppDirs, but they can then be linked into a normal path if you like. It's an AppDir 'framework of sorts which is either a wrapper around mostl-elsewhere installed programs, using normal shell commands. But the same framework can be used to build AppDirs from source. It includes a couple of example diffs which can be adapted to different kinds of sources:
http://distro.ibiblio.org/pub/linux/dis ... er-0.4.tbz
The concept is easiest to implement by choosing a main location under which all apps can be managed. Then you just compile each one using the prefix /main/app-location/this-app -or if the user compiles it himself then it will 'locate itself' wherever it is compiled from.

# Or are the config. files for compiling apps. so different from app. to app.?
This is the big problem with build systems. There are a whole bunch of systems for configuring sources. Quite a few can be built with no special options other than prefix, using the standard './configure ;make; make install'. But there are jillions of sources which need something else or more. My src2pkg is probably the most independent among the automated build systems, and knows about lots of configure/build methods. It can build several types of packages, including *.pet.

# Need to automate this to make a big collection of apps. that`re relocatable.
You can only repeatable, orderly builds by scripting everything -src2pkg writes the scripts for you -at least the basic one. You then add any code lines needed for any extra items or actions. For changes to code, you need to have clean diffs -src2pkg applies them automatically or you can use a list.

# In xmahjongg`s case "relative/path" would be "../share/xmahjongg"
Yeah, that works assuming the binary is in an adjacent directory where one directoty up there is a dir named 'share'. That fits the dir structure under /usr and /usr/local. But, if you want the thing with all in one dir(AppDir or 'bundle' style), then you'd use:
"share/xmahjongg"

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#8 Post by sunburnt »

Wanted to try it before replying.
I`m posting a file with the compile and make output.

# The source file dir. is at: /usr/local/src/xmahjongg-3.7

I modified the file: main.cc as you said.
I assume there`s no normal files until you run install?

Need the finished files in an empty dir so I can make a squash file.
I don`t think your src2pkg utility will make squash file packages?

I still don`t see how that one line tells it how to include everything.
I hope I can get this to work on lots of the packages out there.
Attachments
comp-make.txt.gz
(2.46 KiB) Downloaded 232 times

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#9 Post by technosaurus »

I used to sometimes mod the sources to include the icons in /usr/share/pixmaps or /usr/share/mini-icons (depending on the type of icon) ... until an app was looking for a specific image that got overwritten and caused it to segfault. The symlink method is much safer, but I think it is gobolinux that does exactly what you speak of (I think by setting the prefix to something like /mnt/appname and the mount script adds :/mnt/appname/bin:/mnt/appname/sbin to $PATH and other such things)
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#10 Post by amigo »

mount script adds PATH -the trouble with that is that you wind up with a new PATH element for each program. get's very messy looking an dit adds time needed for startup because the sehll must search through each PATH element until it finds the app you are calling.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#11 Post by sunburnt »

Thanks technosaurus; Did configure option and make install placed the files correctly.
The app. works! The apps. files aren`t merged, but it works from here:
# /tmp/apps.sq/mnt/xmahjongg.tcz.sfs/bin/xmahjongg
With it`s other dirs.:
# /tmp/apps.sq/mnt/xmahjongg.tcz.sfs/share/xmahjongg

NO original dirs. or files exist where they normally are! ( So no cheating...)

If it all was compiled into one exec. file, it would be nice, no sq. file needed !
Imagine all apps. this way! But it`s not critical, making sq. files is easy to do.

Thanks again amigo; Some of your mods to the files failed to make.
A few did make install, but the tiles etc. are still separate. Most failed to run.
#This ran, but did not merge anything:
./configure --prefix=/usr/local --datadir=/usr/local/share

# The configure option: "--datadir=" sets what`s merged into the exec. file?
..... It seems like what`s needed`s an option to include everything in /src/(app.)
..... Then just copy to /src/(app.) all images, icons, libraries, etc. and they`re in...

### I do now understand how to set the placement of the install files.

### And most importantly, it seems to fix the app`s. path-to-its-files problem.
......... More testing on other source packages will tell how well this works.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#12 Post by technosaurus »

magic ermine can do what you are wanting (all in 1 binary)- I have tried it before and have permission to distribute any open source apps.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#13 Post by sunburnt »

technosaurus; Went to the web site and there`s only a 15 day trial.
It does sound exactly like what I spoke of, but I`ve had good luck so far.
I`ve gotten several apps working with you and amigo`s help, many thanks!

A lot of apps. have dependencies that Puppy doesn`t have, so they grow big.
It`s time consuming searching, downloading, compile, and find they need more...

Same for the Panel I`m looking for, downloaded lots, but most are task bars +.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#14 Post by technosaurus »

i'm getting a bit off topic here, but related.

If you are build something with a new library dependency, add -ffunction-sections and -fdata-sections to the lib when you compile it statically & add the same CFLAGS to your program too along with -Wl,--gc-sections to your LDFLAGS.
If you want to get a picture of how much space it saves you you can use -Wl,--print-gc-sections
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#15 Post by sunburnt »

Now I`m going to have to read more on this. I know nothing of which you speak.
One by one here...

# add -ffunction-sections and -fdata-sections to the lib when you compile it statically
Add these to one or more of the files in the extracted source package?

# add the same CFLAGS to your program too along with -Wl,--gc-sections to your LDFLAGS
Same here I assume? CFLAGS and LDFLAGS are variables in source config. files?

# -Wl,--print-gc-sections
And this last one shows the space savings for having used the previous instructions?

If the difference in the savings is closer to the non-static build it`d be great.

# All this I assume is for static builds? I need more clarification...
A static build seems to be mainly libs. included? Or other data files as well?

I think keeping most of the libs. separate is good for saving space.
Unique ones to the app. should go in it. This demands a statistical analysis.
What libs. are most common? This Q is a monster as there`s thousands of libs.

An added dir. to LD_LIBRARY_PATH will make it possible to add new libs.
So not a problem. Puppy already has one of course: /root/my-applications/lib
I usually add libs. there if I`m doing things manually. Easy to find and delete them.

Is there a quick way to know what libs. a distro. has? ( cat ld.so.conf | find )
"ldd" shows what`s needed. The two could be compared for a missing list.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#16 Post by technosaurus »

You can usually enable c/ldflags by
CFLAGS="-Os ..." LDFLAGS="-Wl,-Os,-s,..." ./configure ...
Other times configure will ignore your request and you have to edit the makefile(s)

If devx is installed
echo /lib/*.so /usr/lib/*.so /usr/X11/lib/*.so ¦tr " " "\n"
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#17 Post by sunburnt »

technosaurus; I appreciate the help, but I`m still puzzled... "enable c/ldflags"
What you`re saying is: In a term. type:

Code: Select all

CFLAGS="-Os..."
LDFLAGS="-Wl,-Os,-s,..."
./configure ... 
Where "..." is more options being assigned to the variable?
I`ve read on compiling but nothing has said anything about what you say.
Where do I read about this stuff, the variables, options, and what they do?
I assume with the right options an all-in-one exec. is possible?

You say you`ve worked with magic ermine? You have a non-trial copy?
I`m thinking it would solve all of this hassle with all apps.... Am I correct?
How much do they want for it? I assume to buy it means use for anything.

# Why is devx needed to list all of the lib. files?
I noticed a while back that not all libs. end in ".so", what about them?
Like: " libblkid.so.1.1.0 " and it`s link: " libblkid.so.1 ".

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#18 Post by technosaurus »

>>technosaurus; I appreciate the help, but I`m still puzzled... "enable c/ldflags"
What you`re saying is: In a term. type:

Code: Select all

CFLAGS="-Os..." \
LDFLAGS="-Wl,-Os,-s,..." \
./configure ... 
Where "..." is more options being assigned to the variable?
I`ve read on compiling but nothing has said anything about what you say.
Where do I read about this stuff, the variables, options, and what they do?
I assume with the right options an all-in-one exec. is possible?

<<fixed above ( no return between flags)
I read the gcc/ld options from gcc.gnu.org and manpages

>>You say you`ve worked with magic ermine? You have a non-trial copy?
I`m thinking it would solve all of this hassle with all apps.... Am I correct?
How much do they want for it? I assume to buy it means use for anything.

<<I asked the developer questions about adding resource files (ex. /usr/share/package/* and how to exclude libs, etc... and she sent me a license (she also does a tool called statifier that is similar and gpl)

>># Why is devx needed to list all of the lib. files?
I noticed a while back that not all libs. end in ".so", what about them?
Like: " libblkid.so.1.1.0 " and it`s link: " libblkid.so.1 ".

<< it isn't, but using *.so.* will have a lot of dupes


Edit...
I posted a sfs tool that may do do what you want.

http://murga-linux.com/puppy/viewtopic.php?t=70804
It is not alpha yet, but works
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#19 Post by sunburnt »

Okay... What a silly thought, that it`d be just the way you wrote it...
I`m going to look at statifier and hopefully find out what it does.

I looked at the page you posted for an sfs tool, but the thread is for jwm menus.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#20 Post by technosaurus »

Normally, when you link with say -lfoo the linker will look through $LS_LIBRARY_PATH (a colon separated list of library locations) for libfoo.so and then try libfoo.a and move on to the next dir (footnote, this causes many pets compiled in 4.3.x? ...honestly I forget which puppy... to get libz.a to be compiled in statically because there is a libz.a in /lib but the libs.so is in /usr/lib, which comes after /lib)
Adding -static or -Bstatic reverses the preference, I think one goes through the entire $LS_LIBRARY_PATH looking for libfoo.a before even trying libfoo.so (not 100% sure ... I have to look them up when I use them)

In contrast --enable-static just tells autotools to build a static library (whichis really just an archive of the object files). By default, many programs will still build and link against its own internal shared libraries, unless you also use --disable-shared. The same goes for library packages, but the quick & dirty way to force any library to be compiled in statically, is to simply delete the libfoo.so symlink (provided you have libfoo.a, that is).

My last post in jwm_tools is a script I call frugalunion, it mounts any filesystem supported by busybox mount (includes squashfs and many others) then links or copies files as appropriate and runs the program(s) that have .desktop files. Running it again will remove the links/files and unmounts. (I use busybox mount because it will handle loops automatically, while the full mount doesn't.)
I still want to add code to copy vs link everything in $HOME and remove the dependency on rox.
It is designed to work basically like magic ermine. My only note of caution would be to limit the .desktop files (you could make a quick GUI in yad, xdialog or gtkdialog to select one ... I am thinking libre office)
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

Post Reply