How do you get rid of orphaned libraries?

For discussions about programming, programming questions/advice, and projects that don't really have anything to do with Puppy.
Message
Author
musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

How do you get rid of orphaned libraries?

#1 Post by musher0 »

Hello all.

As the title says. It's for the Puduan alpha Pup.

I tried "deborphan", but it's no good for Puppies since we don't store our packages
at the same place that the Debian people do. Same thing with "rpmorphan" for
RHLE-derived distros (CentOS, Fedora, etc.).

I remember Iguleder saying that he used one such utility for his LibrePup, but I don't
think that he ever mentioned the source or the name of the utility. OTOH, it's entirely
possible that it was his own script too, given his talent. It trimmed about 10 Mb's off
his LibrePup.

Whatever the case may be, I'll be grateful for any leads. BFN.
Last edited by musher0 on Wed 24 Aug 2016, 06:24, edited 1 time in total.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
ally
Posts: 1957
Joined: Sat 19 May 2012, 19:29
Location: lincoln, uk
Contact:

#2 Post by ally »

diffutils?

:)

learnhow2code

#3 Post by learnhow2code »

find all the binaries (except libraries) and run ldd on all of them, then subtract THOSE listed from a list of all libraries. the list remaining is the list of libraries you (probably) dont need. to that i add-- even deborphan makes mistakes.

id try symlinking puppy libs to the place deborphan expects them-- or even copying them if necessary. that wont help if it needs /var/lib/dpkg/info though.

Sailor Enceladus
Posts: 1543
Joined: Mon 22 Feb 2016, 19:43

#4 Post by Sailor Enceladus »

I found an interesting post by 01micko the other day:
01micko wrote:If you are talking about slacko64 then these missing libs are intentional (well perhaps not all of them).

Some libs are hard dependencies and some are not. Slackware is a bit different :)
http://www.murga-linux.com/puppy/viewtopic.php?p=866820#866820

So some libraries and binaries are just there, in case you install something later (ie. qt4, gtk3). I guess I should stop deleting random things because "checkdeps /" says they're missing something then, because they might be useful later. Hmmm. Heh.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#5 Post by musher0 »

Sailor Enceladus wrote:I found an interesting post by 01micko the other day:
01micko wrote:If you are talking about slacko64 then these missing libs are intentional (well perhaps not all of them).

Some libs are hard dependencies and some are not. Slackware is a bit different :)
http://www.murga-linux.com/puppy/viewtopic.php?p=866820#866820

So some libraries and binaries are just there, in case you install something later (ie. qt4, gtk3). I guess I should stop deleting random things because "checkdeps /" says they're missing something then, because they might be useful later. Hmmm. Heh.
Sounds like the twilight zone.
Tah dee tah dum, tah dee tah dum. (Remember the Twilight Zone song?)
https://www.youtube.com/watch?v=XVSRm80WzZk

Prepare your Pup for the return of..., or the coming of... Hm.
The Mamas and the Papas? :lol:
Cleopatra?
Lucy? (You know, the millions-years old skeleton!?) :lol:
The Borgs? :(

But thanks for your reseach / ideas, Sailor.

@ally:
I know that diff is of no use for this particular purpose.
But I don't remember what else there is in the diffutils package.

@learnhow2code.
Yeah, I thought of doing that.
But I was hoping there would be a ready-made solution for a dev in lazy mode!

Haha! I know!
Run diff on the entire list of libs compared to the whatchamacallit learnhow2code list! :D

If you were girls I'd kiss you! :lol:

BFN.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#6 Post by amigo »

There can be no tool for that, because the necessary (detailed) data does not exist under any puppy.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#7 Post by musher0 »

amigo wrote:There can be no tool for that, because the necessary (detailed) data does not exist under any puppy.
Oops. Thanks amigo.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#8 Post by s243a »

amigo wrote:There can be no tool for that, because the necessary (detailed) data does not exist under any puppy.
Why wouldn't learhow2code's suggestion work? Which incidently is also what I would have suggested.

Aside from that though we could also interface with some dependency database that could be external to the os.

learnhow2code

#9 Post by learnhow2code »

s243a wrote:
amigo wrote:There can be no tool for that, because the necessary (detailed) data does not exist under any puppy.
Why wouldn't learhow2code's suggestion work? Which incidently is also what I would have suggested.
i am fairly confident ldd would do it all, its just a matter of actually running ldd on everything.

if theres a reason this wouldnt suffice, im curious what it is. there are plenty of people here that know more about compiling and deps than i do.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#10 Post by musher0 »

Hello all.

I'm still in lazy mode, looking for ready-made solutions.

What is the attached script worth in the context of Puppy?
From: http://www.linuxquestions.org/questions ... es-794133/
Retrieved: 10 minutes ago.

Is this any good for us?
Could it be adapted? Improved?

Thanks in advance.

~~~~~~~~~~
Found through DucDuckGo using the search phrase:
linux orphaned libraries NOT deborphan
Attachments
orphans.sh.zip
(5.44 KiB) Downloaded 162 times
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#11 Post by musher0 »

Hi gang.

Here is an implementation of the idea.

There's something I'm not doing right and I can't find it. The result is unreal!

Please help, I'm stumped. TIA.
Attachments
unreal-result.lst.zip
(5 KiB) Downloaded 138 times
ldawkgresort.sh.zip
(761 Bytes) Downloaded 134 times
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

Sailor Enceladus
Posts: 1543
Joined: Mon 22 Feb 2016, 19:43

#12 Post by Sailor Enceladus »

Wow musher0, even with "basic" that orphan program took quite a while, and says I have 3521 babies with no parents. :shock:

Haha, your twilight video reminded me, I found this really trippy 10 minute song with a Kaleidoscope moving in the background on youtube, but I don't remember what it's called any more.... now I must listen to every trippy Kaleidoscope video on youtube until I find this particular song again! :lol: /random (edit: hahaha I think I found it, or maybe it was a different one in there)

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#13 Post by s243a »

I suggest that we sort the orphand libraries into packages so that we know what package they are from. This could help decide if we want to remove it.

So I suggest that once we find an orphin we compare it with the list of files fore each package in the /root/.package/ directory

The only problem I see is that I don't think that built-in packages have a complete list of files in /root/.package/ but I could be wrong.

Another thing is that any file that is in /opt/ should be quite segregated into packages and sperate from most of the system. Files under /opt/ should be easy to identify which package that they belong to.

edit: Related to the above. I think I would do it one package at a time. Figure out which package is least critical for you by what it is supposed to do and then try to build a trimmed down version of it based on the orphaned libraries. Test your system and if it work then move on to trimming the next package if you desire. It might be helpful to know which packages in the package manager (packages not installed) may depend on these libraries in case you plant to install them in the future.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#14 Post by musher0 »

You seem to have a great logical mind, s243a :) except... have a look at my "unreal"
list above!!! There are so many orphans in there! And I'm not sure they really are
orphans. There are too many, it's unreal.

Before we put your plan into action, it looks like we'll have to find some real orphans! ;)

BFN.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#15 Post by s243a »

musher0 wrote:You seem to have a great logical mind, s243a :) except... have a look at my "unreal"
list above!!! There are so many orphans in there! And I'm not sure they really are
orphans. There are too many, it's unreal.

Before we put your plan into action, it looks like we'll have to find some real orphans! ;)

BFN.
Perhaps we could try data-mining the puppy package manager since, it lists the dependencies before you install software from it.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#16 Post by musher0 »

Yes that could work... to a point.

However, after doing some research, Sailor Enceladus said:
> "So some libraries and binaries are just there, in case you install something later"
If that statement is verified as true (which I suspect from my own little experience
above), it could explain the high, unreal, number of "orphaned" libs waiting to be
adopted by a program.

If that is true, would there be a corresponding program in our "packages list"?

As for me, -- if that approach is proven right -- I don't really like it. If you want a QT
program I think that you should load the QT library sfs and then install the QT
program you want. The Qt library should not be in the Puppy already if there are no
QT programs offered in the out-of-the-box Puppy. IMO, it's simpler that way.

Maybe that's the reason why Puppies are getting bigger and bigger.

Anyway, I think we have lots of testing to do before we come to any conclusion.

BFN.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

learnhow2code

#17 Post by learnhow2code »

musher0 wrote:Sailor Enceladus said:
> "So some libraries and binaries are just there, in case you install something later"
If that statement is verified as true (which I suspect from my own little experience
above), it could explain the high, unreal, number of "orphaned" libs waiting to be
adopted by a program.
he has a point.
I don't really like it. If you want a QT
program I think that you should load the QT library sfs and then install the QT
program you want. The Qt library should not be in the Puppy already if there are no
QT programs offered in the out-of-the-box Puppy. IMO, it's simpler that way.
its not simpler-- its more streamlined. as to "which is better" i really think it depends on the amount of in-use infrastructure vs. the amount of "just in case" infrastructure.

you put roads in assuming cars will come. if you insist that cars bring their own road when they come, this is more efficient from a city-planners point of view but for a car manufacturer its a nightmare... actually this metaphor is all wrong. the point is, if qt is going to be used by many things and it adds little weight to puppy, theres a case to be made for it.

if it adds a lot of weight or isnt used by many things (or if this just-in-case philosophy is taken too far) then the simplicity of "its already there" is outweighed by the complication of "how can we get rid of it?"

qt is just one example, so it can go either way. tinycore is what you get when you keep pushing for "bare bones for everyone." theres something about "a package for a package for everything" and debian has very little in the way of orphans-- but what makes it different from tinycore is that when you install it, you have a furnished apartment. you dont have to run out to ikea and buy everything at once.

and what puppy (for most people-- i actually think bare bones is a great idea for some) does is actually find a middle ground between tiny core and debian-- it has a lot of small furnishings to explore, and isnt bare at all. its minimalist, as opposed to stripped. that could change, but for > 10 years it has stayed minimalist, while bare bones remains an option (instead of a standard.) perhaps thats a hint as to what appeals to more people. or not.

i used tiny core for quite some time, talking to robert when it was new. eventually i got tired of watching it load packages on boot, and it went from novel to tedious. its still a great idea-- its the smart fortwo of distros.

anyone going minimalist should consult the law of diminishing returns-- at a certain threshold, minimalism goes from actual simplicity to an overbearing full-time job. having tried several approaches-- mine is now to get a reliable distro, strip the unreliable and offensive parts, add whats needed, and (really above all) use a window manager that doesnt waste resources.

in puppy its jwm+rox, for me its icewm and pcmanfm. thats a simple optimization with a giant return-- few things you can do for a distro will streamline you more than that. squashfs and running in ram are quite the tools also. after the window manager, the browser is our biggest problem.

anikin
Posts: 994
Joined: Thu 10 May 2012, 06:16

#18 Post by anikin »

musher0 wrote:Hello all.

I'm still in lazy mode, looking for ready-made solutions.

What is the attached script worth in the context of Puppy?
From: http://www.linuxquestions.org/questions ... es-794133/
Retrieved: 10 minutes ago.

Is this any good for us?
Could it be adapted? Improved?

Thanks in advance.

~~~~~~~~~~
Found through DucDuckGo using the search phrase:
linux orphaned libraries NOT deborphan
Ran a couple tests on Debian Live and DebianDog Wheezy. To jump to a quick conclusion, this script is no good for me. Maybe it's good for Slackware, I don't know, but not on Debian. It treats almost every lib and binary it sees as orphaned. It shows no precision, whatsoever. This is in a stark contrast to debfoster & deborphan, the combo I normally use for remastering. Both work beautifully, not that I totally rely on their precision, no, but at least, I know what to expect of them and it goes without saying, they are very well documented and easy to use for us, little Linux neophytes.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#19 Post by amigo »

Dependencies can only be accurately documented at build-time. There are also some caveats for special cases, like where a program depends on another non-library binary (like man depends on groff) or uses dynamically loaded libs/plugins which constitute a 'soft' dependency.

The package manager for puppy is not able to provide this information accurately because the information is not made available during build-time, by including the info *inside* the package. External databases do not work because you can never be sure that the version of the database is the same one as was used to produce the package. Of course, this is the smae problem with using packages built elsewhere.

Resolving a dependency is not about finding the same version of a needed library -it must be have been compiled with the same options and toolchain as the original library one is trying to match. In short, unless the needed lib and the wanted binary are produced together in a unified way, you can never be sure that the library will really satisfy the dependency that the program calls for.

The situation with Slackware is only slightly better than with puppy in that the packages are sort of produced in a unified way, but dependencies must be abstracted using ldd & Co. But Slackware does not rebuild packages when the toolchain and/or dependent libs change, which means that you can't even be sure that a certain Slackware package is really compatible with the libs it *appears* to depend on.

Another thing is that 'ldd' is not really accurate.

You cannot rely on any external data about dependencies from other distros or builds.

learnhow2code

#20 Post by learnhow2code »

amigo wrote:Another thing is that 'ldd' is not really accurate.
i was curious if something like this would come up. before i use ldd for much of anything, please let me know what ldds limitations are-- youve dropped a hint here, im only asking you expand on it if possible. certainly not a call for proof or exposition (though id read that with interest, too) :) links welcome in lieu of more. i can search-- id like your input if possible.

Post Reply