How do you get rid of orphaned libraries?

[learnhow2code]

But see, even having the symbols that binary calls for doesn't mean they are the same libs which get linked -even if the version is the same, even if the toolchain is the same, even if the *explicit config options are the same.

The thing is that many sources use auto-discovery to find out if the build system has the necessary build-time and run-time objects needed to build the source. What I'm getting at, is that one library can contain symbols and functionality -because some optional external libs were present on the build system. ffmpeg is probably an extreme example.

If you try to use a binary which thinks the lib contains that optional symbol and the library you have does not contain those symbols, you have a case where an *apparently-resolved* dependency will not work.

i get what youre saying, but not where youre going with it. so answer me this: are you trying to say: "but even if you stop using ldd, and start using something more accurate and safe, you still wont get what youre trying to get, because you dont know that the binaries are NOT satisfied by some unnamed/unrequest thing that was linked as a stand-in?" because thats what i think you could be saying.

the implication of that would be "an unverifiable promise from a package system is something that could be worth more than concrete requirements that are ignored by the maintainer of a reasonably intact package-- even though in this case (because puppy packages have no such goal) it ISNT worth more, which means there is no real solution except one we have little control over.

[anikin]

... That is great for Debian-ists, but no good for us Puppy-ists...
Back to square one I suppose.

Puppy-ist is a person who's striving to become a Debian-ist. That's how it's defined in my book. You are not quite there yet, mon frère Seriously, though, I have a question: what is <puduan> - what's in that name? I must have missed some earlier discussion.

[learnhow2code]

what is <puduan> - what's in that name?

puppy+devuan. though the "d" is interesting:

puppy devuan

i wouldve gone with pupuan, or "pup-one," though i suppose if people didnt like it youd be opening it up to some colorful phrases.

[anikin]

Thanks for the explanation, learnhow2code. I suspected there had to be a connection with devuan ... reminds me of Papua New Guinea, though https://www.youtube.com/watch?v=wfWMv8Y1V5E

[amigo]

My Four Points should have been five. Dependency-resolution requires a proper repository which fulfills the strict needs of the package manager.

There simply is no way to accurately predict whether any 'foreign-made' binary object(lib or executable) will be 100% compatible with another foreign or locally-produced object.

So, the software *delivery* system also has to be a part of the Big Picture. Each of the Five Points could be elaborated -file-naming and versioning conventions, directory layout, structure and nature of the database and of course what data is needed/wanted --all these have to be taken into consideration. There are also actions to be taken when installing/upgrading/uninstalling packages -usually some sort of script, or the package database contains a list of options which cause the package manager to carry out tasks other than installing files -like creating users,groups or whatever.

For years I tried to coax src2pkg into creating an enhanced database for use with slackware packages. However, Slackware contains a couple of packages which absolutely kill any chance of sane and complete dependency info. Finally, I created my own package format (tpkg) and the tools for handling the packages. It was a snap to get src2pkg making packages with *lots* of info and overcome the drawbacks of Slackware's packaging system.

After 2-3 years I started designing a new package format which is more capable and faster.

Using the tpkg tools, answering the OP's question would be pretty easy, but this aspect of package management requires a Blacklist/Whitelist approach -there are packages which do not depend on any other package and yet are indispensable. This one of the things where the new (tpm) package format is improved.

[musher0]

Thanks for the explanation, learnhow2code. I suspected there had to be a connection with devuan ... reminds me of Papua New Guinea, though https://www.youtube.com/watch?v=wfWMv8Y1V5E

No, no, no, it should be reminding you of the beautiful city of Padua, in Italy.

[learnhow2code]

one of the ways to reduce orphan libraries is to not have them included in the first place. this requires a change of tack.

i was really thinking something more practical, but even if you renamed "orphan libraries" to "bastard libraries" people might get rid of them faster.

"try my distro!"
"how many bastard libraries does it have?"
"um, i dont know!"
"..."

mine is a complete bastard os, so this doesnt always work.

[s243a]

If a different library is linked then couldn't one look for symbolic links? Also someone has mentioned the need for a consistent directory strucure but a binary looks in typical directories for suitable lbraries. One dosn't need to search the entire file system.

[Sailor Enceladus]

I know that orphan program musher0 linked to from linuxquestions showed a lot of false positives. It told me /usr/bin/7za is an orphan, for instance, and I USE it in XArchive to extract 7z files! lol. I think woof-ce had something similar, I tried the "remove unused dependencies" option in it, and it just blew everything up. If the woof-ce version doesn't work, I'd say that's a pretty safe bet that only manual removal of things by being a complete linux genius and knowing exactly what uses what is the answer. Possibly only 01micko is on this level, or close to it, with his constant refinement of slacko since 5.3, but things are always changing too, so it's a constant battle of understanding everything there is to know about each of the 11000-17000 files that go into the puppy.sfs file, and how/if they relate to each other, or to anything; I think. Might have to google every file.

[musher0]

And then every one in this thread paused, sat down in the lotus position, took deep
breaths and spent the day meditating on Sailor's wise paragraph above.

(My sentence may be fanciful, but it is not a mockery at all.)

[learnhow2code]

well, i tried my hand at woof-ce (legacy) today.

i ran a lot of scripts.
i ran a gui.
i downloaded stuff.
i selected options.
i made folders.
i watched things run.
it didnt work.
so i had a look at the script that probably does all the real work: the builddistro-Z script.

alrighty!

so when i work on puppy things, i think i wont be using woof-ce. perhaps rationalize is so much cleaner, but i like bash-- i dont like it THAT MUCH. i will try to find another way.

at least i tried. i think i can say that approach is NOT what i want. and what DOES this have to do with this thread, anyway? simple: whatever version of mushers issue im going to "solve" or not, its probably going to be solved in a way that is completely different/incompatible anyway. sorry, musher-- it cant be done. (now thats said, someone will prove it can-- thats great!)

[Sailor Enceladus]

(now thats said, someone will prove it can-- thats great!)

lol but you jinxed it by adding this part, now we are doomed forever (If this really is bad at all)

[s243a]

I know that orphan program musher0 linked to from linuxquestions showed a lot of false positives. It told me /usr/bin/7za is an orphan, for instance, and I USE it in XArchive to extract 7z files! lol. I think woof-ce had something similar, I tried the "remove unused dependencies" option in it, and it
just blew everything up. If the woof-ce version doesn't work, I'd say that's a pretty safe bet that only manual removal of things by being a complete linux genius and knowing exactly what uses what is the answer. Possibly only 01micko is on this level, or close to it, with his constant refinement of slacko since 5.3, but things are always changing too, so it's a constant battle of understanding everything there is to know about each of the 11000-17000 files that go into the puppy.sfs file, and how/if they relate to each other, or to anything; I think. Might have to google every file.

Amigo mentioned a white/black list. Perhaps we could build one by running strace on some programs we like. Maybe start with some of the programs in the puppylinux menu.

[learnhow2code]

Amigo mentioned a white/black list. Perhaps we could build one by running strace on some programs we like.

that was quicker than i thought it would be.

[TeX Dog]

A billion lines of code ago (1BC) we used access bit in the filesystem and did not preload libs. After a complete run of the test cases only those files that where 'accessed' where bundled and shipped. Only caused problems for a few clients which our test cases where weak.
My understanding all that low level file access control are currently not compiled in since it doesn't play nice with layer file systems, like puppy which still can't be 'truly' installed.
It was also used for the ordering of laying out files in the filesystem, when seek time was poor and that mattered. Knoppix was the last live distro to use such a idea, that is why a DVD sized anti-puppy colossus loads so quickly. Wish he will share HIS build system.

just looking around for more modern ideas
using ps grab the process ids, the call command pmap to show loaded libs pipe that those to file, sort removing dups to get all used up to that point (guess starting all apps you normally use)

[technosaurus]

...
It was also used for the ordering of laying out files in the filesystem, when seek time was poor and that mattered. Knoppix was the last live distro to use such a idea, that is why a DVD sized anti-puppy colossus loads so quickly. Wish he will share HIS build system.

log all file accesses with lsof during startup, then add them to the squashfs in that order

[technosaurus]

If you want to find orphaned library dependencies for all binaries in your path (excluding those needed by dynamically loaded modules/plugins that are normally outside of PATH) you can use this little script that require objdump (it only lists direct dependencies, not everything that happened to be linked whether it was needed or not):

Code: Select all

for x in ${PATH//://* }/* ; do [ -x "$x" ] && objdump -x $x 2>/dev/null & done|grep NEEDED|sort |uniq -c |sort -n >binaudit.txt

for each of the libraries in the list you'd want to get their dependencies also - repeat until no new dependencies are found (may take a while)

Then just iterate over all of the libraries in the library path and {re}move them if you can't grep them out of the audit file

I wrote that script a long time ago, these days I would use IFS=: instead of the ${PATH//://* } bashism ... which now that I look at it should have 2 for loops - 1 for each path in PATH and another for each binary in path/*
... I don't have a way to test it right now, so I hope this is enough for someone else to figure it out - if you try and get stuck, just post what you have and I'll help debug

[rufwoof]

You're likely the man to know technosaurus. Debian have utilities to remove unused libs (apt-get autoremove), and a script to run through and validate each file (crc type check so can pin down a possible virus for instance). But sadly I've not found a script/method to validate file permissions, to reset all system files back to correct permissions in the event of doing something silly like a recursive permissions change action (yep ... that's what I did). I ended up redoing from scratch ... newly install and all of the changes/updates ....etc.

Perhaps a new install should have a record made of all the system file permissions at the time kept back as a reference so at least a recursive change could be created to reset at least those files. Combined with adding to that record set when files were added/replaced.

???

[amigo]

That info should be contained in the packages.

[rufwoof]

...
It was also used for the ordering of laying out files in the filesystem, when seek time was poor and that mattered. Knoppix was the last live distro to use such a idea, that is why a DVD sized anti-puppy colossus loads so quickly. Wish he will share HIS build system.

log all file accesses with lsof during startup, then add them to the squashfs in that order

Just discovered strace

strace -f -e trace=open -o process.trace <command to start process>

that will list what was opened by a run of a command. So if you know what files/programs are being run and in what order you could deduce the associated files. But technosaurus' lsof is probably better/easier.