What does the humongous /usr/lib/libLLVM.so.3 do?

[mikeb]

Hey, ho, don't shoot me !! I'm only the one who spotted the fatso!!

no intention to

Good to have a rant now and then .... feel free to have one on any threads I happen to bring up pet hates

Anyway we do seem to share the same Bloat Spotters...perhaps we are related ?

mike

[musher0]

libsambaclient seemed another fat dinosaur (samba no longer uses it I believe)... but its linked to mplayer (or was) a recompile got rid of that one saving 5MB+

mike

Hi, mikeb.

Back to this one. Re-reading it, it's not so clear anymore... Which one needs to be
recompiled: libsambaclient or mplayer?

I ask because, since I relayed your info on the French side of the forum, some members
have been quite interested in the idea of getting Puppy to shed 5 Mgs!

A couple of derivative questions, if I may:
you know this because you re-compiled the bugger, right? Which could mean that
lazybones such as myself could benefit from your re-compilation?

Thanks in advance for filling us in.

BFN.

musher0

[mikeb]

Funny world...I am actually compiling it at the moment.... doing a build with the skins gmplayer as I slightly hate gnome-mplayer and its plugins.

Yes i gave my build to rerwin for his consideration for the Lucid updates but don't think he used it.

Both it and the shared ffmpeg are not lucid versions but later so it took a bit of finding to get the sources originally used. I also left out such as libdirectfb . The idea was to make a drop in but slimmer replacement. I also did gnome-mplayer though thats a funny one and had to use a slightly later version as the original sources been removed....that one might simply benefit from a newer version if you can get it to build...i disable crap like gnome dependancies.

I might still have the results up on dropbox so will post links back here when I take a look...watch this space.

mike

[musher0]

Thanks, mikeb! TWUL.

[mikeb]

Info in PM

what does TWUL mean?????

mike

[musher0]

Talk With U Later

[mikeb]

PM = private message...juicy files in there.

I got the skins build done ok.... but had to use mplayers swscale rather than the shared one ??!!...

Note lucids mplayer can pretty much work on 4.12 if glibc is slightly updated to 2.7 ....

I digress sort of

mike

[peebee]

I have been building slacko-current based on slackware-current - all is working pretty good except I have the unsatisfied dependencies shown below.

The slackware-current llvm package is a humongous 54MB so I really don't want to have to include it if at all possible in a 250MB iso.....it's absence doesn't seem to be causing many problems except 01micko reported:

GLX isn't working.. Aha! here's why..

Code: Select all

[   162.762] (EE) AIGLX error: dlopen of /usr/lib/xorg/modules/dri/nouveau_dri.so failed (libLLVM.so.3.7: cannot open shared object file: No such file or directory)

Anybody got any suggestions? - like maybe making a stub libLLVM which satisfies the dependencies in the smallest way possible?

Cheers
peebee

/usr/lib/d3d/d3dadapter9.so.1.0.0:
libLLVM.so.3.7 => not found
/usr/lib/dri/gallium_drv_video.so:
libLLVM.so.3.7 => not found
/usr/lib/libXvMCnouveau.so.1.0.0:
libLLVM.so.3.7 => not found
/usr/lib/libXvMCr600.so.1.0.0:
libLLVM.so.3.7 => not found
/usr/lib/libxatracker.so.2.2.0:
libLLVM.so.3.7 => not found
/usr/lib/vdpau/libvdpau_nouveau.so.1.0.0:
libLLVM.so.3.7 => not found
/usr/lib/vdpau/libvdpau_r300.so.1.0.0:
libLLVM.so.3.7 => not found
/usr/lib/vdpau/libvdpau_r600.so.1.0.0:
libLLVM.so.3.7 => not found
/usr/lib/vdpau/libvdpau_radeonsi.so.1.0.0:
libLLVM.so.3.7 => not found
/usr/lib/xorg/modules/dri/kms_swrast_dri.so:
libLLVM.so.3.7 => not found
/usr/lib/xorg/modules/dri/nouveau_dri.so:
libLLVM.so.3.7 => not found
/usr/lib/xorg/modules/dri/r300_dri.so:
libLLVM.so.3.7 => not found
/usr/lib/xorg/modules/dri/r600_dri.so:
libLLVM.so.3.7 => not found
/usr/lib/xorg/modules/dri/radeonsi_dri.so:
libLLVM.so.3.7 => not found
/usr/lib/xorg/modules/dri/swrast_dri.so:
libLLVM.so.3.7 => not found
/usr/lib/xorg/modules/dri/vmwgfx_dri.so:
libLLVM.so.3.7 => not found
/usr/lib/xorg/modules/drivers/vmware_drv.so:
libLLVM.so.3.7 => not found
libLLVM.so.3.7 => not found

[01micko]

If you check out slackware-current changelog, on November 24 LLVM was rebuilt with clang compiler - which makes sense.

Code: Select all

+--------------------------+
Tue Nov 24 03:31:43 UTC 2015
a/dbus-1.10.4-i586-1.txz: Upgraded.
a/kmod-22-i586-1.txz: Upgraded.
a/lilo-24.2-i586-1.txz: Upgraded.
a/sysvinit-scripts-2.0-noarch-23.txz: Rebuilt.
       rc.6: Don't clear /var/lock/subsys.
       rc.S: Clear /var/lock/subsys here instead, so that the directory will be
       cleared out on startup after a power failure.
       rc.sysvinit: Run kill scripts for the current, not previous, runlevel.
       Thanks to Sl4ck3ver.
a/upower-0.9.23-i586-2.txz: Rebuilt.
ap/cups-filters-1.0.76-i586-2.txz: Rebuilt.
ap/lm_sensors-3.4.0-i586-1.txz: Upgraded.
       Thanks to Robby Workman.
d/intltool-0.51.0-i586-2.txz: Rebuilt.
       Fix warnings with perl-5.22.0. Thanks to Stuart Winter.
d/llvm-3.7.0-i586-2.txz: Rebuilt.
       Build using cmake and clang. This results in a smaller package size, fixes
       compiler-rt, and changes the shared library name from libLLVM-3.7.so to
       libLLVM.so.3.7.0 (which requires recompiling any binaries linked to libLLVM).
       Thanks to Heinz Wiesinger.

More info on LLVM

The Xorg drivers don't seem to have been built with clang (well they could be but I don't see it in the changelog) however from the same day MESA is rebuilt. Maybe Patrick will rebuild Xorg and drivers linked against LLVM (with clang compiler) before release. Hard to say for sure but it would make sense and make those packages smaller.

Code: Select all

x/mesa-11.0.6-i586-1.txz: Upgraded.
       Patched to find the new LLVM library.
       Thanks to Heinz Wiesinger.

This explains why GLX fails.

If you crack open the LLVM package you see that most of the bloat is executables in /usr/bin. To my way of thinking, these will be unnecessary for our purposes so can be discarded in a package template in woof (I'll attach one). The actual libLLVM.so.3.7.0 is ~33M. All the lib*.a and lib*.la will go to devx removing more bloat. I'm guessing we can get compressed size on disk to around 10 to 15MB (a real guess!).

Perhaps an optional 'clang' template could be made putting clang and friends on the devx. I'll think about that when I migrate to -current.

Cheers!

------------------------------------------------------------------------------------

Usage for the packages-templates:

extract the attached package in the woof tree
add the following to your specs file

yes|llvm-cut|llvm|exe,dev,doc,nls

(I have no idea if more deps are needed. Before you do this just add llvm to your running setup and reboot. see if GLX works and check /var/log/xorg.0.log for errors)

run ./1download an llvm will be downloaded
run ./2createpackages and choose llvm-cut

Check that you have a sane llvm-cut in packages-slacko

build the iso

---------------------------------------------------------------------------------------

UPDATE:

from the slackware-current LLVM package I ran the doinst.sh (to create the symlinks) , deleted everything except /usr/lib (which is what the template does) deleted all the *.la, *.a libs (which go to devx) and repackaged as llvm-3.7.0-i586.pet, weighing in at 12MB (xz compressed). Size on disk is ~57MB.

I then booted into my slackocurrent install and installed the pet.

Results are good:

Code: Select all

# glxgears
Running synchronized to the vertical refresh.  The framerate should be
approximately the same as the monitor refresh rate.
302 frames in 5.0 seconds = 60.249 FPS
300 frames in 5.0 seconds = 59.869 FPS
300 frames in 5.0 seconds = 59.889 FPS
# export vblank_mode=0
# glxgears            
ATTENTION: default value of option vblank_mode overridden by environment.
6878 frames in 5.0 seconds = 1375.497 FPS
6884 frames in 5.0 seconds = 1376.691 FPS
6888 frames in 5.0 seconds = 1377.411 FPS
6882 frames in 5.0 seconds = 1376.382 FPS
6881 frames in 5.0 seconds = 1376.085 FPS
# . /etc/DISTRO_SPECS 
# echo $DISTRO_FILE_PREFIX-$DISTRO_VERSION
slackocurrent-15.11.9

[peebee]

Many thanks Mick - all the above incorporated into next slacko-current - glxgears now working AOK on my nvidia desktop with nouveau driver.

[hamoudoudou]

LLVM still included in all puppies, excepted some slim ones
Francophones, can debate here
Musher0 ask feed back about Libs unused by users to issue a Puppy similar as x-slim for Slacko (mistfire) or Racy 5.3 (104mb Browser included)

[spiritwild]

Hmm, I removed said file and the phone rang. Raspy voice said I had 3 days.

Is that pretty common?

[musher0]

It is.
But don't worry about it.
They're just selling the wind.
They can't hurt anyone.

[8Geee]

Been reading this thread with interest.
I noted...

1.) Not in Slacko5.7 or my derivitive AtomicPup.
2.) PeeBee's list does not include any Intel vid driver. Tungsten = no, Gallium = yes

It appears all other vid drivers access it including swrast.

EDIT: Wikipedia- LLVM Features

Regards
8Geee

[musher0]

Hi 8gee.

I've been running xenialPup-706 without the LLVM library for 2 months now
and there is no difference in visual performance on this computer.

Another month without problems and I'll print myself a T-shirt that reads:
"LLVM is a hoax."

BFN.

[technosaurus]

Uggg. Llvm is so frustrating. It has no simple way of removing support for architectures that it supports, but are completely useless for 99% of people. For example most x86_64 linux users don't need to build Qualcomm's hexagon or Microsoft windows binaries much less jit build them. On top of that it is written in c++ vs c, so compilers have a more difficult time eliminating unused cruft - thus the gratuitous amount of relatively large binaries. They tried to mitigate this to some extent by optionally combining most of the little static libraries into a single shared library, but not all binaries or dependent projects can take advantage of the shared library version of llvm either due to lack of maintenance (internal binaries) or for performance reasons (Mesa) It _could_ be made smaller and still function by modifying the build process to exclude unnecessary *.a libs from the *.so and even further by excluding some *.o files from the *.o files. IIRC ttuuxxx did something similar for an older version of Firefox, but most people won't have the skill set to do it and of those that do, most won't have the patience. I lack the patience to do anything other than a 1 time hack job like my mostly static build of midori, but the sources are poorly organized/named (not as bad as Mozilla, but bad) so it's difficult to pinpoint which objects and libraries can be omitted and, since it is c++, more difficult to find unnecessary references to them in the sources.

TLDR it's possible to make it smaller but not easy... a good GSOC project.

As for Mesa, the primary user, it can be built without llvm and then the drivers that absolutely need llvm can be built against a static PIC build of llvm (so only those drivers absorb the cost, but only the minimum cost); however this is a complicated arrangement that most automated build systems can't support.

[musher0]

Ok...

technosaurus wrote:
> "TLDR it's possible to make it smaller but not easy... a good GSOC project."
Translation:
"Too long didn't read, it's possible... Google Summer of Code project."
I was lucky to have the search gods smile on me tonight!

~~~~~~~

FYI and everyone's, figosdev, on
http://softwarefreedom.jcink.net/index. ... &#entry116
8th post down, wrote:
> "the latest version of mkfigos deletes llvm.
> the one effect ive noticed is that webkit doesnt work without it.
> thats a pretty big thing,"

BFN.

[BarryK]

The latest EasyOS, 0.9.5, doesn't have llvm.

This distro is built with packages compiled from source in 'oe-qky-src', my fork of OpenEmbedded.

The only thing that I am missing out on is support for some video hardware, affecting a very small minority of users I think.

For the record:

https://github.com/bkauler/oe-qky-src

[musher0]

Ah, Barry, you were always the visionary!

Thanks for this info!

BFN.

[Marv]

Morning musher0,
A quiet Sunday morning so just for hoots over coffee, working on my fairly modified upupbb 18.05 +10, I removed the llvm libs and samba (a pretty thorough but probably not perfect manual scrub) and resquashed the main SFS. I am an all intel shop at this point, four i5s, two or three core 2 duos, and two Bay Trail desktops. Stopped using samba partly because of throughput and reliability issues with large files and partly just because it made me nervous. I just keep a USB3 HDD or two by my main computer and back up and transfer using them. It's fast and bulletproof and the extra walking doesn't hurt me. The space saved on the install doesn't matter to me at this time since my current hardware is all capable enough but I was just curious.

Things that I've tried and work here on this stripped version of upupbb:
My integrated pcmanfm and trashcan, glxgears, pmusic, gnome-mplayer, Slimjet 18.0.5.0 run-as-spot from SFS, Sylpheed, QtWebBrowser 3.8.5 portable install, gnumeric (simple financial spreadsheet), abiword (create, format, and print a testpage), networked printers (one HP4500 all-in-one and a Brother 2170 laserprinter) The Brother was already installed. I turned the firewall off and installed the usual HPlite. Both printers were autodetected by CUPS and the HP installed and printed a test page perfectly.

Things that fail: Haven't run into one yet. Certainly video driver dependent so I'm probably an outlier here with the all intel and pretty simple use but... I'll use this pup for a while and see what does break. I like it for it's non-gvfs nature.

Just fooling around

Update: A few days of steady use, including a kernel swap and the usual system tests of that, and I haven't found anything that is broken on this platform and pup by this change. I found my stripper script so the samba removal is complete and checked per the .packages list.