What does the LLVM library do and how badly do we need it?

For discussions about programming, programming questions/advice, and projects that don't really have anything to do with Puppy.
Message
Author
musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

What does the LLVM library do and how badly do we need it?

#1 Post by musher0 »

Hello all.

As the title says.

We have this huge LLVM library in the pup*.sfs of all Puppies. I read in the
wikipedia article about it that it is mostly good to help execution of programs
written in the Haskell and Java languages.

Maybe I understood this wrong, which is why I'm asking.

It's pushing 50 Mg's, so if we only need on certain occasions, why not make a
separate sfs of it, that people could load as needed? It would be a nice way to
trim down our future Pups.

TIA for any insights. BFN.
Last edited by musher0 on Tue 26 Jun 2018, 18:47, edited 1 time in total.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#2 Post by s243a »

Here's some stuff from google:
LLVM is an abbreviation of "Low Level Virtual Machine"; LLVM is:

A compilation strategy
A virtual instruction set
A compiler infrastructure
https://wiki.haskell.org/LLVM
LLVM is written in C++ and is designed for compile-time, link-time, run-time, and "idle-time" optimization of programs written in arbitrary programming languages. Originally implemented for C and C++, the language-agnostic design of LLVM has since spawned a wide variety of front ends: languages with compilers that use LLVM include ActionScript, Ada, C#,[4][5][6] Common Lisp, Crystal, CUDA, D, Delphi, Fortran, Graphical G Programming Language,[7] Halide, Haskell, Java bytecode, Julia, Kotlin, Lua, Objective-C, OpenGL Shading Language, Pony,[8] Python, R, Ruby,[9] Rust, Scala,[10] Swift, and Xojo.
https://en.wikipedia.org/wiki/LLVM

User avatar
peebee
Posts: 4370
Joined: Sun 21 Sep 2008, 12:31
Location: Worcestershire, UK
Contact:

Re: What does the LLVM library do and how badly do we need it?

#3 Post by peebee »

musher0 wrote:We have this huge LLVM library in the pup*.sfs of all Puppies. .....
It's pushing 50 Mg's, so if we only need on certain occasions, why not make a
separate sfs of it
It's a dependency of many xorg/xserver drivers so these may not work if it is removed.
Those drivers could possibly be compiled specially for Puppy but that is a big job (out of interest does BarryK have LLVM in his Easy builds?? I think he may be on a much older version of xorg??)
Shipping a separate sfs in the iso would not reduce the iso size.....

So LLVM is a necessary evil from building Pups from components provided by other systems I'm afraid.

p.s. llvm-cut contributes c. 12.6MB to the iso size.....50MB is the amount of memory it occupies once loaded.
ImageLxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#4 Post by musher0 »

@s243a: thanks for the additional info.

@peebee: also thanks for your reply.

However, on this trimmed xenialPup-706, I've transplanted the LLVM lib to the devx. I've
been running this Pup for 4 hours in a row without LLVM without any problems. Are
you sure this LLVM thing is not some kind of folklore or overstatement? You're an
experienced dev, of course, and I want to believe you, but what you say and my hands-
on experience do not jive.

I remember having had a similar conversation with former member ASRI. (It's on this
forum somewhere.) He ran his educational Pup without LLVM for a while and had no
problems.

How do we test this LLVM lib, to be sure it is needed by us simple mortals? Does it
leave a trace when it kicks in or after it has done so?

BFN.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#5 Post by musher0 »

Hello again, peebee.

This is the the size of the LLVM on my xenialPup-706:
[/usr/lib]>lg -h *LL*
lrwxrwxrwx 1 16 jun 20 19:03 libLLVM-4.0.so -> libLLVM-4.0.so.1
-rw-r--r-- 1 51M oct 20 2017 libLLVM-4.0.so.1
If you know of a smaller implementation, and where to find it, please let us know?

TIA.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

Sailor Enceladus
Posts: 1543
Joined: Mon 22 Feb 2016, 19:43

#6 Post by Sailor Enceladus »

I think if you build Slacko 14.1 it doesn't need llvm, but 14.2 uses it for some drivers. Maybe it's the same for Tahr vs. Xenial.

One way to see what may use it is to remove llvm with "Remove Builtin Packages" then run checkdeps / from terminal:
http://www.murga-linux.com/puppy/viewto ... 690#958690

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#7 Post by musher0 »

Many thanks, Sailor Enceladus.

@peebee: the github page for llvm-cut does not exist anymore.

Does anybody have a lead?

TIA
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
peebee
Posts: 4370
Joined: Sun 21 Sep 2008, 12:31
Location: Worcestershire, UK
Contact:

#8 Post by peebee »

musher0 wrote:Many thanks, Sailor Enceladus.

@peebee: the github page for llvm-cut does not exist anymore.

Does anybody have a lead?

TIA
Testing branch is recently different to master:
https://github.com/puppylinux-woof-CE/w ... s/llvm-cut

https://github.com/puppylinux-woof-CE/w ... _FIXUPHACK
ImageLxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#9 Post by musher0 »

Hi peebee and all.

In other words, nothing at the moment. Thanks for being so frank!

In any case, I wasn't asking about our Puppy situation, but generally, if there exists
a trimmed down LLVM library.

I ran the test suggested by Sailor E. and got pretty much the same results.So the
utility of this LLVM lib is pretty much limited to Puppyists with fancy video cards.

So from now on, LLVM will be a separate sfs in my Pups: those who need it,
download it. Same as we've been doing for ages, with the devx. In so doing, we'll
be shaving 51 Mb's unpacked (or 29 Mb's squashed) off the iso for the majority
who don't need it.

Why make the majority pay -- download wise -- for the needs of the few.

Finally, this thread WAS useful. Thanks to all who contributed.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
rcrsn51
Posts: 13096
Joined: Tue 05 Sep 2006, 13:50
Location: Stratford, Ontario

#10 Post by rcrsn51 »

At some point in the Puppy past, the decision was made to include accelerated video (mesa/dri). This would let users run things like Google Earth OOTB. This is how LLVM gets introduced.

So another approach is to remove all the mesa/dri stuff from the woof build.

Users can add it (and all the dependencies) as needed. This is how Stretch-Live handles it.

dancytron
Posts: 1519
Joined: Wed 18 Jul 2012, 19:20

#11 Post by dancytron »

rcrsn51 wrote:At some point in the Puppy past, the decision was made to include accelerated video (mesa/dri). This would let users run things like Google Earth OOTB. This is how LLVM gets introduced.

So another approach is to remove all the mesa/dri stuff from the woof build.

Users can add it (and all the dependencies) as needed. This is how Stretch-Live handles it.
Thinking out loud.

So, a compromise approach might be to use ydrv*.sfs or one of the other *drv*.sfs files to hold LLVM and all the mesa-dri stuff. Normal users that want Accelerated video out of the box could just leave it be and those who would rather save 50 Meg can delete it from their frugal install.
Last edited by dancytron on Tue 26 Jun 2018, 20:52, edited 1 time in total.

User avatar
rcrsn51
Posts: 13096
Joined: Tue 05 Sep 2006, 13:50
Location: Stratford, Ontario

#12 Post by rcrsn51 »

You would need to determine something first. Do sfs modules get loaded BEFORE X starts?

I used a similar trick in Devuan live where I put the firmware required by the radeon video driver in a squashfs module. It DID get loaded first and X started.

dancytron
Posts: 1519
Joined: Wed 18 Jul 2012, 19:20

#13 Post by dancytron »

rcrsn51 wrote:You would need to determine something first. Do sfs modules get loaded BEFORE X starts?

I used a similar trick in Devuan live where I put the firmware required by the radeon video driver in a squashfs module. It DID get loaded first and X started.
I think the answer to that for *drv*.sfs files is "Yes" they are loaded before X gets started. Regular ones, I'm not so sure.

See http://www.murga-linux.com/puppy/viewto ... 486#906486

User avatar
rcrsn51
Posts: 13096
Joined: Tue 05 Sep 2006, 13:50
Location: Stratford, Ontario

#14 Post by rcrsn51 »

If not, an X restart might be required.

As a proof-of-concept, I made a squashfs module of libgl1-mesa-dri + mesa-utils. I dropped it into a Debian-live frugal install and it appears to be working.

dancytron
Posts: 1519
Joined: Wed 18 Jul 2012, 19:20

#15 Post by dancytron »

rcrsn51 wrote:If not, an X restart might be required.

As a proof-of-concept, I made a squashfs module of libgl1-mesa-dri + mesa-utils. I dropped it into a Debian-live frugal install and it appears to be working.
I don't do it anymore, but I used to use apt2sfs to make an sfs with libgl1-mesa-dri, mesa-utils, the realtek drivers for my ethernet, and a few other things. It worked fine.

For Puppy, it seems like an *drv*.sfs with mesa, mesa-utils, llvm, and some of the less common video card drivers would be worth someone trying.

kirk
Posts: 1553
Joined: Fri 11 Nov 2005, 19:04
Location: florida

#16 Post by kirk »

LLVM is required for Gallium3D, r300, and radeonsi Mesa drivers. So if you don't need those, you can build mesa without llvm.

The normal way to compile LLVM is with the "-DLLVM_BUILD_LLVM_DYLIB=ON" option which results in it being huge. You can compile LLVM with the "-DBUILD_SHARED_LIBS=ON" which makes for a relatively small build, but it's marked experimental and for good reason, it also results in radeon dri drivers being broken in strange ways. For Fatdog64-720 we compiled LLVM static, which ended up saving some space.

User avatar
nosystemdthanks
Posts: 703
Joined: Thu 03 May 2018, 16:13
Contact:

#17 Post by nosystemdthanks »

i admit that im just as curious about this as musher, though since i always use python i accept that there are going to be some large binaries (or binaries that add up to a lot) here and there.

my philosophy is that if it fits on a cd in 2018, we are doing pretty good about size! you can hardly find a 1gb usb anymore, theyre all 4 and 8 and 16, the cheapo ones. of course i care about download size as much as anybody.

while i think corepup is a great idea just so wanderer (or anybody with similar interests) can easily swap components in/out, i used tinycore when it was fairly new and ive come to the opinion that "a tiny core" is only an interim goal, i.e. its not how 99% of people want to distribute a distro. its a fine place to start, i encourage anybody who wants to start there.

but most people are going to want xorg, everyone "should" want python (but not everybody does) and most people want a web browser that can do javascript, even if they can turn it off.

with the puppy community you can never be sure though... i mean the sort of weirdness that you find here is exactly what makes puppy so customiseble. from a variety of tools and methods even.

if you look at the 12-year history of this forum, customiseable has ultimately won in popularity over small size every time... when i used puppy every day, it took up less than half a cd. dsl was smaller, but less fun, most people here (no surprise) liked puppy better.

which is why puppy is now full-cd sized, i guess. if i can, i keep my own stuff between 500 and 600. one of my favourite tools is qemu, because you can make an iso without llvm and boot it immediately, then find out what works without it.
[color=green]The freedom to NOT run the software, to be free to avoid vendor lock-in through appropriate modularization/encapsulation and minimized dependencies; meaning any free software can be replaced with a user’s preferred alternatives.[/color]

User avatar
Smithy
Posts: 1151
Joined: Mon 12 Dec 2011, 11:17

#18 Post by Smithy »

Good find Musher ya lunatic.
So does that mean the DRI folder can be zapped as well? 139mb.
Does that mean we will lose the comforting glx gears (the grinding mills of God in full screen).
More ram means "better" plugins from my perspective :)
I've bumped this to remember what I have removed from pup..

User avatar
rcrsn51
Posts: 13096
Joined: Tue 05 Sep 2006, 13:50
Location: Stratford, Ontario

#19 Post by rcrsn51 »

Smithy wrote:Does that mean we will lose the comforting glx gears (the grinding mills of God in full screen)
You don't have to lose anything. You just need a way to make accelerated video an optional feature without breaking stuff.

User avatar
Smithy
Posts: 1151
Joined: Mon 12 Dec 2011, 11:17

#20 Post by Smithy »

So the DRI has to stay?
The LLV has been gone for days with no (apparent) ill effects. Mind you Intel, Artful.

Post Reply