Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Tue 16 Jul 2019, 06:16
All times are UTC - 4
 Forum index » Off-Topic Area » Programming
Busybox - fat and fast?
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [15 Posts]  
Author Message
MochiMoppel


Joined: 26 Jan 2011
Posts: 1852
Location: Japan

PostPosted: Wed 10 Jul 2019, 22:20    Post subject:  Busybox - fat and fast?  

I was a bit shocked when radky wrote here that woof-CE has moved to busybox 1.30.1.

The lack of backward compatibility for the timeout command was one thing, but what I was most concerned of was the size. My slacko 5.6 comes with busybox v1.21.0. At 689K it is not really lightweight compared to single GNU commands, which typically weigh only around 50K. But now this new v1.30.1 is a real monster with its 958K.

I always assumed that running a busybox command must be slower than a slim GNU command and running a fat busybox version must be slower than running a leaner version.

Seems I was wrong. I tested the timeout command on all 3 versions:

For busybox 1.21.0
Code:
time busybox timeout -t 1 yes


For busybox 1.30.1
Code:
time busybox timeout 1 yes


For GNU timeout (coreutils 8.24)
Code:
time timeout 1 yes


Results:
    busybox timeout (BusyBox v1.21.0; size 689K)
    real 0m1.006s
    user 0m0.037s
    sys 0m0.383s

    busybox timeout (BusyBox v1.30.1; size 958K)
    real 0m1.002s
    user 0m0.040s
    sys 0m0.490s

    timeout (GNU coreutils 8.24; size 47K)
    real 0m1.006s
    user 0m0.050s
    sys 0m0.480s



I also compared other commands. Not all showed dramatic differences, but I could see a tendency that newer busyboxes are faster than older (more efficient code?) and busyboxes are not slower than the coreutils. Surprising, but good to know Laughing
Back to top
View user's profile Send private message 
Lobster
Official Crustacean


Joined: 04 May 2005
Posts: 15438
Location: Paradox Realm

PostPosted: Thu 11 Jul 2019, 06:03    Post subject:  

Is busybox compressed with something like UPX?
https://upx.github.io/

_________________
Raspberry Pi 4 to run Puppi
NEW BusterPup Beta - test now
Back to top
View user's profile Send private message Visit poster's website 
williams2

Joined: 14 Dec 2018
Posts: 105

PostPosted: Thu 11 Jul 2019, 14:20    Post subject:  

Quote:
Is busybox compressed with something like UPX?


Yes. Busybox is in the main Puppy sfs (squash file system). Everything in the squash file system is compressed.

There is not much point in having a upxed file in an sfs, it is already compresssed.

Older versions of upx did not permit multiple instances of an executable to share resources. I think the newer upx does permit sharing.

The busybox in bionicpup64 8.0 is compressed because it is in the main sfs file. It is about 938k, but it is larger because it is statically linked. That is, it does not need to use library files.

The first time busybox runs, it is copied and decompressed from the sfs file into a ram buffer. It is copied to cache space and copied into ram to be executed.

After that, any time busybox is run again, the system gets it already uncompressed from the cache memory in ram,

But busybox is already executing in ram. Init, which is busybox, and which starts the linux system is always in ram executing. And init runs several instances of getty. Getty is also busybox.

When two or more instances of a program are run, the system is smart enough to allow the identical parts of the program to be shared by each instance.

For example, you might have mpv playing a movie and you might have an urxvt window, and you might be looking at a text file in geany.

If you then open 2 or 3 more urxvt windows, and open 2 or 3 more geany windows, and run another movie in mpv, the system does not need to copy urxvt or geany or mpv to ram again. It can use the first copies that are still in ram.

Because busybox is always executing in ram, running a busybox utility program is very efficient. The executable is already in ram, ready to be used. So it is fast and takes up very little space.

For example, opening a urxvt window and typing ash shows ash using 4k of ram, according to htop. And getty is using 4k each.

Another thing, because busybox was compiled statically, and does not need any library files, it should run in almost any linux, 32 bit or 64 bit. And linking to library files is slower than not needing to link.
Back to top
View user's profile Send private message 
Lobster
Official Crustacean


Joined: 04 May 2005
Posts: 15438
Location: Paradox Realm

PostPosted: Thu 11 Jul 2019, 15:00    Post subject:  

Thanks @Williams2 very comprehensive explanation, my next question was going to be about multiple instances ...

Cool Very Happy

_________________
Raspberry Pi 4 to run Puppi
NEW BusterPup Beta - test now
Back to top
View user's profile Send private message Visit poster's website 
musher0

Joined: 04 Jan 2009
Posts: 14225
Location: Gatineau (Qc), Canada

PostPosted: Thu 11 Jul 2019, 15:03    Post subject:  

Except that, compared to the originals, some utilities in busybox are so trimmed down,
they are almost useless: take the busybox less and the busybox lzop, for example.

There may be a new busybox out, but its principle remains the same: chop-chop-chop the
original utility until it's a ghost ot its former self.

If you sing the praise of busybox, also hurry up and install a recent copy of the GNU
coreutils & co, in case some busybox utility can't do the job the original does.

IMO, busybox has a decent implementation of ash -- if you want speedy execution for an
uncomplicated script. That's about it.

_________________
musher0
~~~~~~~~~~
Je suis né pour aimer et non pas pour haïr. (Sophocle) /
I was born to love and not to hate. (Sophocles)
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3243

PostPosted: Thu 11 Jul 2019, 18:11    Post subject:  

:%d
_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh

Last edited by rufwoof on Sun 14 Jul 2019, 16:01; edited 2 times in total
Back to top
View user's profile Send private message 
musher0

Joined: 04 Jan 2009
Posts: 14225
Location: Gatineau (Qc), Canada

PostPosted: Thu 11 Jul 2019, 22:32    Post subject:  

Hi rufwoof.

busybox is useful to cross the "initrd border". After that, its usefulness depends on the
user's needs. It may be sufficient for the needs of a power user such as yourself, but
run-of-the-mill users need to know the complete story about busybox.

IMO, it's NOT
Quote:
functional enough for general usage,
as you mention. Unless one lets the people at busybox to decide what "general usage"
is for the rest of us.

The example I always give to prove this point is: the busybox less has four lines of on-
line help, whereas the real less by greenwood software has four pages of on-line help.

Where did 95% of the functions go in the busybox less? In the shredder, because
someone at busybox thought, for example, that less' capacity to read many texts at
once and offer the user the capacity to go from one text to the other, wasn't worth it --
and so on for the other functions of less (show ANSI colors, highlight text, etc.).

While the busybox less may be a programming feat, it is giving the wrong idea about
the real less, and spreading fake news about what the real less can do for the user.

And so on for lzop, mount, etc., and all the utilities busybox pretends to be replacing.

I think the conclusion is obvious: the user should be made aware of what busybox is:
it's useful in the limited initrd context, and that's it.

To users: if you feel it would be nice that a busybox utility had a certain feature,
please go look for the original utility: it probably has that that feature and busybox
is probably short-changing you!


BFN.

_________________
musher0
~~~~~~~~~~
Je suis né pour aimer et non pas pour haïr. (Sophocle) /
I was born to love and not to hate. (Sophocles)
Back to top
View user's profile Send private message 
MochiMoppel


Joined: 26 Jan 2011
Posts: 1852
Location: Japan

PostPosted: Fri 12 Jul 2019, 00:47    Post subject:  

williams2 wrote:
When two or more instances of a program are run, the system is smart enough to allow the identical parts of the program to be shared by each instance.
I understand this, but what I don't understand:
My system starts with the old 689K bb and keeps it running. When I then - as in my test - start a new 958K bb, I would expect that this is not treated as merely another instance of the already running version. After all the new bb contains considerable changes and it is bigger and therefore additional stuff has to be loaded. I don't see how starting a new and different version can result in faster execution when compared to starting exactly the same version that is already running.
Back to top
View user's profile Send private message 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 8907
Location: Perth, Western Australia

PostPosted: Fri 12 Jul 2019, 08:56    Post subject:  

I haven't upgraded busybox to 1.30.x in EasyOS, as the 'dc' applet is broken.

It has been changed to work more like the full dc, which will break some scripts that use the busybox version.

But the real problem is that it does not always work. It is supposed to do arithmetic on numbers with fractions, but I found that in certain cases it truncated the fractional part of the result,

I can't recall the details.

I did upgrade busybox for the initrd, but do not use dc in that situation.

_________________
http://bkhome.org/news/
Back to top
View user's profile Send private message Visit poster's website 
MochiMoppel


Joined: 26 Jan 2011
Posts: 1852
Location: Japan

PostPosted: Fri 12 Jul 2019, 09:31    Post subject:  

BarryK wrote:
I haven't upgraded busybox to 1.30.x in EasyOS, as the 'dc' applet is broken.
Was broken in 1.30.0
1.30.1 is a bugfix release that contains "fixes for bc/dc". May solve your problems.
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3243

PostPosted: Fri 12 Jul 2019, 13:49    Post subject:  

:%d
_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh

Last edited by rufwoof on Sun 14 Jul 2019, 16:01; edited 5 times in total
Back to top
View user's profile Send private message 
MochiMoppel


Joined: 26 Jan 2011
Posts: 1852
Location: Japan

PostPosted: Fri 12 Jul 2019, 19:07    Post subject:  

@rufwoof: How does this relate to the thread topic?
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4855
Location: Blue Springs, MO

PostPosted: Sat 13 Jul 2019, 22:25    Post subject:  

In case nobody else noticed, it is now possible to embed shell scripts in busybox.

Advantages
For the initrd/initramfs, that saves close to 4kb and 1 inode per small script
It is still compressible (on squashfs or using upx)
It should also give faster access times to scripts especially for full installs.
/bin/sh can be linked to bash and busybox compatible scripts can still run faster
Only 1 file to distribute

The downside:
Larger .text section in busybox (though smaller in total)
Scripts would no longer be user editable
Requires busybox rebuild to fix a script

rufwoof wrote:
Just for reference, a script I use to pull out all libs used by a program - so you can drop them into initrd ...
What is this tinycore stuff? We do static builds for initrd binaries... could be useful for containers though (as in Barry's EasyOS)

MochiMoppel wrote:
BarryK wrote:
I haven't upgraded busybox to 1.30.x in EasyOS, as the 'dc' applet is broken.
Was broken in 1.30.0
1.30.1 is a bugfix release that contains "fixes for bc/dc". May solve your problems.
select N for the "big" version of dc if you want the old one that is not based on the new bc code (IMHO the bc code needs about 5-10 more cleanup passes before it should be a default applet anyhow)
_________________
Check out my github repositories. I may eventually get around to updating my blogspot.
Back to top
View user's profile Send private message Visit poster's website 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 8907
Location: Perth, Western Australia

PostPosted: Sun 14 Jul 2019, 03:02    Post subject:  

MochiMoppel wrote:
BarryK wrote:
I haven't upgraded busybox to 1.30.x in EasyOS, as the 'dc' applet is broken.
Was broken in 1.30.0
1.30.1 is a bugfix release that contains "fixes for bc/dc". May solve your problems.


Ah, that's good. Thanks for the info.

_________________
http://bkhome.org/news/
Back to top
View user's profile Send private message Visit poster's website 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 8907
Location: Perth, Western Australia

PostPosted: Sun 14 Jul 2019, 03:17    Post subject:  

technosaurus wrote:

MochiMoppel wrote:
BarryK wrote:
I haven't upgraded busybox to 1.30.x in EasyOS, as the 'dc' applet is broken.
Was broken in 1.30.0
1.30.1 is a bugfix release that contains "fixes for bc/dc". May solve your problems.
select N for the "big" version of dc if you want the old one that is not based on the new bc code (IMHO the bc code needs about 5-10 more cleanup passes before it should be a default applet anyhow)


That's good too, if I can keep the old behaviour of dc.

Interesting about the embedded scripts.

_________________
http://bkhome.org/news/
Back to top
View user's profile Send private message Visit poster's website 
Display posts from previous:   Sort by:   
Page 1 of 1 [15 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Off-Topic Area » Programming
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0627s ][ Queries: 12 (0.0039s) ][ GZIP on ]