toybox 957 + patches

Core libraries and systems
Post Reply
Message
Author
Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

toybox 957 + patches

#1 Post by Ibidem »

toybox (http://landley.net/toybox) is a multicall binary (like busybox), with much more infrastructure and simpler code.

Here's a toybox binary compiled statically against musl;
269656 bytes stripped, with the following "toys":

Code: Select all

 basename bzcat cal cat chgrp chmod chown chroot chvt cksum clear cmp 
comm count cp cut date df dirname dmesg dos2unix du echo eject env 
expand expr false find free grep groups head help hostname id ifconfig 
insmod kill killall klogd link ln logger login logname losetup ls 
lsmod lspci lsusb md5sum mdev mkdir mkfifo mknod mkswap mktemp modinfo 
mountpoint mv nbd_client nc netcat nice nohup od oneit passwd paste 
patch pidof pivot_root printenv pwd readahead readlink realpath rev 
rm rmdir rmmod seq setsid sha1sum sleep sort split stat swapoff swapon 
switch_root sync syslogd tac tail taskset tee time timeout touch true 
truncate tty uname uniq unix2dos unlink unshare uptime usleep uudecode 
uuencode vconfig vmstat w wc which who whoami xargs xzcat yes 
Full help can be gotten by

Code: Select all

for i in toybox `./toybox`; do ./toybox $i --help >>toybox-help.txt;done
I disabled hello (example), toysh (doesn't work reliably yet/Rob Landley has stated that it's not expected to really work yet), sed, test, mke2fs (all not functional yet), and catv.
I added lspci (wrote this weekend, tried to send the patch...), syslogd (from a patch floating on the list), and a patch to shrink xzcat.
mount not included.

lspci as included is a superset of busybox lspci; lspci -e prints all 24 bits of the class like elspci -l.

UPDATE: Grab the last toybox I post; this is the oldest and lacks several features.
Attachments
toybox.gz
gzip -9 of static toybox.
gunzip toybox.gz to use.
(149.87 KiB) Downloaded 624 times
Last edited by Ibidem on Wed 14 Aug 2013, 02:44, edited 1 time in total.

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

toybox 985 + patches

#2 Post by Ibidem »

OK, all the patches above have been merged.
I've added text output to lspci (default and -nn); it looks in
/usr/share/pci.ids
for text.
renice (from Strake) has been merged.
Currently enabled:

Code: Select all

basename bzcat cal cat catv chgrp chmod chown chroot chvt cksum clear 
cmp comm count cp cut date df dirname dmesg dos2unix du echo eject 
env expand expr false find free grep groups head help hostname id 
ifconfig insmod kill killall klogd link ln logger login logname losetup 
ls lsmod lspci lsusb md5sum mdev mkdir mkfifo mknod mkswap mktemp 
modinfo mountpoint mv nbd_client nc netcat netstat nice nohup od oneit 
passwd paste patch pgrep pidof pivot_root pkill printenv pwd readahead 
readlink realpath renice rev rm rmdir rmmod sed seq setsid sha1sum 
sleep sort split stat swapoff swapon switch_root sync syslogd tac 
tail taskset tee test time timeout touch true truncate tty uname uniq 
unix2dos unlink unshare uptime usleep uudecode uuencode vconfig vmstat 
w wc which who whoami xargs xzcat yes
This is static, linked with musl, and sstrip'd.
Attachments
toybox.gz
gunzip toybox.gz; chmod +x toybox
(158.72 KiB) Downloaded 602 times

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

toybox 1004 + patches

#3 Post by Ibidem »

Here's toybox 1004 + patches.
It includes su (needs suid root).
Full list of applets:

Code: Select all

basename bzcat cal cat chgrp chmod chown chroot chvt cksum clear cmp 
comm count cp cut date df dirname dmesg dos2unix du echo egrep eject 
env expand expr fallocate false fdisk fgrep find free grep groups 
head help hostname id ifconfig init insmod kill killall klogd link 
ln logger login logname losetup ls lsmod lspci lsusb md5sum mkdir 
mkfifo mknod mkswap mktemp modinfo mountpoint mv nbd_client nc netcat 
netstat nice nohup od oneit passwd paste patch pgrep pidof pivot_root 
pkill printenv pwd readahead readlink realpath renice resolve_modalias 
rev rm rmdir rmmod seq setsid sha1sum sleep sort split stat su swapoff 
swapon switch_root sync syslogd tac tail taskset tee test time timeout 
touch true truncate tty uname uniq unix2dos unlink unshare uptime 
usleep uudecode uuencode vconfig vmstat w wc which who whoami xargs 
xzcat yes
lspci now supports -e -m -k [-n | -nn] and -i /path/to/pci.ids
resolve_modalias prints a list of modules matching a given modalias.

By the way, I'm curious who is downloading this and what for. Would you please mention that?
Attachments
toybox.gz
You should know the drill by now.
(171.92 KiB) Downloaded 650 times

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#4 Post by amigo »

"all the patches above have been merged" You need to post these patches since you are distributing a binary from modified sources...

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#5 Post by Ibidem »

amigo wrote:"all the patches above have been merged" You need to post these patches since you are distributing a binary from modified sources...
Amigo: 3 points.
1) toybox license:
http://landley.net/hg/toybox/file/03f72b57a092/LICENSE wrote: Copyright (C) 2006, 2013 by Rob Landley <rob@landley.net>

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
2) The source I use is in the "build" branch here: https://github.com/idunham/toybox/

3) The license is much nicer than the GPL for recovery and use as a portable binary: you don't need a copy of the license to copy the binary.

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

toybox (~0.4.6?) + patches

#6 Post by Ibidem »

Rob claims he's got a new release ready, which should be 0.4.6. But the release notes aren't up yet.
The major new commands, all default n because they haven't been cleaned up, are dhcp (a dhcp client, fairly similar to udhcpc), dhcpd (dhcp server, similar to udhcpd), and route--all thanks to Madhur Verma and Kyungwan Han.
If you want to use dhcp, grab a default.script that works with udhcpc and put it in /usr/share/dhcp/default.script.
Other new commands in upstream toybox are pmap (Ashwini Sharma), pwdx (Lukasz Skalski; prints pwd of a specified command), acpi (mine; reports battery use with -b and power supply status with -a), and a preliminary version of umount from Rob Landley.
I've dug up a mount implementation (from Ashwini Kumar, Madhur Verma, and Kyungwan Han) that was overly large and special-cased nfs, pared it down, and updated it for a couple of changes in toybox.
I haven't properly submitted resolve_modalias, since it's basically demo code that can be useful. And I'm carrying the lspci -i patch, because I've been trying to give Rob a chance to catch up.

Currently implemented commands:

Code: Select all

acpi basename bzcat cal cat catv chgrp chmod chown chroot chvt cksum 
clear cmp comm count cp cut date df dhcp dhcpd dirname dmesg dos2unix 
du echo egrep eject env expand expr false fdisk fgrep find free grep 
groups head help hostname id ifconfig init insmod kill killall klogd 
link ln logger login logname losetup ls lsmod lspci lsusb md5sum mkdir 
mkfifo mknod mkswap mktemp modinfo mount mountpoint mv nbd_client 
nc netcat netstat nice nohup od oneit passwd paste patch pgrep pidof 
pivot_root pkill pmap printenv ps pwd pwdx readahead readlink realpath 
renice resolve_modalias rev rm rmdir rmmod route seq setsid sha1sum 
sleep sort split stat su swapoff swapon switch_root sync syslogd tac 
tail taskset tee test time timeout touch true truncate tty umount 
uname uniq unix2dos unlink unshare uptime usleep uudecode uuencode 
vconfig vmstat w watch wc which who whoami xargs xzcat yes 
The binary and a diff from hg revision 1068 is attached.

(Currently I'm working on cpio, for what that's worth. I have a useable generator (cpio -o) and some code towards -i and -t.)
Attachments
build.diff.gz
The diff from toybox 1068.
(6.68 KiB) Downloaded 540 times
toybox.gz
sstrip'd &amp; gzip'd build of toybox. See https://github.com/idunham/toybox/tree/build or https://github.com/idunham/toybox/compare/build for source.
(195.04 KiB) Downloaded 541 times

User avatar
Q5sys
Posts: 1105
Joined: Thu 11 Dec 2008, 19:49
Contact:

#7 Post by Q5sys »

ibidem,
This is awesome work. Are you building toybox with uclib and tcc or no?
I know Landley was working towards having a fully independant toolchain and gnu tools replacement. I havent looked into it recently to find out how far along he is.

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#8 Post by Ibidem »

Q5sys wrote:ibidem,
This is awesome work. Are you building toybox with uclib and tcc or no?
I know Landley was working towards having a fully independant toolchain and gnu tools replacement. I havent looked into it recently to find out how far along he is.
Using musl as libc and gcc as compiler.
Rob decided that (1) musl was the libc to use (because it's modern, standards-conformant, has clear code, is MIT-licensed, and is making progress on its own much faster),
(2) LLVM/Clang would probably be the binutils/gcc replacement, by virtue of being nearly there while he was a long ways from starting "qcc", and
(3) if he does a compiler/toolchain, it will start with tcc + the qemu code generator, and he'll start it once toybox hits 1.0. That's a ways off.

toybox is the replacement for gnu tools other than development tools. It also replaces/will replace a number of linux tools. It shouldn't implement "everything that busybox has" (Rob maintains that that's too much), and it has stricter code standards and more code sharing, as well as infrastructure that doesn't need patching...
I intend to start a thread about writing/porting toybox "toys" (the same as applets in busybox) soon.

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

toybox 1119 + patches

#9 Post by Ibidem »

Here's toybox 1119 + patches (buildfix for dumpleases, a couple flags for cpio, mount as previously described, resolve_modalias); as usual, it's statically linked against musl and sstrip'd.
Sources may be found in the usual place ("build" branch of my git repo) if you care.
Applets:

Code: Select all

acpi addgroup adduser basename blkid bzcat cal cat catv chgrp chmod 
chown chroot chvt cksum clear cmp comm count cp cpio cut date dd df 
dhcp dhcpd dirname dmesg dos2unix du dumpleases echo egrep eject env 
expand expr fallocate false fdisk fgrep find free fsck fstype grep 
groupadd groups halt head help hostname id ifconfig init insmod kill 
killall klogd link ln logger login logname losetup ls lsmod lspci 
lsusb md5sum mdev mkdir mkfifo mknod mkpasswd mkswap mktemp modinfo 
mount mountpoint mv nc netcat netstat nice nl nohup od passwd paste 
patch pgrep pidof pivot_root pkill pmap poweroff printenv ps pwd readahead 
readlink realpath reboot renice resolve_modalias rev rm rmdir rmmod 
route seq setsid sha1sum sleep sort split stat su swapoff swapon switch_root 
sync syslogd tac tail taskset tee test time timeout top touch traceroute 
true truncate tty umount uname uniq unix2dos unlink unshare uptime 
useradd usleep uudecode uuencode vconfig vmstat watch wc which whoami 
xargs xzcat yes

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#10 Post by Ibidem »

And just tonight:
Based on toybox 1136, plus a couple build fixes and a couple more commands...

Code: Select all

acpi addgroup adduser basename blkid bzcat cal cat chgrp chmod chown 
chroot chvt cksum clear cmp comm count cp cpio cut date dd df dhcp 
dhcpd dirname dmesg dos2unix du dumpleases echo egrep eject env expand 
expr fallocate false fdisk fgrep find free fsck fstype grep groupadd 
groups halt head help hostname id ifconfig init insmod kill killall 
klogd link ln logger login logname losetup ls lsmod lspci lsusb md5sum 
mdev mkdir mkfifo mknod mkpasswd mkswap mktemp modinfo mount mountpoint 
mv nc netcat netstat nice nl nohup od passwd paste patch pgrep pidof 
pivot_root pkill pmap poweroff printenv ps pwd readlink realpath reboot 
renice resolve_modalias rev rm rmdir rmmod route seq setsid sha1sum 
sleep sort split stat su swapoff swapon switch_root sync syslogd tac 
tail taskset tee test tftpd time timeout top touch traceroute true 
truncate tty umount uname uniq unix2dos unlink unshare uptime useradd 
usleep uudecode uuencode watch wc which whoami xargs xzcat yes 
New bits that I'm aware of are tftpd and some cleanup.
Just for fun, I've booted with this and openbsd pdksh. It boots, though some work needs to happen.
Attachments
patch.diff.bz2
Diff from toybox 1136
(8.07 KiB) Downloaded 451 times

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#11 Post by musher0 »

Hello, Ibidem.

Thanks for this, I'll study it.

May I suggest the following be used to better format the output of the toybox-help.txt file?

Code: Select all

> toybox-help.txt;for i in toybox `./toybox`; do echo "~~~~~~~~~~" >> toybox-help.txt;./toybox $i --help >>toybox-help.txt;echo >>toybox-help.txt;echo >> toybox-help.txt;done
The above one-liner will insert 2 empty lines between the command descriptions;
those separations should make the document easier to read.

Best regards.

musher0
Attachments
toybox-help_2lines-paragraph-3.jpg
(42 KiB) Downloaded 576 times
toybox-help.sh.zip
In regular bash script form.
(353 Bytes) Downloaded 463 times
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#12 Post by musher0 »

Hello again, Ibidem.

Are links to the commands allowed, like for busybox, or only little bash scripts
similar to the second line of the illustration above?

Thanks in advance.

musher0
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#13 Post by Ibidem »

musher0 wrote:Hello again, Ibidem.

Are links to the commands allowed, like for busybox, or only little bash scripts
similar to the second line of the illustration above?

Thanks in advance.

musher0
Short explanation: yes, links are allowed.

Technical explanation-the first string in argv[][] to not match "toybox" is the applet name it attempts to execute.
So all of these three commands are equivalent:

Code: Select all

acpi
toybox acpi
toybox toybox toybox-i486 acpi
Now, a note or two:
toybox aims to implement POSIX2008/POSIX2013; it is not intended to be a full clone of coreutils. Don't count on every option you are aware of for every command being available; refer to the standards if you think something's missing. Part of this is Rob Landley's anti-bloat agenda.

Also, it's still far from reaching the release date.
And some of the commands I've turned on are not quite complete, but appear to me to have "enough" functionality.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#14 Post by musher0 »

Thanks for the explanations, Ibidem.

You say that the toybox applets are not ready for productivity use yet. Now, if I find
that a toybox applet does a better job than the busybox one, I can use the toybox
one instead, I suppose?

Also, is it just me, or does the grep entry really re-appear 3-4 times in the help file?

BFN.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#15 Post by Ibidem »

musher0 wrote:Thanks for the explanations, Ibidem.

You say that the toybox applets are not ready for productivity use yet. Now, if I find
that a toybox applet does a better job than the busybox one, I can use the toybox
one instead, I suppose?
Some of the toybox applets are not ready for all uses.
The ones I built seemed useful to me, but may not have all the features you're after.

Yes, you can just replace the busybox applets where toybox is better. Just add links with the names you want.
Also, is it just me, or does the grep entry really re-appear 3-4 times in the help file?

BFN.
Yes, that's the case.
fgrep is the same as grep -F, and egrep means grep -E. So they're aliased to "grep", which means they have the same help.
A few other commands have similar issues; md5sum/shasum are one example.
The root issue is that the build system extracts one help message per file, so an "OLDTOY" gets the wrong help message.
Rob says that's something he wants to fix, but he intends to re-write that part of the build system first.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#16 Post by musher0 »

Thanks.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#17 Post by Ibidem »

New toybox build.
Features a rudimentary getty, with which I can boot the system using toybox init. Usage is simply
ttyN::respawn:/sbin/getty
(it ignores baud rate, and autodetects tty).
Also fixes some breakage in fdisk, and adds modprobe, ftpget, ftpput (all from Ashwini Sharma) and xfs_freeze (simple FIFREEZE/FITHAW wrapper, for if you want to temporarily make an fs readonly without causing any disruption beyond writes blocking).

Applets:

Code: Select all

acpi addgroup adduser basename blkid bzcat cal cat chgrp chmod chown 
chroot chvt cksum clear cmp comm count cp cpio cut date dd df dhcp 
dhcpd dirname dmesg dos2unix du dumpleases echo egrep eject env expand 
expr fallocate false fdisk fgrep find free fstype ftpget ftpput getty 
grep groupadd groups halt head help hostname id ifconfig init insmod 
kill killall klogd link ln logger login logname losetup ls lsmod lspci 
lsusb md5sum mdev mkdir mkfifo mknod mkpasswd mkswap mktemp modinfo 
modprobe mount mountpoint mv nc netcat netstat nice nl nohup od oneit 
passwd paste patch pgrep pidof pivot_root pkill pmap poweroff printenv 
ps pwd readlink realpath reboot renice resolve_modalias rev rm rmdir 
rmmod route seq setsid sha1sum sleep sort split stat su swapoff swapon 
switch_root sync syslogd tac tail taskset tee test tftpd time timeout 
top touch traceroute true truncate tty umount uname uniq unix2dos 
unlink unshare uptime useradd usleep uudecode uuencode vconfig vmstat 
watch wc which whoami xargs xfs_freeze xzcat yes 
Sources are in the usual place (https://github.com/idunham/toybox build).

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#18 Post by Ibidem »

And another build of toybox:
-A more complete getty
-fixup modprobe
-freeramdisk, openvt, deallocvt
-gzip (based on sflate)
-cpio -d
-grep -zZ
-I'm disabling dhcpd and related dumpleases, tftpd, and telnetd.

Commands:

Code: Select all

acpi addgroup adduser basename blkid brctl bzcat cal cat chgrp chmod 
chown chroot chvt cksum clear cmp comm count cp cpio cut date dd deallocvt 
delgroup df dhcp dirname dmesg dos2unix du echo egrep eject env expand 
expr false fdisk fgrep find free freeramdisk fstype ftpget ftpput 
getty grep groupadd groupdel groups gzip halt head help hostname id 
ifconfig init insmod kill killall klogd link ln logger login logname 
losetup ls lsmod lspci lsusb md5sum mkdir mkfifo mknod mkpasswd mkswap 
mktemp modinfo modprobe more mount mountpoint mv nc netcat netstat 
nice nl nohup od openvt passwd paste patch pgrep pidof pkill pmap 
poweroff printenv ps pwd readlink realpath reboot renice rev rm rmdir 
rmmod route seq setsid sha1sum sleep sort split stat su swapoff swapon 
switch_root sync syslogd tac tail taskset tee telnet test time timeout 
touch true truncate tty umount uname uniq unix2dos unlink unshare 
uptime useradd usleep uudecode uuencode vconfig vmstat watch wc which 
whoami xargs xfs_freeze xzcat yes
Sources are in the usual place; I built this with

Code: Select all

make CC=/opt/musl/bin/musl-gcc CFLAGS="-Os -static" KCONFIG_BUILD=CONFIG.build; sstrip toybox; ./toybox gzip -gc <toybox >toybox.gz
(note that the above requires sstrip from http://www.muppetlabs.com/~breadbox/sof ... ckers.html)
Attachments
CONFIG.build.gz
gzipped .config
(1.22 KiB) Downloaded 306 times

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#19 Post by Ibidem »

Size optimization, from smallest to largest:
gcc-3.4, -Os -fomit-frame-pointer 390k
gcc-4.2, -Os -fomit-frame-pointer 393k
gcc-4.2, -Os 394k (a few bytes larger)
gcc-3.4, -Os 394k (largest of the four)

Some small fixes have taken place, and xfs_freeze is now fsfreeze.
TOybox 0.4.8 is on the way.

Post Reply