Build scripts for small debian live images (similar to woof)

A home for all kinds of Puppy related projects
Post Reply
Message
Author
emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

Build scripts for small debian live images (similar to woof)

#1 Post by emil »

Update 15.02: Hi,

I used the scripts to build a usable "office image"
desktop is an icewm/rox combo with XFE tools.
It includes Libre Office, Icewease/Icedove+owl, Media player, gimp etc..
Wireless, Wicd and many drivers are included

get scripts with:

Code: Select all

git clone https://github.com/emilwidmann/Debian-light-builder.git
or classic download:
Download build scripts: CAT-scripts-0.41.tar.gz (6,7 MB)
Download ISO: CAT-0.41.iso (412 MB)
md5sum

Changes:
+starts X as 'root' not as 'user'
+ mounts home medium rw
+ synaptic
+keyboard setter
+keep command (create squashfs from apt-get)
+grub2
+rsync, openssh-client
+CD/DVD Tools
+ hfs, reiser, fuse-posixovl, ntfs-3g filesystems
+ Libreoffice, cups, Iceweasel, -dove, evince
+ Smplayer, audacious, gimp. mtpaint, mirage
+ volumeicon
+ some other customisations and little improvements

Interface is still very basic with no frills, I use mostly rox filer for opening configuration directory, showing available apps or also to open the /media folder for attached drives.

The build scripts are split into modules even more, the main config script is named now CAT. I still use live-build version 3 with persistence file with included persistence.conf.
A size of 4 GB for persistence file works for me until now. System on a dual core with 1 GB is still very responsive. Will test this system during the next months
cheers 8)
emil

------------------------------------
Original Post 05.02.14
Hi,

there is some interest in small and light debian live systems which have a puppy look and feel (e.g. Light-Debian-Core-Live-CD by saintless and other forum members).

Light-Debian-Core-Live-CD is produced by making a minimal debian live installation and then by a step by step remaster of the iso file. Thierry Monteil, a mathematican from France, who maintains a project "sage debian live" has written a build system based on the live-build tools (you can download the original build scripts from his page).

build scripts md5sum : 60534785efc54b457f8a1998124a6665

Using his scripts I wrote a module which produces a 130 MB iso.
download md5sum c1ffbba66cf6c45ca82da58419ca07c7 CAT-0.2.iso
Image

it includes icewm, rox, xfe tools, iceweasel browser, wicd network manager, wireless network tools and free firmware drivers.

How does it work:

1) install Debian-Light-Wheezy or the CAT-0.2.iso
installation is similar to puppy, you can make frugal installations with the files vmlinuz, initrd.gz, filesystem.squashfs in a directory /live and use a grub entry like

Code: Select all

title CAT
  root (hd0,5)
  kernel /live/vmlinuz boot=live config persistence quickreboot noprompt autologin nonetworking
  initrd /live/initrd.img 
IMPORTANT: The scripts use version 3.x of the debian live tools, so we have to use "persistence" boot parameter, not "persistent" as it was used in older versions of debian live-boot
We have to run debian, because for the build scripts the debian live tools are necessary. In principle you could install those in puppy and use the scripts, but I have not tried this.

2) Boot Debian/CAT, download the build scripts to a suitable location
3)

Code: Select all

tar xvf Debian-light-builder.tar.gz
4) cd to the Debian-light-builder directoy
5) install the necessary tools with

Code: Select all

sudo ./build.sh install
6) start the build of the iso with

Code: Select all

sudo ./build.sh icewm
the build will apt-get packages and will take some minutes.

The build system is organised into modules and corresponding config files. Thanks to Thierry Monteil the code is structured clearly and is rather easy to navigate.

With the build system one can automate the creation of the Debian live systems which are similar to classic puppy system in size and functionality, To share work it is not necessary to exchange full isos, but only the modules for creations (a few MB), It would be possible to setup the build scripts on a git, svn, mercurial repo for cooperative developement. The strong points at the moment are a working apt-get package managememt and e.g. multi user capability. Regarding small apps, MIME integration and general user friendliness, the project could learn much from the puppy linux world.

Some closing remarks:
1) I propose the name CAT for the build scripts of small debian live systems for historical reasons as a tribute to forum member sickgut. If the best case scenario happens and the project is improved with puppy apps and ideas, then the product could be a CATDOG.iso
2) In a follow up post I will give more details on some technical aspects.
3) There might be discussion if this is a puppy linux project and if it is the right place to post on the murga forum. Please don't discuss this or flame on this thread, If there is a problem please moderators move or delete this thread as you feel appropriate. Forum members who want to flame me can pm me or even send me an email (emil.widmann!AT!gmail.com), I might be slow to respond though.
My oppinion is, that this is a puppy linux related project, because it grew from the puppy community and seeks to incorporate and learn from puppy to get better.
4) Although I cannot expect that saintless and the team who works on the "Light-Debian-Core" project will change their present remaster-workflow I think the build scripts have a lot of advantages. In the beginning progress might be slower and more complicated, but in the end it will be worth to have a coherent system which can build all sort of variants (like pae/non pae or 64 bit)
5) my original goal was to develop a new small sage live CD, but I have not much time and I couldn't quite finish it, so I just share what I have.

all the best
emil
Last edited by emil on Sun 02 Mar 2014, 17:08, edited 14 times in total.

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

some differences and something similar

#2 Post by emil »

Multi User
At the moment the system is set up to autologin as user. To perform admin tasks you can either use sudo, start a root terminal (desktop icon) or you become root with "sudo su". Sometimes it is helpful to start a rox filer or xfe manager from a root shell. Excluding the "user-setup" and "nodm" packages from the build scripts would probably allow a root x session like puppy. Edit: To have a root autologin just modify the "nodm" hook in icewm or CAT module, either remove whole line or change 'user' to 'root'
Edit: Version CAT-041 will autologin as "root" like in puppy.

Package Management:
the deb files of the debian repositories are used, pet files do not work out of the box. Since no indices are included in the iso, it is necessary to do

Code: Select all

sudo apt-get update
before you can do e.g.

Code: Select all

sudo apt-get install gimp vlc chromium
files:
Debian live has a similar structure than puppy. The core files of a frugal install are vmlinuz, initrd.img and filesystem.squashfs. It is important that they have to be placed in a folder /live on the medium (CD, USB, HD partition) to be found. Also you shouldn't have a 2nd /live folder somewhere else on your systen (I rename those folders I don't need to /live_noload)
Edit: using boot parameters in your grub configuration it is possible to change paths and filenames, e.g. "live-media-path=PATH" you can specify different path than /live.
See here for a list of boot paramters.

boot codes:
example boot code for frugal install:
see earlier post

sfs modules:
It is possible to use sfs files, however they have the file ending "squashfs". To autoload different squashfs modules the textfile "filesystem.module" has to be created in the /live directory.

it contains an ordered list, example:

Code: Select all

filesystem.squashfs
big_standalone_package1.squashfs
test.squashfs
boot and filesystem scheme:
The boot process of the debian live system can be studied in the folder /lib/live. There is also the /mnt folder which contains the rootfs, the overlay and the mounted medium

savefile / persistence
similar to puppy there is the possibility of a savefile or a save partition (eg. hd partition or usb). The default name for this file in Debian live is "persistence" (in former versions it was named "live-rw"). This file should be placed on a different partition or medium (like USB stick) than the /live folder, because the live medium is mounted read only under normal operation.

The creation of such a persistence file is like in puppy:
dd if=/dev/null of=persistence bs=1G seek=1 # for a 1GB sized image file
/sbin/mkfs.ext2 -F persistence

however unlike puppy you will also need a configuration file inside.
Edit 17.02:
I use the following persistence.conf file:

Code: Select all

# persistence configuration file
# store changes in directory in union (only differences)
# This should offer a puppy linux like persistence scheme 

/ union

to copy it to the persistence file you have to do something like

Code: Select all

mkdir mpt
mount persistence mpt
cp persistence.conf mpt
umount persistence
On a first glance the whole system seems inflexible and complicated compared to the simple puppy setup. But in generall there are a lot of options to stear the behaviour of persistence, the whole information can be found in the debian live tools manuals. For example we have excluded the /home folder (user directory) from persistence in the persistence.conf file. The reason is that we have included a script in the build of the iso file, which remounts the live medium as rw (during the boot process) and then binds the /home directory directly to the /home directory on the medium. This serves as example for a different type of persistence

In generall debian packages are bigger than puppy packages, so you need larger persistence files compared to puppy. However it is possible to apply the scripts for purging docs, locals, manuals we used for the iso creation to save some space.
Last edited by emil on Sun 02 Mar 2014, 16:27, edited 6 times in total.

User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

#3 Post by saintless »

Thank you, Emil.
I like CAT name and the reason for it.
Learning much more new things from your first two posts here.
I will test, share and help with pleasure.

Edit: Can you. please check out the link. I can't download Cat-O.1:
download md5sum 26c896620cb2748a20df704264e4eef8 CAT-0.1.iso

Toni

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

#4 Post by emil »

saintless, there was a serious bug in the uploaded CAT.iso I have to fix it then I can reupload it.

User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

#5 Post by saintless »

Thank you, Emil.
I can use the script from Light-Wheezy then.
Take your time with Cat iso.

Toni

mcewanw
Posts: 3169
Joined: Thu 16 Aug 2007, 10:48
Contact:

#6 Post by mcewanw »

Looks very interesting emil. The more ideas and projects like this the better I feel. I'll give it a try sometime. Please keep working on this.

William.
github mcewanw

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

#7 Post by emil »

Ok, uploading again, CAT-0.1.iso should be there in 30 minutes.

Some practical advice: on low ram systems and with small save files it can be useful to move the /var/cache directory to a drive and symlink it.

start 2 rox filer in a root terminal, then move the cache directory to e.g. /media/sda3. then create a link in /var
So all the packages you download will not clobber your system.

For testing isos I used virtualbox. For installing virtualbox (apt-get install virtualbox) it is useful to temporarily set apt recommends and apt suggests to true, otherways it will install the machine without the gui interface.

For setup of the VM, just add the iso file in the build directory to the cd drive in the virtual machine and boot up.

cheers
emil

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

#8 Post by emil »

Ok, the CAT-1.0.iso is up again. The build scripts have also JWM and openbox modules, but those are pretty raw and lack polish and some purging.

User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

#9 Post by saintless »

Thank you, Emil.
Downloading now. I will try your advice to symlink /var/cache.
Toni

User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

#10 Post by mavrothal »

Tried CAT-0.1 (both the original and re-uploaded versions) and in all cases end up with a blank screen and an "X" mouse.
Tried in in VB, VMware and Parallels, VMs. All the same.
Below are some errors spitted during the boot sequence. Might point to the problem.
Attachments
Screen shot 1.png
(108.65 KiB) Downloaded 2741 times
Screen shot 2.png
(131.16 KiB) Downloaded 2712 times
Screen shot 3.png
(23.21 KiB) Downloaded 2779 times
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==

User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

#11 Post by saintless »

Hi, Emil.

Just to make it clear for all. This project is not about Cat-0.1.iso but for building script which produces Cat-0.1.iso. Am I right?

Works for me in Gateway E-1400 (650 Mhz processor + 256 Mb RAM - no SWAP).
Installed mtpaint and htop. Looks light and fast.
Emil, is the initrd.img edited in some way from the original? I get some harmless warnings I don't have with the same kernel in Light-Wheezy.
Much to test and learn here :)

BTW - Debian Wheezy Live full persistence is exactly with only one line / union in persistence.conf but it has a bug. If you use it this way the system overlays the modules backward. I mean it overlays 01.squashfs on top of 02.squashfs
This was the main reason to choose downgrade default kernel for Light-Wheezy + it brings back /live/cow
The example you gave for full persistence will work fine. I tested similar format for persistence.conf as a solution to the full persistence / union bug and it worked.
Attachments
cat.jpg
(68.96 KiB) Downloaded 2779 times

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

#12 Post by emil »

Hi all thanks for testing!
I am a stressed guy in RL, so I had also made a mistake yesterday with the build scripts, upload was broken somehow. I reuploaded and gave md5sum - they are at 5 MB.

mavrothal, that looks bad, I have no idea. The only possibility I see that I have purged out some special drivers drivers you need. package user-setup etc should be there. On the other hand it always worked for me in Virtualbox with normal setup (486 version). Please don't use the original CAT iso, it has nasty bug with the umounta script which might result in wiped disks - delete it

Saintless thanks for testing - as you see some solutions are rather basic, and your version is much more refined in the meantime, but it is a start.
regards
emil

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

#13 Post by emil »

Just to make it clear for all. This project is not about Cat-0.1.iso but for building script which produces Cat-0.1.iso. Am I right?
Yes to be clear, the project are the build scripts and how to develop them further to give the raw debian live more puppy power (small size, apps, friendliness). The CAT iso I gave is an example from the present stage, a "proof of concept" so to speak.

Further developement would not primarily be a remaster of the CAT iso, but by modyfinig the build scripts (e.g. the icewm module).

Of course remasters of the iso are fine to try out solutions and experiment, but the changes should be brought in a form where they are scripted in the build system. So if somebody has written his own module or has scripted how to handle MIME types, share it please.

cheers
Emil

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

#14 Post by emil »

is the initrd.img edited in some way from the original?
No I didn't alter the initrd.img. I don't know for sure, but I guess that the initrd.img is also created by the debian live builder (not copied from existing iso ). If there are warnings then we have to look if it is a bug or if this is ok.

User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

#15 Post by saintless »

I will check out the warnings and compare the boot with debian live cd initrd for the same kernel. I will write back when I have information.

Toni

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

#16 Post by emil »

mavrothal,
maybe your problem is from the inclusion of the /home persistence module in my latest builds, because it tries to mount the live medium rw during boot and this is of course not possible with a CD.

Maybe you can try a frugal hd install with the files from the iso. Otherways I will rebuild the iso without direct /home persistence module.
cheers
Emil

mcewanw
Posts: 3169
Joined: Thu 16 Aug 2007, 10:48
Contact:

#17 Post by mcewanw »

Just downloading before heading for bed. Hope to try it out tomorrow, thanks, William
github mcewanw

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

#18 Post by emil »

I just uploaded CAT-0.2.iso.

http://boxen.math.washington.edu/home/e ... AT-0.2.iso

This is built without the "home" persistence module. So the live medium will be mounted read only.

If you have a frugal install, you can do the following:

Code: Select all

sudo mount -o remount, rw /lib/live/mount/medium
to make the home directory persistent in the save file, you have to uncomment the
/home union

entry in persistence.conf file.
mavrothal, it would be nice if you could test this iso to see if this was the reason.
thank you
emil

User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

#19 Post by mavrothal »

emil,
tested CAT-0.2.iso in the VM.
The (not really) bad news is that the errors shown previously are still there.
The good news is that I'm posting from it :D
I guess removing the "home" persistence module did the trick.

Now, can someone do an dpkg/apt-get pet or sfs for puppy? :wink:
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==

big_bass
Posts: 1740
Joined: Mon 13 Aug 2007, 12:21

#20 Post by big_bass »

There have been several lost names that new forum users (dont remember) those that have tried to make puppy more linux like and follow open source standards

I see that this is a strong desire by several out of the box thinkers

When Dr. Ian Malcolm heard that by design there would be no
life produced (without the controlling parties consent) he reflected a moment and said
Dr. Ian Malcolm: No, I'm, I'm simply saying that life, uh... finds a way.
meaning if others have failed to get the point across this doesnt mean they failed
they have discovered that there are better ways to enjoy linux
and life finds a way

Joe

Post Reply