Build scripts for small debian live images (similar to woof)

[konaexpress]

konaexpress, good point.
The CAT-0.2.iso is still on, it is reduced to the basics.
Download

for puppy user it might be a hassle that X is startet as user, but starting a root shell and 1 or two root Rox filers will be enough for puppy root feeling...
The scripts have the KITTY module, which should be similar to CAT-0.2.
sudo su
emil

Thanks, downloading it now.

John

[konaexpress]

Let me know if you want the scripts and I will go look in my code and see what they are again.

Thanks, John.
I like to see this scripts.
We still look for proper solution to make option for user account to reboot and shutdown and this is involved with using separate script for the start menu buttons.
Maybe your example will give me the right idea to use in Light-Wheezy.
I guess it will be best to post it here when you have time:
http://murga-linux.com/puppy/viewtopic.php?t=90660

Toni

Sure, let me go look it up real quick.

John

EDIT: I posted the code in the other thread for you

[emil]

I just wanted to add that while running the build scripts for full CAT-041, screen output in the console will stop. This happens while updating a SSL certificate from the Turkish government. Probably this happens because Xterm is running without access to the right UTF8 locals. However the script runs fine in the background and will eventually finish. You can watch the progress of the build by reloading the build.log in the build folder and by watching the filesystem.squashfs grow in the build/chroot folder.

[emil]

Hi saintless,

I don't know if you prefer a separate scripts for reboot and shutdown, but you can have them easy as start menu buttons from /usr/share/icewm/preferences:

Code: Select all

#  Command to shutdown the system
ShutdownCommand="/sbin/poweroff"

#  Command to reboot the system
RebootCommand="/sbin/reboot"

Thank you very much for pointing me to the icewm preferences. I have to admit I didn't study those in detail, as well as the menu configuration files (The default Debian Menu is almost unusable borked IMHO)

There are two possibilities to change those. The first is to provide a complete new file (in the includes.chroot directory of the icewm module, e.g. /usr/share/icewm/new-preferences ). Then one would have to write a hook which deletes the old preferences and replaces it with the new file.

basically a hook like

Code: Select all

rm /usr/share/icewm/preferences
mv /usr/share/icewm/new_preferences /usr/share/icewm/preferences

If it is just to replace some configuration settings, then one could use sed commands, as for example in the nodm hook (in the CAT module).
for the icewm preferences this would be

Code: Select all

sed -i 's;^# ShutdownCommand*;ShutdownCommand="sudo /sbin/poweroff";g' /usr/share/icewm/preferences
sed -i 's;^# RebootCommand*;RebootCommand="sudo /sbin/reboot";g' /usr/share/icewm/preferences

This commands would automate those configuration changes.
cheers
emil

[saintless]

Thank you for the tips and for pointing us to this method, Emil.
I experimented a bit with the build script last night and the structure of the process start to become much clear to me.
It is really great work and very nice Tierry Monteil shared this building method with the linux community.

Toni

[emil]

saintless:

I think for now this puppy deb packages from Fred will work without problems on standard debian wheezy and squeeze for multiuser:
http://www.smokey01.com/saintless/Fredx ... wheezy.deb
http://www.smokey01.com/saintless/Fredx ... -0.8.3.deb
http://www.smokey01.com/saintless/Fredx ... utdown.deb
http://www.smokey01.com/saintless/Fredx181/pfind.deb
http://www.smokey01.com/saintless/Fredx ... %29.tar.gz

Toni thanks for the links to the above deb packages. I really wanted to include them properly in my new CAT builds but ran into several problems.

In the live-build manuals there is a section 8.3 which describes the use of
"Installing modified or third-party packages"

it describes two methodes to achieve this:
1) copy those packages to /config/packages.chroot
2) using a local apt repository

the first method would be the easiest, however it didn't work for me as expected.
As usual I placed them in a packages.chroot directory inside the module and had a

Code: Select all

cp -r ${BASE_DIR}/modules/icewm/packages.chroot" ${BUILD_DIR}/config"

in the icewm-config.sh script. However the packages didn't appear in the build, so probably I made an error somewhere.

The method with the local apt repository worked great, but not for the scripts, because during the build process in the chroot lb_build couldn't access it. However the setup of the local repository was without http server which could make the repository visible on the network. I could imagine such a setup to work, so it would be possible to just add those packagenames with the "addpackages" comand in the module configuration.
This would be the best solution for builds with lots of selfmade and modified deb packages.

As a last resort I unpacked the debs with dpkg -x and added the files to the
includes.chroot directory. This worked perfectly well, I had pupvolume-monitor and desktop-drive-icons working in the build.
However at the same time I also added the volumeicon-alsa and got this strange window appearing on the desktop. Alas, I didn't follow the Debian-light-Wheezy thread close enough and missed that you experienced the same problem, so I suspected the desktop-drive icons/pupvolume monitor to be the cause and removed them, before I finally found the volumeicon and the sleep in the startup file to solve this. I didn't reinclude the debs then later, I ran out of time and motivation. It work's for me the way it is and the moment I have no 3rd party binaries in the build, so it is pure debian.

My advice for inclusion of 3rd party debs is extract them and include the files directly over includes.chroot.

packages.chroot might work or not, local apt-repository is great but will need some tinkering to make it work.
cheers
emil

[saintless]

My advice for inclusion of 3rd party debs is extract them and include the files directly over includes.chroot.

packages.chroot might work or not, local apt-repository is great but will need some tinkering to make it work.

Thank you, Emil.
This will save me a lot of work to find out how to include self-made packages.

Toni

[emil]

This will save me a lot of work to find out how to include self-made packages.

welcome.

One other bit of information I want to share is about the boot folder. It is still included in the CAT build. I tried to remove it and the symlinks to vmlinuz and initrd.img in / using a hook in the CAT module but this failed. Live build obviously needs this folder later in the build process. I can just speculate, but maybe when building a new initrd.img from ground up, I haven't remembered the error messages in detail.

This will add some MB to the final image which are not entirely necessary. The folder and links could probably removed by hand (unsquashing / resquashing) the filesystem. If this job should be done by the scripts then one could add a posttreatment of the image, as already present in the code of build.sh for hdd images (by Thierry Monteil for sagedebianlive - the code after ### Post-processing ).

If such a possibilty is implemented I suggest to trigger it with an variable like REMOVE_BOOT_FOLDER=true.

I decided to keep it as it is and stay pure debian live, I also don't understand all implications of removing this folder. Well the real reason might be that I would have needed to much time to implement it properly at my skill level.

kind regards
Emil

[saintless]

I can just speculate, but maybe when building a new initrd.img from ground up, I haven't remembered the error messages in detail.

Hi, Emil.
I saw your commented explain in red inside a script about the boot folder in some file. Don,t remember which one was at the moment
Yes, it is needed for building initrd.img for the kernel. I'm not sure if it is not enough just to keep empty /boot folder for this. Need testing.
Removing /boot folder content removes 14 Mb compressed data from the final iso.

Toni

[emil]

Another speculation: maybe the Boot Folder is needed for the original debian installer.
In normal frugal installations it seems redundant, so it may be worthwile to implemet such code to auto remove the BOOT folder.

[emil]

I come to agree that 3.0 persistence scheme is badly broken. At least it has some serious design flaws.

I used the persistence.conf as I described, with all subfolders mentioned. But this doesn't work, because /lib is explicitly excluded (The idea probably was that it contains /lib/live, which should be protected). So installing additional packages (libraries) persistently doesn't work.
The workaround in the manual is to declare / union, but as saintless and others found this breaks squashfs overlay.

So best would be, as alread found out by saintless long time ago, to build with live tools 2.x. At least until vs 3 is fixed.
cheers
emil

Edit 17.02: After repeated testing I found that

/ union

in persistence.conf works for me.

[saintless]

...because /lib is explicitly excluded
...So installing additional packages (libraries) persistently doesn't work.
...to build with live tools 2.x. At least until vs 3 is fixed.

Hi, Emil.
If you prefer to use original live-boot-initram-fs-tools v.3 here are 3 ways to have full persistence working:
1. Use always only one squashfs module with persistence.conf content / union. It works without problems this way.
2. If you use more than one module, lets say
01-first.squashfs
02-second.squashfs
use the same persistence.conf with content / union, but rename the modules this way:
01-02-second.squashfs
02-01-first.squashfs
It will work this way also.
03. Use this content of persistence.conf (note lib is missing there but /link is symlink to /lib) with only one module and regular order for more than one squashfs modules:
01-first.squashfs
02-second.squashfs

We have a dirty fix now without downgrading live-boot, but I start to like your idea more and more. Having copy on write /live/cow will make the things simple and well known for us. I will try to follow your steps and maybe we will get what we want faster.

In the worst case here is the dirty fix for wrong persistence overlay problem. In the base system we need to simlink /lib to /link for example and to replace the content of persistence.conf from / union to this:

Code: Select all

/bin union
#/dev union
/etc union
/home union
/link union
/media union
/mnt union
/opt union
/portable union
#/proc union
/root union
#/run union
/sbin union
/scripts union
/selinux union
/srv union
#/sys union
/tmp union
/usr union
/var union

# means ignore this folders but I think we have full persistence without them. /link is simlink to /lib so any changes in /lib will be saved.

This is our last solution if we can't find better one.

Lets not forget the advantage of using persistence.conf save file - it can save only the folders we like. And it will work well this way. This is something we can not do with live-boot v.2 initrd.img
This way we keep main module of Light-Wheezy up to date with latest live-boot as regular Wheezy and the original initrd.img download option from the site (in case someone prefer to use it with persistence.conf instead live-rw).

Toni

[emil]

Hi Toni,

thank you very much for your profund answer and suggestion. I will do so as you say and try to keep live boot 3 the default. Do you have any objection to name /link as /lib_link, just to make more obvious whats happening?

also I would have to add a hook in the CAT module which would create that symlink.

Let me report that as a reaction I did a KITTY build with setting in config DISTRO="squeeze" (with icewm, iceweasel)

it built to 116 MB. removing the /boot folder brought it down to 103 MB. But it wasn't able to boot afterwards, it gave an error in the initrfs. Do you also have advice from your experience how to handle this? Will it work with an empty boot folder?

kind regards
emil

[anikin]

Hi all,

Thank you, emil and saintless for your work, I learn a lot from your projects and eagerly follow your discussion. So it's possible to have COW work - changes are saved. The only problem is, that I do not want to use persistence or a save file. I actually hoped that COW will allow me to capture a snapshot of the running system (like ro1, ro2 in Puppy). Is that possible?

Thank you in advance.

[saintless]

Hi, Emil.
Glad to help

Do you have any objection to name /link as /lib_link, just to make more obvious whats happening?

name it as you wish. I named it link only for the test.
Note I haven't test this proper since we moved to downgraded kernel. It will be best if you test it for a while before make the hook for the link.

Let me report that as a reaction I did a KITTY build with setting in config DISTRO="squeeze" (with icewm, iceweasel)

Thank you, it will give me information how to edit proper the build script and to build low-ram system.
I tried to make some radical changes and to exclude the wicd wireless download but it didn't work as I expected. The script did not create iso image at the end. I will learn how to edit proper the script in time

removing the /boot folder brought it down to 103 MB. But it wasn't able to boot afterwards, it gave an error in the initrfs. Do you also have advice from your experience how to handle this? Will it work with an empty boot folder?

Yes, keeping empty /boot folder should fix this, but you also need to use make new initrd.img command instead of updating/replacing existing initrd.img. Something like this for example:

Code: Select all

mkinitramfs -o initrd-3.2.0-0.bpo.4-rt-pae.img 3.2.0-0.bpo.4-rt-pae

But maybe the error you got is different, because initramfs will not create initrd.img if there is no /boot foder. Is initrd.img missing?

Toni

[saintless]

Hi, Anikin.

I actually hoped that COW will allow me to capture a snapshot of the running system (like ro1, ro2 in Puppy). Is that possible?

Yes, it is easy. Just use for example:

Code: Select all

mksquashfs /live/cow /live/image/live/02-my-snapshot.squashfs

Fred made a special script named remastercow for light and porteus wheezy. It is included in /opt/bin and does also proper cleaning from GUI. It needs yad installed to work on Cat from GUI but I think it will be easy to make CLI script to do it or to install yad and test it. Download link in archive (look at remastercow one):
http://www.smokey01.com/saintless/Fredx ... ter.tar.gz
I'm sure Emil also wrote about similar script keep.sh but haven't test it yet.

Toni

[konaexpress]

Instead of yad, just use BaCon. Way less to install.

John

[anikin]

Hi emil,

I updated the first post
Download build scripts: CAT-scripts-0.41.tar.gz (6,7 MB)
hope you like it...

Using your CAT-scripts-0.41, I built a SID CAT.
I removed some items from the scripts, made the build JWM based, added geany ... and now posting this from my own, self-crafted OS.

Code: Select all

user@debian:~$ sudo -i
root@debian:~# uname -a
Linux debian 3.12-1-686-pae #1 SMP Debian 3.12.9-1 (2014-02-01) i686 GNU/Linux
root@debian:~#

Not everything worked out as expected, namely, nodm doesn't work, which I tend to blame on the hook, geany and some other added items didn't install. However, for a first go, the end result is excellent.

Thank you.

[emil]

I built a SID CAT

Wow, congrats!
could you describe how nodm failed? From your post I see you are loged in as user - Do you have the old version of the hook, where the autologin is set to user in the /etc/default/nodm file?

I also wonder about packages failing to install. If they are in the repos of the sid distribution they should be found...

I would be very curious about any new modifications and also about testing persistence with the modifications Toni suggested or setting full persistence with

Code: Select all

/ union

.
in persistence.conf

cheers
emil

[emil]

In the meantime I build a squeeze CAT and then tried to run the scripts from the squeeze distribution, which will use the live build 2.xx version.

Although the scripts ran for a great part they failed somewhere in the middle, so it will not be possible to use them OOB to create CAT versions with the old persistence mechanism (live-rw, home-rw). But it was worth a try...