Woof-based Puppy builders wanted

News, happenings
Message
Author
User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

#41 Post by saintless »

jamesbond wrote:IIRC the non-updatable package list is in apt-get, not in dpkg --- dpkg allows you to do *anything*.
apt-get, aptitude, synaptic all use dpkg and actually the hard work is always dpkg responsibility. Using dpkg -i packagename from terminal will add information in /var/lib/dpkg files and will check out for missing dependencies, but it will not sort out dependencies. dpkg will show the missing dependencies and will leave the package not configured till the dependencies are manually installed with dpkg or apt-get -f install is executed. Or you should download all dependencies in a folder and use dpkg -i -R to install them all.
apt-get, synaptic, aptitude will install/uninstall all dependencies automatically by only one package name given in the command, but the actual work is for dpkg again. I might missing something since I'm not dpkg specialist but this is from my experience.

I think leaving to the user the responsibility to keep dpkg database safe is a big risk. At least pet2deb converter should come with red color warning use this at your own risk. Breaking dpkg database makes the main advantage useless. This risk should be kept at minimum.

Toni

User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

#42 Post by saintless »

Stemsee,

if some files are missing in this container forget to boot it with puppy kernel. It will look for /etc/rc.d files that are not included if you use Ubuntu packages to build the container. You can boot it with Debian kernel and initrd.img but Puppy is much different.

Toni

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#43 Post by Iguleder »

jamesbond wrote:Come on, how come it's only me, mav, saintless and stemsee? Others?
I'm in, as long as the end product allows me to build a 64-bit, all-static, GTK1 Puppy with tinyxserver, Linux-libre and my own package manager. :wink:
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#44 Post by jamesbond »

mavrothal wrote:
jamesbond wrote:I guess going forward, we need to have names for these (right now, all these independent components are all called "Woof-CE" - which is super confusing.
a) The name of the repository that holds puppy-builder code, and puppy core code ("Woof repository").
b) Puppy builder code ("Woof build system").
c) Puppy core code (rootfs-skeleton, initrd-skeleton).

Please decide on what you want to call these three, but please don't call all of them them "Woof-CE". The script I'm talking about touches on point b, the puppy builder code (with minimal modification to puppy core code). Toni's concern are mainly for Puppy core (point c).
There is a major headache in this. Puppy is not standard compliant, specially when it comes to menus and file location that's why it has all these package-templates in woog-code to rework standard packages.

I think the way to go is get a woof3 branch going where, we keep woof-code/{boot,rootfs-skeleton,rootfs-packages} and maybe kernel-skeleton
Also keep kernel-kit.
Then make things in rootfs-skeleton standard-compliant and rewrite the build scripts/tweaks.
It sound much easier than it actually is :twisted:
Nah, I'm not thinking that far. I'm just thinking about terminology, because today "Woof-CE" is used to name these 3 distinct components. My suggestion is this:
a) the repository - keep the name "Woof-CE"
b) the builder - use the name Woof2, next generation perhaps Woof3, Woof4, Woof-next, etc
c) the puppy core files - let's called it "puppy-base".
Both b) and c) are kept in a). As for the structure, we'll go with what works.

stemsee - LXC is just like qemu, except that you share the kernel with the host os. I don't see the relevance of LXC with puppy building, except perhaps for testing (I use qemu for testing). If you want to see how it works in puppy-flavoured distro, get Fatdog64 :) And saintless is correct - with what we have today, Puppy will *not* boot on Debian kernel; it needs some kernel config which isn't in Debian kernels. (Ubuntu kernel, may be - I have seen some Ubuntu kernel comes with aufs - but definitely definitely not Debian kernel).
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#45 Post by jamesbond »

Iguleder wrote:
jamesbond wrote:Come on, how come it's only me, mav, saintless and stemsee? Others?
I'm in, as long as the end product allows me to build a 64-bit, all-static, GTK1 Puppy with tinyxserver, Linux-libre and my own package manager. :wink:
:lol: What I've done is for Ubuntu, so the only thing that meets your criteria is "64-bit" :)

Looking far ahead, I'm thinking that perhaps we should have different builders for different source distros:
- one for Ubuntu/Debian (they are close enough to be considered as one distro, only perhaps the pkglist is different)
- one for RPM-based distros (I don't know yet whether we can combine them - is fedora/mageia similar enough?)
- one for Slackware-based distros
- and of course, the ultimate one: one for packdude-based Puppy with GTK1 and all-static binaries.

And perhaps unified by one front-end which does nothing other than asking questions and then execute the right builders. This way, modification to ubuntu builders will not accidentally break rpm builders, and vice versa. Maintenance is also easier, people only need to maintain the source distros that you're interested in.

If you look at the source of debootstrap, you'll see that it has different "modules" for different versions of Debians (the "debootstrap" itself is basically just a front-end to these modules), so I think the idea has some precedence.

For packdude, all I need from packdude is that:
a) it is compilable on a bog standard OS (doesn't require asm libs that only runs in 64-bit with AVX, for example), and
b) that it can install packages in a specified chroot directory, including its administration files, and correctly recorded installation path relative to chroot dir,
c) and it can install from a local repo.
The last one is not necessarily needed, but it makes stuff easier for multiple builds (I don't have to re-download packages every time I attempt to build puppy).

I think packdude has a) and b), we only need c) + the ability to download packdude's repo package list and parse them and download individual packages without installing (either with wget, or with packdude itself - your choice).
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#46 Post by Iguleder »

jamesbond wrote:a) it is compilable on a bog standard OS (doesn't require asm libs that only runs in 64-bit with AVX, for example), and
Clean C99. Check.
jamesbond wrote:b) that it can install packages in a specified chroot directory, including its administration files, and correctly recorded installation path relative to chroot dir,
packdude -p. Check.
jamesbond wrote:c) and it can install from a local repo.
The last one is not necessarily needed, but it makes stuff easier for multiple builds (I don't have to re-download packages every time I attempt to build puppy).
packdude -u file://path/to/repo. Check.

EDIT: this really troubles me. With this design, what makes Puppy different from a minimal Debian installation with JWM and ROX-Filer? And what about package upgrades - if we use our own kernel instead of the Debian one, what happens when there's a kernel package in Debian?
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

#47 Post by saintless »

Iguleder wrote:With this design, what makes Puppy different from a minimal Debian installation with JWM and ROX-Filer?
Different mount points created inside /media for Debian and inside /mnt for Puppy/
/etc/rc.local in Debian and /etc/rc.d/rc.local in Puppy (there is no /etc/rc.d in Debian).
/root/.xinitrc in Puppy is /root/.xsession in Debian
I'm sure other small changes like the above will be found.
For example mount programs downloaded from Debian will look for /media which is not included in Puppy. AptOnCD from Debian will fail if there is no /media/cdrom0 link to /media/cdrom directory...
Debian uses its own menu system that does not provide desktop file for all packages.
Nothing much at first look but it makes impossible to run many deb packages on Puppy without problems.
And what about package upgrades - if we use our own kernel instead of the Debian one, what happens when there's a kernel package in Debian?
Upgrading the kernel can be refused easy with configuration file.
Some packages has to be provided for Puppy kernel instead downloaded from Debian. Like linux-headers, VirtualBox...
Nothing will happen since the kernel is only available for download and install option. It will not work if it is installed with Puppy inird.gz (but it will work after download and install with auto-created after installing initrd.img giving Debian boot and save file options. Then we can switch from Puppy to Debian changing vmlinuz and initrd in the boot code and the save file name from pup_save.2fs to persistence (or live-rw).

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#48 Post by jamesbond »

Iguleder wrote:EDIT: this really troubles me. With this design, what makes Puppy different from a minimal Debian installation with JWM and ROX-Filer?
Indeed. What makes puppy, a puppy? Make makes Mint different from Ubuntu? What makes Ubuntu different from Debian? Surely not only the wallpaper and gtk themes?

But for practical purposes:
1. Today's upup: built from ubuntu + pet, full ubuntu repos, but not all working, people complain.
2. Today's t2pup (racy, wary): built from pet, small repos, people complain.

What I'm seeing:
3a. DebianDog: pure debian, full debian repos, all working, people happy.
3b. This effort: pure ubuntu, full ubuntu repos, (hopefully) less broken packages than original upup, people happy (at least happier).

The only reason (I can see) to build from other distro's binaries is so that you can use their repos. If it proves otherwise (because those packages becomes broken when installed), then there is no point of depending on that parent distro anymore.

So - either we make puppy "more compatible" with the parent distros so we can use their repos better (DebianDog and my script is two different way of attacking the same problem); or totally drop ubuntu/debian/slackware and start building pure-breed puppies (T2, LFS or otherwise) - and we're back at square-one of "puppy's lack of packages."

As for upgrades - we'll tackle it when we get there :wink: As mavrothal said, there is ways to prevent core packages getting updated. I've done it for slapt-get, I'm sure we can do that for apt-get too.

EDIT: packdude package lists - are they parsable through shell scripts?
Last edited by jamesbond on Sat 31 May 2014, 10:58, edited 1 time in total.

User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

#49 Post by mavrothal »

Iguleder wrote: EDIT: this really troubles me. With this design, what makes Puppy different from a minimal Debian installation with JWM and ROX-Filer? And what about package upgrades - if we use our own kernel instead of the Debian one, what happens when there's a kernel package in Debian?
Yes if you just use deb packages and maybe change a couple of things you indeed get minial-debian. But keeping rootfs-skeleton and rootfs-packages you have all the puppy goodies and scripts dropping (excluding if conflicting) the corresponding deb packages. No wireless infra for example or the equivalents of pApps you may have included.
The major problem I still see with this approach is apps that pull a trackload of dependencies and for those puppy usually provides pets compiled differently to decrease bloat. The other major problem is that some distros do not split packages like puppy while is some packages NLS files are bigger than the actual binaries.
Hmm... maybe at the end of the day we still need to have a bistro-compat-spesific puppy repo.

PS: The "problem" with having full distro packages is that in order to provide the same functionality as current puppies you'll probably need an 1GB machine and end up running at comparable speeds as the mother distro. So why bother...
Actually I think that is this 1% puppy-flaws that provide 99% of its strengths...
Last edited by mavrothal on Sat 31 May 2014, 11:05, edited 1 time in total.

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#50 Post by Iguleder »

jamesbond wrote:EDIT: packdude package lists - are they parsable through shell scripts?
Sure. Here's an example of packdude -l (a list of all packages in the repository):

Code: Select all

alsa_lib|1.0.27.2|Core audio library
aterm|1.0.1|Terminal emulator
bc|1.06|A numeric processing language
beaver|git30052014|A text editor
bftpd|4.2|An FTP server
binutils|2.17|Assembly tools
ncurses|5.9|A terminal handling library
bwm_ng|0.6|A network monitor
calcoo|1.3.15|A calculator
calcurse|3.1.4|A personal organizer
conky|1.9.0|A system monitor
curl|7.37.0|A file transfer library
dhcpcd|3.2.3|A DHCP client
dialog|1.2-20140219|A tool for displaying dialogs through shell scripts
diffutils|3.3|Patch creation and file comparison tools
dillo|git30052014|A web browser
dropbear|2014.63|A SSH server and client
gnu_efi|3.0u|A common functions library for UEFI applications
elilo|3.16|A UEFI boot loader
evilwm|1.1.0|A window manager
sdl|1.2.15|A graphics and audio abstraction layer library
ffmpeg|2.2.2|A complete multimedia solution
findutils|4.4.2|File search tools
fuse|2.9.3|An interface for user-mode file systems
gbase|0.5|A base converter
gcalendar|0.6.0|A calendar
gcolor|0.4|A color chooser
gdk_pixbuf|git30052014|An image loading library
gdmap|0.2|A disk usage visualizer
grun|0.8.1|An application launcher
gtkcat|0.1|A disk cataloger
gtkdialog1|1.3|A tool for displaying graphical dialogs through shell scripts
gtkdiskfree|1.6.5|A disk usage analyzer
gtkedit|1.0|A text editor
gtkfind|1.1|A file search tool
gtkfontsel|1.1|A font chooser
gtklepin|0.2|A note taking application
gtksee|0.6.0b-1|An image viewer
guiftp|0.1|A FTP client
lbzip2|2.5|Compression tools
xz|5.0.5|Compression tools
libarchive|3.1.2|An archive handling library
guitar|0.1.4|An archive manager
gzip|1.6|Compression tools
htop|1.0.3|A process viewer
ircii|20131230|An IRC client
isolinux|4.07|A BIOS boot loader for ISO9660 file systems
libnl_tiny|svn30052014|An IPC library
iw|3.14|A wireless network interface configuration tool
jwm|2.2.2|A window manager
lazy_utils|git30052014|System tools
less|458|A file viewer
libxml2|2.9.1|A XML parsing library
linux|3.10.40|An operating system kernel
lynx|2.8.8|A web browser
mandoc|1.12.3|A man page formatting tool
mhwaveedit|1.4.23|An audio recording and editing tool
tinyalsa|git30052014|Core audio library
mpg123|1.20.0|A MP3 player
nano|2.2.6|A text editor
nc|110|A generic TCP client or server
ncdu|1.10|A disk usage analyzer
ogg122|git30052014|An Ogg Vorbis player
sqlite|3080403|An embedded database
packdude|git30052014|A package manager
useatbelt|git31052014|A driver loader
patch|2.7.1|A file patching tool
rfkill|0.5|A tool for disabling network interfaces
rox|git30052014|A file manager
screen|4.0.3|A terminal multiplexer
shed|1.15|A hex editor
squashfs_tools|4.3|SquashFS file system manipulation tools
sylpheed|1.0.6|An e-mail client
tinyunmute|git30052014|An audio volume unmuting tool
tinyxserver|git30052014|A display server
vile|9.8|A text editor
wget|1.15|A HTTP and FTP client
wpa_supplicant|2.1|A tool for association with wireless networks
x11vnc|0.9.13|A VNC server
xchat|git30052014|An IRC client
xev|1.1.0|A X11 events inspection tool
xhippo|3.5|An audio player
xmodmap|1.0.8|A X11 keyboard map manipulation tool
ytree|1.97|A file manager
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
neerajkolte
Posts: 516
Joined: Mon 10 Feb 2014, 07:05
Location: Pune, India.

#51 Post by neerajkolte »

jamesbond wrote:Come on, how come it's only me, mav, saintless and stemsee? Others?
We the users are there too. We don't have the knowledge of systems yet. So we are silently watching the progress.
jamesbond wrote:either we make puppy "more compatible" with the parent distros so we can use their repos better
Yes that would really make people happy. I really liked the way this thread, Debiandog, and Packdude are going.
Keep up the good work.

- Neeraj
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson

“We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.â€￾
- Amara’s Law.

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#52 Post by Iguleder »

I think we should define our goals first.

As I see things, our goals are:
1) Being an independent, lighweight distro
2) Sustainability - easy development suitable for a small development team, without a great burden of maintenance
3) Being a good distro which does many things (multimedia, internet, development) and does them well

Using the packages of another distro gives us (2) and (3), but makes us less independent. For example, if Ubuntu moves to systemd (and it does) and Puppy is a mini-Ubuntu, we have to deal with that. Someone else decides for Puppy what's good for Puppy.

Maybe a hybrid approach is the right thing to do: run Ubuntu in a chroot (along with Apt, Ubuntu's X server, etc') on top of an independent base without many packages (just a kernel and Busybox).
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

stemsee

kernel

#53 Post by stemsee »

"Unfortunately fully compatible is impossible in my opinion if you keep Puppy kernel, ... "

I was using my kernel-3.13.9-EmSee-pae and running debian dog rather nicely! I don't think the kernel is a puppy kernel or the problem, rather the initrd is a puppy initrd. This issue needs clarifictaion.

Compiling the kernel for chroot, lxc, requires additional options like cgroups, and group-scheduling support etc

Very minor configuration.

User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

#54 Post by mavrothal »

Iguleder wrote:I think we should define our goals first.

As I see things, our goals are:
1) Being an independent, lighweight distro
2) Sustainability - easy development suitable for a small development team, without a great burden of maintenance
3) Being a good distro which does many things (multimedia, internet, development) and does them well

Using the packages of another distro gives us (2) and (3), but makes us less independent. For example, if Ubuntu moves to systemd (and it does) and Puppy is a mini-Ubuntu, we have to deal with that. Someone else decides for Puppy what's good for Puppy.

Maybe a hybrid approach is the right thing to do: run Ubuntu in a chroot (along with Apt, Ubuntu's X server, etc') on top of an independent base without many packages (just a kernel and Busybox).
The whole woof idea was/is this exact hybrid approach.
The question is can we make the building process more simple/effective while keeping the puppy characteristics?
If not, what and how much do we sacrifice?
If we go for maximum compatibility where innovation/difference is coming from?
If we are to make a simple build system that will just take debs and makes them into an an OS so more can people can build woof-puppies, what new are they going to offer? Why would we need more people to make pups? So we can have a different initial set of debs?...

So, I thing that being an "independent, full-featured, fully functional OOB, with maximum installation flexibility but lightweight and efficient" distro is what defines puppy.
For systemd/GNOME3 there is Fedora/RHEL
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==

User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

#55 Post by saintless »

Iguleder wrote:...For example, if Ubuntu moves to systemd (and it does) and Puppy is a mini-Ubuntu, we have to deal with that. Someone else decides for Puppy what's good for Puppy.
I think Ubuntu or Debian will never remove the choice to switch between systemd and sysvinit. It will be huge mistake if it happens. This doesn't makes the distro less independent but only more flexible. If you use latest Gnome systemd boot will be needed but otherwise you can't use Gnome anyway on any linux. If we refuse to include systemd boot as option in Puppy more and more packages will not work in the future from Debian/Ubuntu repository. Then why trying to use this repositories with Puppy?
The package manager problem can't be solved with new made package manager and updating again and again separate repository for puppy with new package versions will be never ending process.
It will be new Puppy with the same old problems.

User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

Re: kernel

#56 Post by saintless »

stemsee wrote:"Unfortunately fully compatible is impossible in my opinion if you keep Puppy kernel, ... "

I was using my kernel-3.13.9-EmSee-pae and running debian dog rather nicely! I don't think the kernel is a puppy kernel or the problem, rather the initrd is a puppy initrd. This issue needs clarifictaion.
Try to apt-get install virtualbox and linux-headers or Gnome3 with kernel-3.13.9-EmSee-pae and reboot using systemd and you will see it is not fully compatible anymore with Debian. With sure you will find more problems if you keep testing.

stemsee

#57 Post by stemsee »

Iguleder wrote:Maybe a hybrid approach is the right thing to do: run Ubuntu in a chroot (along with Apt, Ubuntu's X server, etc') on top of an independent base without many packages (just a kernel and Busybox).
This SEEMS like the best/most versatile solution (which is what i tried with saucy salamander CE on linux 3.15.0-rc6). Focusing on getting the independent base that runs several chroot systems simultaneously, with the option of sending server output to multi-monitor/desktop setup, means you can run android, on desktop 1 puppy on 2, ubuntu on 3, debian on 4, simultaneously without the overhead of full virtualisation. And have the option to layer further base sfs in the puppy chroot to add extra flavours. When i layered a precise onto a slacko and then a dpup on top of that, ppm gave access to al repos!! And all packages installed and worked successfully.
Last edited by stemsee on Sat 31 May 2014, 12:31, edited 1 time in total.

User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

#58 Post by saintless »

mavrothal wrote:...If we go for maximum compatibility where innovation/difference is coming from?
If we are to make a simple build system that will just take debs and makes them into an an OS so more can people can build woof-puppies, what new are they going to offer? Why would we need more people to make pups? So we can have a different initial set of debs?...
Innovation/difference comes from adding puppy options to native debian/ubuntu system as remastering, loading-sfs, and all scripts that make puppy different, but removing nothing from the options debian/ubuntu gives us. Keeping such distro light as Puppy is possible.

This question really needs answer: Why we need more people to make pups and why we need so many pupps at first place?

Toni

stemsee

Why?

#59 Post by stemsee »

saintless wrote:This question really needs answer: Why we need more people to make pups and why we need so many pupps at first place?
By experimenting and making mistakes we learn and make improvements, refinement and other progress. I always/only learn from my and others mistakes! We have an idea and we should pursue it to benefit all of geek-kind!

Last week I had an idea for a new instrument. Yesterday I made it. Tomorrow I will perform with it and people will be delighted and distracted in an ameliorating experience. Then I'm going to sell it and buy a new laptop! That's why we need a new operating system, 'cos I'm bored with this one already! lol

User avatar
peebee
Posts: 4370
Joined: Sun 21 Sep 2008, 12:31
Location: Worcestershire, UK
Contact:

What is Puppy?

#60 Post by peebee »

Iguleder wrote:EDIT: this really troubles me. With this design, what makes Puppy different from a minimal Debian installation with JWM and ROX-Filer? And what about package upgrades - if we use our own kernel instead of the Debian one, what happens when there's a kernel package in Debian?
Hi - another lurker here with not enough knowledge to contribute much....

[philosophy]What makes Puppy is this forum - and the wealth of software developed and made available by all the contributors to the forum - if a future system does not allow the software on the Puppy forum to be used - pets and sfs's - then to my mind it's no longer Puppy....software does not need to be stored in a single repository to be accessible and useful[/philosophy]

Cheers!
peebee
ImageLxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64

Post Reply