Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sat 02 Aug 2014, 00:32
All times are UTC - 4
 Forum index » Taking the Puppy out for a walk » Announcements
Woof-based Puppy builders wanted
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 3 of 8 [107 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7, 8 Next
Author Message
mavrothal


Joined: 24 Aug 2009
Posts: 1571

PostPosted: Sat 31 May 2014, 01:47    Post subject: Re: Next-gen Woof  

jamesbond wrote:

I have a proof-of-concept under-300-lines script that builds puppy.sfs *directly* from Ubuntu, tested with trusty and utopic (32-bit).
It uses genuine Woof-CE puppy scripts and genuine Woof-CE initrd

<snip>

6. You still need Woof-CE as all the puppy core lives there.
7. Uses native Debian package manager. Does not use PET.

Any interest? Comments?

Yeh I'm interested Very Happy
Puppy does use a number of pets though not in woof and characteristic to puppy (jwmconfig for example).
However this could be fixed bu adding them flat into the recently added woof-CE/woof-code/rootfs-packages directory
Does you 300-liner considers rootfs-packages and if it does does it also runs any pinstall included there?

This can be a really interesting development as rootfs-skeleton and rootfs-packages can hold what really makes puppy a Puppy and then a package manager (apt-get, yum, pacman, slaptget, packdude) can offer full distro compatibility

_________________
Kids all over the world go around with an XO laptop. They deserve one puppy (or many) too Very Happy
Back to top
View user's profile Send private message 
saintless


Joined: 11 Jun 2011
Posts: 2226
Location: Bulgaria

PostPosted: Sat 31 May 2014, 02:48    Post subject:  

Hi, Jamesbond.
jamesbond wrote:
I take the Ubuntu (or Debian) packages and build them up. The result is Puppy to the core, yet will work and is fully compatible with Ubuntu/Debian (I hope Laughing )

Unfortunately fully compatible is impossible in my opinion if you keep Puppy kernel, but the result will be much more Debian/Ubuntu repository compatible system than Dpup or Lucid.
Still this is very interesting project and it worth separate development. Is it possible to upload some ready for testing resulted iso?

What I see as a problem from quick thoughts:

1. Different system structure - for example rc.local is located different in Debian and Ubuntu and .xinitrc is .xsession. I know such small things seems not important but some packages in Debian or Ubuntu repository may and most probably will depend on them.

2. Forget about packages related to kernel downloaded from Ubuntu or Debian - linux-headers for example and maybe VirtualBox and similar programs.

3. I know it is not very welcome replacement here, but still... Forget about using systemd and programs depending on systemd from Debian or Ubuntu repository. With Debian/Ubuntu kernel and initrd.img you can boot with systemd by adding this to the boot code init=/bin/systemd With Puppy kernel and initrd.gz this option is lost forever.

4. Some programs from Debian/Ubuntu repository will not work without changes for root account.

5. Dpkg reads the information from /var/lib/dpkg/info , status, available. It is very easy to break dpkg database using different package managers as option to install pet or other distro packages. It should be dpkg (+apt-get or Synaptic) only or you are looking for troubles.

Toni

_________________
Farewell, Nooby, you will be missed...
Back to top
View user's profile Send private message MSN Messenger 
mavrothal


Joined: 24 Aug 2009
Posts: 1571

PostPosted: Sat 31 May 2014, 02:58    Post subject:  

As far as I know all package managers allow for non-updatable packages.
Kernel can be a problem when it comes to drivers, but ubuntu at least is using pretty recent kernels and as long as the puppy kernel is the same version and has devtmpfs=y should not have issues, even if the user decides to go with the you-do-as-I-say systemd

_________________
Kids all over the world go around with an XO laptop. They deserve one puppy (or many) too Very Happy
Back to top
View user's profile Send private message 
jamesbond

Joined: 26 Feb 2007
Posts: 2059
Location: The Blue Marble

PostPosted: Sat 31 May 2014, 03:31    Post subject:  

Quote:
Puppy does use a number of pets though not in woof and characteristic to puppy (jwmconfig for example).
Yup, I'm aware of that, though I don't know what exactly are needed.

Quote:
However this could be fixed bu adding them flat into the recently added woof-CE/woof-code/rootfs-packages directory
Either that, or a collection of tarballs we can simply expand upon building Or if you want to be proper, make them into DEBs Smile

Quote:
Does you 300-liner considers rootfs-packages and if it does does it also runs any pinstall included there?

Yes.

Quote:
This can be a really interesting development as rootfs-skeleton and rootfs-packages can hold what really makes puppy a Puppy and then a package manager (apt-get, yum, pacman, slaptget, packdude) can offer full distro compatibility

Yes, that's the plan.

saintless wrote:
Unfortunately fully compatible is impossible in my opinion if you keep Puppy kernel, but the result will be much more Debian/Ubuntu repository compatible system than Dpup or Lucid.
Thanks Toni, it's good observation. All of your observations are valid. You are right, the word "fully" must be qualified.

Quote:
Still this is very interesting project and it worth separate development.

I'm posting this here in the hope of attracting current and future Woof developers (hopefully enticing the guys who want to do Woof but currently thinks that Woof is "too hard"). If there is enough interest I will start a new thread in Cutting Edge section Smile

Quote:
Is it possible to upload some ready for testing resulted iso?
The current script produces puppy.sfs, not iso, yet. But it is just a quantum leap from puppy.sfs to iso.

Quote:
1. Different system structure - for example rc.local is located different in Debian and Ubuntu and .xinitrc is .xsession. I know such small things seems not important but some packages in Debian or Ubuntu repository may and most probably will depend on them.
I haven't checked them. I thought most distros have .xinitrc and .xsession in $HOME. Puppy does, too. In any case, going forward, we can align Puppy's structure to be more "standards"-compliant, so to speak, for compatibility reasons. That being the case, there is a limit to how far we can mold Puppy before it becomes not-Puppy Smile so your concern is valid.

Quote:
2. Forget about packages related to kernel downloaded from Ubuntu or Debian - linux-headers for example and maybe VirtualBox and similar programs.
Yes, in this case, we need to provide our own DEB repository. Aligning Puppy to work with standard kernel can be done but it's so far off I wouldn't even consider it at the time being.

Quote:
3. I know it is not very welcome replacement here, but still... Forget about using systemd and programs depending on systemd from Debian or Ubuntu repository. With Debian/Ubuntu kernel and initrd.img you can boot with systemd by adding this to the boot code init=/bin/systemd With Puppy kernel and initrd.gz this option is lost forever.
Correct, although, going forward, anybody's who likes systemd can propose modification to Puppy core so that it can work with systemd.

Quote:
4. Some programs from Debian/Ubuntu repWhat I see as a problem from quick thoughts:ository will not work without changes for root account.
Correct. I can't remember, I think dejan or whoever was planning to enable multi-user to Puppy and has committed some code too.

Quote:
5. Dpkg reads the information from /var/lib/dpkg/info , status, available. It is very easy to break dpkg database using different package managers as option to install pet or other distro packages. It should be dpkg (+apt-get or Synaptic) only or you are looking for troubles.
No other package manager, it's dpkg (and its front-ends like apt-get/synaptic onwards). If you want to install pet, use a pet2deb converter and then install it with dpkg.

I guess going forward, we need to have names for these (right now, all these independent components are all called "Woof-CE" - which is super confusing.
a) The name of the repository that holds puppy-builder code, and puppy core code ("Woof repository").
b) Puppy builder code ("Woof build system").
c) Puppy core code (rootfs-skeleton, initrd-skeleton).

Please decide on what you want to call these three, but please don't call all of them them "Woof-CE". The script I'm talking about touches on point b, the puppy builder code (with minimal modification to puppy core code). Toni's concern are mainly for Puppy core (point c).

_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13
Back to top
View user's profile Send private message 
saintless


Joined: 11 Jun 2011
Posts: 2226
Location: Bulgaria

PostPosted: Sat 31 May 2014, 03:45    Post subject:  

mavrothal wrote:
As far as I know all package managers allow for non-updatable packages.

If you install not dpkg registered package it should be very carefully configured.
Make sure not to overwrite existing executables and libs.
Very important because you will change them with different version that will probably not work for the package that installed the overwritten executable or lib.
And after uninstalling the same non-updatable package you will remove the executable or libs that were installed and overwritten already. The result is missing files and dpkg can't solve the problem because this files are actually manually removed and the related program does not work anymore, even dpkg says "All is fine here".

Quote:
Kernel can be a problem when it comes to drivers, but ubuntu at least is using pretty recent kernels and as long as the puppy kernel is the same version and has devtmpfs=y should not have issues, even if the user decides to go with the you-do-as-I-say systemd

No. It will not work. Tested with official Debian kernel and porteus-initrd.img Even in this case systemd boot code does not work. No way to make it work with puppy kernel + puppy inird without changes.

Toni

_________________
Farewell, Nooby, you will be missed...
Back to top
View user's profile Send private message MSN Messenger 
saintless


Joined: 11 Jun 2011
Posts: 2226
Location: Bulgaria

PostPosted: Sat 31 May 2014, 04:00    Post subject:  

jamesbond wrote:
...I thought most distros have .xinitrc and .xsession in $HOME. Puppy does, too. In any case, going forward, we can align Puppy's structure to be more "standards"-compliant, so to speak, for compatibility reasons. That being the case, there is a limit to how far we can mold Puppy before it becomes not-Puppy Smile

Yes, .xinitrc is in $HOME as .xsession. Just the name is different for puppy and debian. /etc/rc.local in debian is /etc/rc.d/rc.local in puppy. Simple symlinks could fix this differences without structure changes I think but never tested.

Quote:
Aligning Puppy to work with standard kernel can be done but it's so far off I wouldn't even consider it at the time being.

It's too good to be true... Smile I tried to make something similar but failed bad. Too complicated for me.

Quote:
...I think dejan or whoever was planning to enable multi-user to Puppy and has committed some code too.

Yes, and it works quite nice as multiuser system. I think even spot can be used for such applications by running spot in terminal and then the program name. Needs testing again.

Quote:
If you want to install pet, use a pet2deb converter and then install it with dpkg.

See my last post. It has to be very well configured deb to prevent troubles with dpkg. I doubt auto-convert script can cover all packages safe. It must add conflict: option to prevent overwriting other files.

Toni

_________________
Farewell, Nooby, you will be missed...
Back to top
View user's profile Send private message MSN Messenger 
jamesbond

Joined: 26 Feb 2007
Posts: 2059
Location: The Blue Marble

PostPosted: Sat 31 May 2014, 04:09    Post subject:  

saintless wrote:
mavrothal wrote:
As far as I know all package managers allow for non-updatable packages.

If you install not dpkg registered package it should be very carefully configured.

True. *All* packages, down to the very first one (libgcc1) is registered with dpkg, including puppy-base files (rootfs-skeleton) and dynamically created busybox symlinks.

IIRC the non-updatable package list is in apt-get, not in dpkg --- dpkg allows you to do *anything*. But my knowledge in dpkg is limited, saintless should know better.

Quote:
No. It will not work. Tested with official Debian kernel and porteus-initrd.img Even in this case systemd boot code does not work. No way to make it work with puppy kernel + puppy inird without changes.
Every initrd has its own assumptions on how the kernel is compiled. With proper configuration, one can reduces this assumption - down to a standard kernel. But it's difficult - thus my earlier note that this will be far off, even if it is ever considered Laughing

Quote:
See my last post. It has to be very well configured deb to prevent troubles with dpkg. I doubt auto-convert script can cover all packages safe. It must add conflict: option to prevent overwriting other files.

I didn't consider this case. Anyone who installs foreign packages run the risk of borking his/her system, as much as running a self-compiled package (just as today, anyone who installs debian package on wary-puppy (T2-based puppy) run the risk of borking his/her system. There is some amount of sanity checks we can do during pet2deb conversion process but I doubt we can (or want to) cover all bases.

cheers!

_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13
Back to top
View user's profile Send private message 
jamesbond

Joined: 26 Feb 2007
Posts: 2059
Location: The Blue Marble

PostPosted: Sat 31 May 2014, 04:13    Post subject:  

More teaser to pique interest:

Code:
# ubuntu puppy pkglist
#
# generic commands: %exit %include %makesfs
# special package commands: %get_pkgs_by_priority %puppy %bblinks
# installer commands: %bootstrap %dpkg %depend %nodepend
# startup default: bootstrap, depend
#
# extra param for commands, params can be quoted
# %include         include-file
# %makesfs         output.sfs [squashfs-param]
# %get_pkg_by_prio priority ["inclusion-egrep"] ["exclusion-egrep"]
# %bblinks         [nousr]
#

%get_pkgs_by_priority "required" ".*lib.*|^tzdata|^bash|^dash|^lsb-base|^ncurses.*|bsdutils|kmod" "^klibc|.*plymouth.*"
%dpkg # switch to dpkg installer
dpkg  # install dpkg
grep
gawk
sed
tar
mawk
gzip
cpio
mingetty

# final
busybox-static
%puppy
%bblinks # fallback for missing utilities
%makesfs puppy.sfs -comp gzip -Xcompression-level 1

This will install about 79 packages, enough to boot to shell. Works on tahr and utopic (just need to say VERSION=trusty ./deb-build.sh or VERSION=utopic ./deb-build.sh - the pkglist remains the same). By now deb-build.sh is 293 lines.

Come on, how come it's only me, mav, saintless and stemsee? Others?

_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13
Back to top
View user's profile Send private message 
mavrothal


Joined: 24 Aug 2009
Posts: 1571

PostPosted: Sat 31 May 2014, 04:20    Post subject:  

jamesbond wrote:

Quote:
However this could be fixed bu adding them flat into the recently added woof-CE/woof-code/rootfs-packages directory
Either that, or a collection of tarballs we can simply expand upon building Or if you want to be proper, make them into DEBs Smile

Yes but this would mean another repo to maintain for each binary_compat.
Since they are mostly scripts or icons, having them flat in woof-CE also facilitates maintenance.


jamesbond wrote:
I guess going forward, we need to have names for these (right now, all these independent components are all called "Woof-CE" - which is super confusing.
a) The name of the repository that holds puppy-builder code, and puppy core code ("Woof repository").
b) Puppy builder code ("Woof build system").
c) Puppy core code (rootfs-skeleton, initrd-skeleton).

Please decide on what you want to call these three, but please don't call all of them them "Woof-CE". The script I'm talking about touches on point b, the puppy builder code (with minimal modification to puppy core code). Toni's concern are mainly for Puppy core (point c).


There is a major headache in this. Puppy is not standard compliant, specially when it comes to menus and file location that's why it has all these package-templates in woog-code to rework standard packages.

I think the way to go is get a woof3 branch going where, we keep woof-code/{boot,rootfs-skeleton,rootfs-packages} and maybe kernel-skeleton
Also keep kernel-kit.
Then make things in rootfs-skeleton standard-compliant and rewrite the build scripts/tweaks.
It sound much easier than it actually is Twisted Evil

_________________
Kids all over the world go around with an XO laptop. They deserve one puppy (or many) too Very Happy
Back to top
View user's profile Send private message 
stemsee


Joined: 27 Jun 2013
Posts: 281
Location: London

PostPosted: Sat 31 May 2014, 04:24    Post subject: thoughts
Subject description: alternative method looming in the shadows
 

LXC-create container. I haven't tried it yet, but I imagine it creates a container with it's own special code included, which presumably create a 'root-fs tree. lxc-create -n ubuntu (option) .... lxc then downloads and installs ubuntu in this container. This container might accept furnishing with woof-CE root-fs and then continue with the normally ubuntu/deb/fed/arch/ insltallation. Make some adjustments and links, then squashfs the container and boot with any kernel and initrd or load into ram or chroot (lxc = chroot on steroids.)
Last edited by stemsee on Sat 31 May 2014, 05:39; edited 1 time in total
Back to top
View user's profile Send private message MSN Messenger 
saintless


Joined: 11 Jun 2011
Posts: 2226
Location: Bulgaria

PostPosted: Sat 31 May 2014, 04:36    Post subject:  

jamesbond wrote:
IIRC the non-updatable package list is in apt-get, not in dpkg --- dpkg allows you to do *anything*.

apt-get, aptitude, synaptic all use dpkg and actually the hard work is always dpkg responsibility. Using dpkg -i packagename from terminal will add information in /var/lib/dpkg files and will check out for missing dependencies, but it will not sort out dependencies. dpkg will show the missing dependencies and will leave the package not configured till the dependencies are manually installed with dpkg or apt-get -f install is executed. Or you should download all dependencies in a folder and use dpkg -i -R to install them all.
apt-get, synaptic, aptitude will install/uninstall all dependencies automatically by only one package name given in the command, but the actual work is for dpkg again. I might missing something since I'm not dpkg specialist but this is from my experience.

I think leaving to the user the responsibility to keep dpkg database safe is a big risk. At least pet2deb converter should come with red color warning use this at your own risk. Breaking dpkg database makes the main advantage useless. This risk should be kept at minimum.

Toni

_________________
Farewell, Nooby, you will be missed...
Back to top
View user's profile Send private message MSN Messenger 
saintless


Joined: 11 Jun 2011
Posts: 2226
Location: Bulgaria

PostPosted: Sat 31 May 2014, 04:48    Post subject:  

Stemsee,

if some files are missing in this container forget to boot it with puppy kernel. It will look for /etc/rc.d files that are not included if you use Ubuntu packages to build the container. You can boot it with Debian kernel and initrd.img but Puppy is much different.

Toni

_________________
Farewell, Nooby, you will be missed...
Back to top
View user's profile Send private message MSN Messenger 
Iguleder


Joined: 11 Aug 2009
Posts: 1873
Location: Israel, somewhere in the beautiful desert

PostPosted: Sat 31 May 2014, 04:57    Post subject:  

jamesbond wrote:
Come on, how come it's only me, mav, saintless and stemsee? Others?


I'm in, as long as the end product allows me to build a 64-bit, all-static, GTK1 Puppy with tinyxserver, Linux-libre and my own package manager. Wink

_________________
My homepage
Back to top
View user's profile Send private message Visit poster's website MSN Messenger 
ICQ Number 
jamesbond

Joined: 26 Feb 2007
Posts: 2059
Location: The Blue Marble

PostPosted: Sat 31 May 2014, 05:00    Post subject:  

mavrothal wrote:
jamesbond wrote:
I guess going forward, we need to have names for these (right now, all these independent components are all called "Woof-CE" - which is super confusing.
a) The name of the repository that holds puppy-builder code, and puppy core code ("Woof repository").
b) Puppy builder code ("Woof build system").
c) Puppy core code (rootfs-skeleton, initrd-skeleton).

Please decide on what you want to call these three, but please don't call all of them them "Woof-CE". The script I'm talking about touches on point b, the puppy builder code (with minimal modification to puppy core code). Toni's concern are mainly for Puppy core (point c).


There is a major headache in this. Puppy is not standard compliant, specially when it comes to menus and file location that's why it has all these package-templates in woog-code to rework standard packages.

I think the way to go is get a woof3 branch going where, we keep woof-code/{boot,rootfs-skeleton,rootfs-packages} and maybe kernel-skeleton
Also keep kernel-kit.
Then make things in rootfs-skeleton standard-compliant and rewrite the build scripts/tweaks.
It sound much easier than it actually is Twisted Evil

Nah, I'm not thinking that far. I'm just thinking about terminology, because today "Woof-CE" is used to name these 3 distinct components. My suggestion is this:
a) the repository - keep the name "Woof-CE"
b) the builder - use the name Woof2, next generation perhaps Woof3, Woof4, Woof-next, etc
c) the puppy core files - let's called it "puppy-base".
Both b) and c) are kept in a). As for the structure, we'll go with what works.

stemsee - LXC is just like qemu, except that you share the kernel with the host os. I don't see the relevance of LXC with puppy building, except perhaps for testing (I use qemu for testing). If you want to see how it works in puppy-flavoured distro, get Fatdog64 Smile And saintless is correct - with what we have today, Puppy will *not* boot on Debian kernel; it needs some kernel config which isn't in Debian kernels. (Ubuntu kernel, may be - I have seen some Ubuntu kernel comes with aufs - but definitely definitely not Debian kernel).

_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13
Back to top
View user's profile Send private message 
jamesbond

Joined: 26 Feb 2007
Posts: 2059
Location: The Blue Marble

PostPosted: Sat 31 May 2014, 05:17    Post subject:  

Iguleder wrote:
jamesbond wrote:
Come on, how come it's only me, mav, saintless and stemsee? Others?


I'm in, as long as the end product allows me to build a 64-bit, all-static, GTK1 Puppy with tinyxserver, Linux-libre and my own package manager. Wink

Laughing What I've done is for Ubuntu, so the only thing that meets your criteria is "64-bit" Smile

Looking far ahead, I'm thinking that perhaps we should have different builders for different source distros:
- one for Ubuntu/Debian (they are close enough to be considered as one distro, only perhaps the pkglist is different)
- one for RPM-based distros (I don't know yet whether we can combine them - is fedora/mageia similar enough?)
- one for Slackware-based distros
- and of course, the ultimate one: one for packdude-based Puppy with GTK1 and all-static binaries.

And perhaps unified by one front-end which does nothing other than asking questions and then execute the right builders. This way, modification to ubuntu builders will not accidentally break rpm builders, and vice versa. Maintenance is also easier, people only need to maintain the source distros that you're interested in.

If you look at the source of debootstrap, you'll see that it has different "modules" for different versions of Debians (the "debootstrap" itself is basically just a front-end to these modules), so I think the idea has some precedence.

For packdude, all I need from packdude is that:
a) it is compilable on a bog standard OS (doesn't require asm libs that only runs in 64-bit with AVX, for example), and
b) that it can install packages in a specified chroot directory, including its administration files, and correctly recorded installation path relative to chroot dir,
c) and it can install from a local repo.
The last one is not necessarily needed, but it makes stuff easier for multiple builds (I don't have to re-download packages every time I attempt to build puppy).

I think packdude has a) and b), we only need c) + the ability to download packdude's repo package list and parse them and download individual packages without installing (either with wget, or with packdude itself - your choice).

_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 3 of 8 [107 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7, 8 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Taking the Puppy out for a walk » Announcements
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1490s ][ Queries: 12 (0.0112s) ][ GZIP on ]