Auto-build a Puppy iso; single script with optional gui

A home for all kinds of Puppy related projects
Message
Author
User avatar
aaaaa
Posts: 39
Joined: Tue 22 May 2018, 14:57

#261 Post by aaaaa »

Profiles for ISOS, multi user, or at least a usable non root user, etc.

Good ideas.

But there's a little problem. Implementing anything in woofce or any other woof is incredibly difficult.

It not only requires devoted developer(s), but also devoted testers, and that's more than a luxury nowadays.

However the project is slowly getting rid of many things that prevent further improvements. There's a pull request to delete ~500 files.

The problem is the design itself inherited from the original woof.. yes that's the main problem.

Just fixing the current bugs is a huge task. Some other changes were also implemented but the code was lost because there was no one to test them, many changes happened, and it's a pain to rebase all of that work.

My advice is as usual see other options.., i see TazPup, which is small, mistfire only takes a few puppy-specific things and tries to blend 2 distros. clever. or the debiandogs if you want a completely working package manager, etc..

However woofce still have the ability to use any distro and a very small iso. But you'll see it in action after a massive earthquake...

User avatar
nosystemdthanks
Posts: 703
Joined: Thu 03 May 2018, 16:13
Contact:

#262 Post by nosystemdthanks »

But there's a little problem. Implementing anything in woofce or any other woof is incredibly difficult.
which is why i gave up on woof immediately and went in a different direction trying to find a shortcut. i ended up with a distro instead, but the fact is that if i were getting paid to implement this stuff on top of woof (not saying it would or should happen, id rather do things my own way for free) id still insist on using my own tools to interface with it.

and that would mean youre stuck with python, which woof doesnt require.

but then without my tools i never would have created a distro. i mean i can code in bash, i implemented a simple logo interpreter in 100 loc of bash. but my bash skills arent up to what woof needs.
It not only requires devoted developer(s), but also devoted testers, and that's more than a luxury nowadays.
yes. id much rather try wiaks version than try out woof again. im not trying to be unkind. i tried woof because people insisted, and i couldnt really comment on it unless i tried it, right?

and then i set out to do something automatic, which wiak has done a far better job with-- while i do something else.
However the project is slowly getting rid of many things that prevent further improvements. There's a pull request to delete ~500 files.
nice work.
The problem is the design itself inherited from the original woof.. yes that's the main problem.
i agree, but its safer for you to say. you also have a much better idea what youre talking about, so its safer and smarter if you say it.
many changes happened, and it's a pain to rebase all of that work.
if it wasnt, i might have tried it.
My advice is as usual see other options.., i see TazPup, which is small, mistfire only takes a few puppy-specific things and tries to blend 2 distros. clever. or the debiandogs if you want a completely working package manager, etc..
or create a script that downloads tahr and devuan live, and create a hybrid iso with a puppy mode that boots into puppy with some files from devuan, or boots into devuan mode with some files (like petget) from tahr. which is the silly (but fun and educational) approach i took. note that i now use that distro all the time, but its no longer puppy-based. the script that builds it came out of working with puppy though.

but the point of course is that we get to choose from many different solutions. the advantage of wiaks of course is that it is based directly on woof-ce.
However woofce still have the ability to use any distro and a very small iso.
which is pretty cool.

whether to rebuild/modify a complex thing like woof or try from a different direction is going to depend on how much of a perfectionist the developer is.

because the person that goes to the trouble of fixing the complicated thing is usually the one who cares most about perfection. and compatibility, whatever the cost. values we can at least admire and salute. they keep puppy more like puppy, and thats in demand.
[color=green]The freedom to NOT run the software, to be free to avoid vendor lock-in through appropriate modularization/encapsulation and minimized dependencies; meaning any free software can be replaced with a user’s preferred alternatives.[/color]

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#263 Post by wiak »

Well maybe I'll get back to this sometime. I'm not sure. I was simply in 'maintenance' mode with it because I've been working on so many other things recently (too many really) and only had enough time and energy (almost) for that. Tinkering with woof-CE via script interventions is quite tricky unfortunately (though more could certainly be done - anything being possible...) - better/easier though if the build system moved towards a debootstrap style of operation (such as used in fredx181's mklive-stretch offering). But that's a big ask in terms of coding complexity and woof-CE re-write, which I'm not involved in anyway.

I had hoped to get back to IUP/Lua code writing, since I also want to learn something different and I'm interested in Lua and found IUP rather nice.

I also want to take a look at EasyOS to see what that is all about; it looks quite interesting from a different perspective point of view.

Then, I also wanted to put my feet up, though it is gratifying that a fair number found makepup somewhat useful at least as an introduction to woof-CE itself.

wiak

User avatar
aaaaa
Posts: 39
Joined: Tue 22 May 2018, 14:57

#264 Post by aaaaa »

A big thanks to wiak for this effort.

Getting into woofce might also mean trouble if you want to edit stuff or keep custom configs, as woofce is going through changes that also affect the list of pkgs to be added. Your stuff might end up producing a broken build.

Well, there's no other way to improve things, woofce is a collection of distros, and important changes have to be enforced in all of them. Consistency has a huge price as well. As i said, i also lost many codelines

Flavors, people don't want to be forced to use a certain DE, but in woofce you are AHAHA! I'm cheating using something other than the standard and enforcing in my build. That's life i guess...

wanderer
Posts: 1098
Joined: Sat 20 Oct 2007, 23:17

#265 Post by wanderer »

yes aaaaa and everyone

that was just a question
I think I will go back to playing with woof-ce
and just try to make a tiny iso
the woof-ce team will continue to refine things
(and they could use the next puppy development subforum)

woof-ce is the core of puppy
though there are many branches

a big thanks is in order to everyone that has made puppy great

barry k
john murga the forum
the woof-ce team
wiak ...
etc etc etc
too many to list

wanderer

User avatar
nosystemdthanks
Posts: 703
Joined: Thu 03 May 2018, 16:13
Contact:

#266 Post by nosystemdthanks »

aaaaa wrote:Flavors, people don't want to be forced to use a certain DE, but in woofce you are AHAHA! I'm cheating using something other than the standard and enforcing in my build. That's life i guess...
i really (actually) dont want to take over this thread, but i will add this:

the easiest way to add profiles to this wonderful script of wiaks would be to add a simple hook for other tools to run-- whether based on python or bash or lua or whatever-- either before mksquashfs runs or even after wiaks script completes the iso.

then no matter what goes on with woof, you can "unleash" (sorry, unleash is the name of one of barrys things, im just making the obvious pun) the second tool on it, which can run automatically from wiaks script. and if its not found, it simply wont run.

the disadvantage is that it wont make it run faster unless we go with wanderers default minimal profile.

i mean if the goal is to have the whole process go faster, the first stage (wiaks) has to be faster by default before it calls something else. and then the second stage for profiles could actually be more than one tool (default: no second stage, no worries) maintained by one or more different people with entirely different goals. these are ideas, not requests.
[color=green]The freedom to NOT run the software, to be free to avoid vendor lock-in through appropriate modularization/encapsulation and minimized dependencies; meaning any free software can be replaced with a user’s preferred alternatives.[/color]

User avatar
aaaaa
Posts: 39
Joined: Tue 22 May 2018, 14:57

#267 Post by aaaaa »

nosystemdthanks wrote: i really (actually) dont want to take over this thread, but i will add this:

the easiest way to add profiles to this wonderful script of wiaks would be to add a simple hook for other tools to run-- whether based on python or bash or lua or whatever-- either before mksquashfs runs or even after wiaks script completes the iso.
We're talking about the PKGS_SPECS.

I tested the script and it does work.

I changed this line
if [ "$KEEPBRANCH" = "false" ]; then

to
if [ "$KEEPBRANCH" = "false" -o ! -f ${woofclonefile} ]; then

You could just select all the packages you want and then puppy automatically resolves dependencies and all, but that's not the case.

The way things work now make profiles a tedious task, you have to do everything manually.

The advantages of this system allows cross-builds, and all sorts of weird stuff that is practically impossible in other build systems, but a high price.

To edit every part of the system you must be highly skilled, and in woofce there is usually only 1 person doing everything or nobody working at all, and the package management was not my priority because i see puppy more like a rescue distro. But i'm editing stuff to make things faster, including the ppm. This stuff needs a small redesign.

Regarding the auto-build script, i see it can be improved and be part of the git repo, but when it comes to my plans, it doesn't involve the forum..

User avatar
nosystemdthanks
Posts: 703
Joined: Thu 03 May 2018, 16:13
Contact:

#268 Post by nosystemdthanks »

aaaaa wrote:in woofce there is usually only 1 person doing everything or nobody working at all, and the package management was not my priority
all understandable. theyre probably afraid to step on toes.

but its typical for almost every free software project to have one person working on it at a time. realtime collaboration is a rare exception, at least most free software has one developer, even when its good.

User avatar
davids45
Posts: 1326
Joined: Sun 26 Nov 2006, 23:33
Location: Chatswood, NSW

Bionic - entry for Compatversion? Or how?

#269 Post by davids45 »

G'day,

Still trying to build a BionicPup with makepup-1.5 but can only get a XenialPup.

I can 'select' the kernel by putting it in the kernel2add directory and leaving the makepup.conf kernel line as "". (As below).

But what entry (and where) is required to pick a Ubuntu version? Looking at the makepup script I see:
# Default config:
nTARGETARCH="2" # i.e. target is x86 (1=arm, 2=x86, 3=x86_64)
nCOMPATDISTRO="5" # i.e. distro is slackware (1=debian, 2=devuan, 3=slackware, 4=trisquel, 5=ubuntu)
nCOMPATVERSION="3" # i.e. release version is 14.2 (1=14.0, 2=14.1, 3=14.2)
WOOFBRANCH="woof-CE-testing" # woofbranch, e.g. woof-CE-testing
nHUGEKERNEL="" # i.e. hugekernel to use is huge-4.4.70-s32-700_PAE
INTERACTIVE="false" # "true" means: Require user input during the build
gui="false" # "true" means: Use gui frontend
KEEPBRANCH="true" # "true" means: Keep previous woofbranch zip-file and woof-CE-branch directory
PAUSE="false" # "true" means: pause makepup script just after merge2out routine to allow package selection changes
DEVX="false" # "true" means: Create DEVX sfs
REBUILDALL="false" # i.e. rebuild changed packages only. Alternative, for now, is "true" for ALL_PACKAGES.
POPUPTHEMES="false" # "true" means: Pop-up 'choose themes' gui during the final 3builddistro-Z process
POPUPEXTRA="false" # "true" means: Pop-up 'choose extra packages' gui during the final 3builddistro-Z process
EXTRACONF="true" # "true" means: use makepup_extra.conf, if exists, to overwrite rootfs-packages.conf extra packages.
ADDPETS="false" # "true" means: adds to build any dotpets you have dropped into folder local-repositories/pets2add/
ADDPKGS="false" # "true" means: adds to build any distro-compatible packages you have dropped into folder local-repositories/pkgs2add/
I'm thinking, like in Slackware, it's the number I put in the nCOMPATVERSION line that would select Tahr, Xenial or Bionic but no luck so far trying '3' for a Bionic - being the third and latest Ubuntu version?

My last-tried config file was:
nTARGETARCH="2"
nCOMPATDISTRO="5"
nCOMPATVERSION="3"
WOOFBRANCH="woof-CE-testing"
DEVX="false"
INTERACTIVE="false"
gui="false"
KEEPBRANCH="false"
nHUGEKERNEL=""
PAUSE="false"
REBUILDALL="false"
POPUPTHEMES="false"
POPUPEXTRA="false"
ADDPETS="true"
ADDPKGS="false"
but gave me a Xenial iso output (by its name - I haven't actually installed it to confirm :oops: ).

And I haven't tried '4' here in case there's also Precise in the Ubuntu list of versions, so Bionic could be '4'?.

David S.

User avatar
peebee
Posts: 4370
Joined: Sun 21 Sep 2008, 12:31
Location: Worcestershire, UK
Contact:

#270 Post by peebee »

1 arm
2 x86
3 x86_64
Type number of target architecture: 2
...ok, x86

Woof builds a Puppy based on the binary packages from another distro.
We sometimes refer to this as the "compat-distro".

1 debian
2 devuan
3 slackware
4 ubuntu
Type number of compat-distro: 4
...ok, ubuntu

The compat-distro usually has release versions
Choose which release you want to obtain the binary packages from.

1 trusty
2 upupbb
3 xenial
Type number of release: 2
ImageLxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#271 Post by wiak »

Hi David and peebee,

I'm not on a Puppy at the moment. If I was, easiest is just to run makpup in gui mode with 'makepup -g' and the drop down will tell you which numbers to use for which distributions for those using commandline mode such as yourself david. However, I think it is:

Code: Select all

-t --target (TARGETARCH): 1 is arm; 2 is x86; 3 is x86_64

-d --distro (COMPATDISTRO): 1 is debian; 2 is devuan; 3 is slackware; 4 is trisquel; 5 is ubuntu

-r --release (COMPATVERSION): 1 is UbuntuPupBB32 or UbuntuTahr64; 2 is UbuntuTahr32 or UbuntuXenial64; 3 is UbuntuXenial32
In which case, for UbuntuPupBB32 you would want (if I've coded correctly? - which is looking doubtful...):

Code: Select all

-t 2 -d 5 -r 1
Or here are the code lines that I use in makepup to find the different combinations:

Code: Select all

    <text><label>$(gettext 'target architecture [-t]: ')</label></text>
	<combobox>
     <variable>TARGETARCH</variable>
     $(combobox_list 2:x86 1:arm 2:x86 3:x86_64)
    </combobox>

    <text><label>$(gettext 'distro base [-d]: ')</label></text>
    <combobox>  
     <variable>COMPATDISTRO</variable>
     $(combobox_list 3:slackware 1:debian 2:devuan 3:slackware 4:trisquel 5:ubuntu)
    </combobox>

    <text><label>$(gettext 'release version [-r]: ')</label></text>
    <combobox>  
     <variable>COMPATVERSION</variable>
     $(combobox_list 3:Slackware14.2 1:DebianStretch 1:DevuanAscii 1:Slackware14.0 1:UbuntuPupBB32 1:UbuntuTahr64 1:TrisquelBelenos 2:Slackware14.1 2:UbuntuTahr32 2:UbuntuXenial64 3:Slackware14.2 3:UbuntuXenial32)
    </combobox>
However, when I upgraded makepup to hopefully find UbuntuPupBB32, I didn't really test it because was in middle of other work, but relied on feedback to tell me if I got it wrong. According to what peebee has written in post above, I seem to have indeed got the code in makepup gui wrong, but my system has run out of space so I have no Puppy on here to test, so please let me know if I have to change the combinations in makepup code and I'll fix it...

EDIT: Just had a look at woof-CE on github and I note that trisquel seems to have been removed, which would change the number ordering. Alas, whoever makes these numeric changes didn't inform me and that ordering is of course required by makepup. Please let me know what current numberings are and I'll alter makepup accordingly (looks like ubuntu in now --distro 4, or maybe trisquel is still there and I've just not noticed it - been a while since I checked).

EDIT: Looks like -t 2 -d 4 -r 2 might indeed be correct and I'll have to modify makepup accordingly. If so, I'll install a pup to a usb in the next day or less and do a quick manual woof-CE puppy build to check the current numeric values needed and modify makepup accordingly - sorry about that...

wiak

User avatar
davids45
Posts: 1326
Joined: Sun 26 Nov 2006, 23:33
Location: Chatswood, NSW

Numbers and Pups

#272 Post by davids45 »

G'day wiak & peebee,

Thanks for your quick and informative replies about my number choices in makepup's config not giving me a BionicPup.

I'll try 2-4-2 and if that's no good, maybe 2-4-1.

Fairly quickly if I watch what's happening on my monitor, I can see the basic Pup being built - if I don't like it, I can stop makepup. Last night I tried 2-5-4 and could see it was still a Xenial Pup being made, so just killed makepup on the terminal, shut down the computer, and went to bed.

Once I get the right Bionic settings, I can try removing default packages I don't want and adding in my preferred ones in one form or another, plus add config files and the like (as .pets) for my desktop.

Then I can try switching kernels to maybe improve boot speed and general Puppy operation.
Incomplete pinboard display during booting is my main new-Puppy bugbear at present, fixed with clicking a pinboard icon linked to a script to restart-X. Maybe I'm loading too many sfs files (with pinboard icons) and have too many partitions (lately, these are very slow to display across the pinboard).

Anyway, makepup should let me try options to improve my particular/peculiar situation (if possible).

Thanks both again.

David S.

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#273 Post by wiak »

Hello David,

Yes, the loss of trisquel changed all the numbers so I'll have to upload a different makepup version. In the meantime, post 2 of this thread (at top) gives some examples of current numeric values that should sort your problems out. Meanwhile, I'll fix the numbers in next release for makepup.

Current numeric values examples in post 2 of this thread here:

http://murga-linux.com/puppy/viewtopic. ... 543#965543

So, currently, for 32bit upupbb, you do indeed need -t 2 -d 4 -r 2

wiak

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

makepup v. 0.1.6 release announcement

#274 Post by wiak »

makepup version 0.1.6 uploaded.

To first post of this thread, here:

http://murga-linux.com/puppy/viewtopic. ... 541#965541

Following Trisquel selection being removed in woof-CE, this new makepup version hopefully fixes numeric values for distribution selection, but needs testing, thanks.

uPupBB32 is currently the default build (i.e. -t 2 -d 4 -r 2)

But please note the following post from PeeBee regarding kernels for uPupBB32:

http://murga-linux.com/puppy/viewtopic. ... 107#994107

EDIT: Note that you do get one popup window during uPupBB32 build (and maybe any Ubuntu?); I can't remember why sorry or if you can just ignore it (please let me know), though I think someone drew attention to this earlier in the thread somewhere... I'd have to study the woof-CE scripts again to remove that popup and don't have time at the moment, sorry. EDIT2: I think now that the popup just vanishes of its own accord if ignored.

EDIT2: I wouldn't recommend building a pup on a usb stick via usb2 interface - that takes forever (very slow), but fine on a hard drive partition in my experience.

wiak

User avatar
davids45
Posts: 1326
Joined: Sun 26 Nov 2006, 23:33
Location: Chatswood, NSW

Manual kernel selection

#275 Post by davids45 »

G'day wiak,

Thanks for taking the time to update makepup to 0.1.6. I'll give it a go or two later today.

Regarding the kernel to be used, is it still true to your recollection that putting the 'huge-kernel' tar.bz desired into the /local-repositories/kernel2add directory will override any other kernel in the config file (or will if I delete any kernel number in the config file line about the kernel to use)?

David S.

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

Re: Manual kernel selection

#276 Post by wiak »

davids45 wrote: Regarding the kernel to be used, is it still true to your recollection that putting the 'huge-kernel' tar.bz desired into the /local-repositories/kernel2add directory will override any other kernel in the config file (or will if I delete any kernel number in the config file line about the kernel to use)?

David S.
Yes, am far as I remember that should override other kernels that appear in the hugekernel list (what you can see in dropdown list when running makepup in gui mode). There is a config file, _00build.conf, where a distro-builder COULD specifically specify a particular kernel that MUST be used: for example debian stretch 32bit build has specified in its _00build.conf:

KERNEL_TARBALL_URL=http://smokey01.com/radky/Woof/kernel-4 ... ch.tar.bz2

and to use a different kernel the builder would first need to comment out that line. But generally distribution creators on woof-CE don't seem to have forced kernels in that manner. For example, in the case of upupbb that KERNEL_TARBALL_URL line is already commented out in its _00build.conf so just drag/dropping new huge kernel into that kernel2add directory provided by makepup should work fine I think.

wiak

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#277 Post by wiak »

makepup 0.1.7 released. Download from first post of this thread.

Hopefully fixes all the numbering to match current woof-CE-testing and woof-CE-master available distros. This needs tested though, and feedback if I'm to fix any errors. I only have rural broadband so proved too slow for me to complete a build as yet - the Pups are getting bigger so taking longer and a lot more bandwidth to make test builds. (Though second time I try later tonight should be fine since makepup allows -k option to keep already downloaded packages).

DPupBuster isn't on woof-CE github yet, so not included in makepup lists for now.

Simple usage is to create empty build directory (on filesystem that has lots of space on it...); put makepup script into that and make it executable with chmod +x; and then run it in gui mode with makepup -g; then uncheck the -k keep previous downloads advanced option, and press "Build your pup!". It takes ages... However, after first build you can rebuild another one without having to download all the distro packages again (just specify the -k keep advanced option when running makepup -g this other time...). So handy for experimenting with different builds. I do not recommend building in usb2 mounted flash drive - probably far too slow for the work woof-CE scripts need to do.

BionicPup64 is currently the default build (i.e. -t 3 -d 2 -r 1)

Now that scottman's package manager program is available (and on woof-CE I see), I believe I could now add a lot more power to makepup operation (future version) in terms of woof building pups very flexibly (assuming the pkg program resolves dependencies). Actually, I could already enhance makepup to do that without needing any upstream mods to woof-CE build scripts since makepup re-writes these scripts for its own purposes on the fly anyway (which is how makepup's pets2add and pkgs2add facility works; however would work much better with dependency resolving cmdline package manager such as scottman's - I would just need to add the facility to use that into makepup, though I am unsure if I want to do any more developing. thinking about it.).

Please let me know if you get round to testing this new makepup since I am unlikely to fix any issues otherwise whilst my own testing is so limited by broadband download speed issues.

wiak

foxpup
Posts: 1132
Joined: Fri 29 Jul 2016, 21:08

#278 Post by foxpup »

wiak wrote: however would work much better with dependency resolving cmdline package manager such as scottman's
That's exactly one of the goals sc0ttman had in mind with pkg!
I love it when a plan comes together ;-) .

User avatar
rerwin
Posts: 2017
Joined: Wed 24 Aug 2005, 22:50
Location: Maine, USA

#279 Post by rerwin »

wiak,
Thank you for continuing to maintain makepup.

Back in January, I experimented with makepup 0.1.6 and encountered a few difficulties, some of which I coded fixes for. You might find them useful additions to the official makepup.

1. I was unable to access pet packages that were xz compressed, so replaced:

tar xzf "${rep}/$dotpet" -C "$rep" --no-anchored pet.specs 2>/dev/null

with:

case $(file -b "${rep}/$dotpet") in
gz*|GZ*) tar xzf "${rep}/$dotpet" -C "$rep" --no-anchored pet.specs 2>/dev/null ;;
xz*|XZ*) tar xJf "${rep}/$dotpet" -C "$rep" --no-anchored pet.specs 2>/dev/null ;;
esac


2. I was uncertain as to when I needed to have pets resident in the local repo, so changed:

Enter y when ready to continue, any other key to quit: '

to:

All user-provided pet packages need to be in /local_repositories/pets2add before proceeding.
All user-provided distro-specific (.deb) packages need to be in /local_repositories/pkgs2add before proceeding.
Enter y when ready to continue, any other key to quit: '


3. I saw some directory names that contained double slashes, so removed the final "/" from:

local rep="../local-repositories/pets2add/"

I did not see any bad effects from removing the slash. I assume the same change should be made to:

local rep="../local-repositories/pkgs2add/"


I am hoping to work with 0.1.7 soon.

Richard

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#280 Post by wiak »

thank you rerwin/Richard. I'll implement your fixes in next version. I have come across another issue though not an 'error' per se. The hugekernel list is simply what the relevant ibiblio webpage provides, but that includes the sha256 and md5 files so it is easy to accidentally select a checksum file rather than the desired hugekernel itself. (As just happened to me in testing...). For now I'm just going to change the default kernel number from 24 to something else since 24 pointed to an sha256 file...

wiak

Post Reply