Auto-build a Puppy iso; single script with optional gui

A home for all kinds of Puppy related projects
Post Reply
Message
Author
User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#81 Post by rufwoof »

A alternative would be to use something like debootstrap, where the official debian can be built (very) quickly with ease ... and then add in puppy functionality (scripts .etc.). Mindful however that Debian consider such to be a Frankendebian when you start importing from third party repositories ... something they advise against. With the correct setup such as localepurge included and only pulling down required, not recommended packages, such installs can be very lightweight (as light as puppy). That's what I do with Debian Jessie, but I don't pull in any puppy script/stuff, I just stick with pure debian main repositories only, and have a desktop that's very puppy like (uses jwm, but whilst I could use rox I prefer pcmanfm ...etc.), but absent the puppy programs (official alternatives choices instead). Other scripts/programs could be added via sfs's, perhaps even just being temporarily loaded for the duration of usage, but unloaded prior to any updating (so the core remains pure Debian). i.e. I could add Puppy scripts to that, but prefer not to as for the things I do there are official Debian equivalents that avoid creating a Frankendebian.

Debdog Stretch is using a unofficial version of that debootstrap type build, but induces the risk that you're pulling in stuff from a third party unapproved repository where those downloads are being run with root level authority i.e. a wide open gate. IMO the official debootstrap approach along with adding puppy sfs's/functionality that can be unloaded as/when desired is the better choice of Frankendebian if you must go down that route.

As a guide to size, 1GB uncompressed is about the size of things, that compresses down to close to a quarter of that. Add in additional programs however and 2GB uncompressed is about the size of things by the time you've added in the likes of video editing ....etc. That compares reasonably with a Puppy that might be 250MB core system including devx and zdrv, that expands to also have a savefolder and additional sfs's for the likes of video editing ...etc. included. Personally I've become much less concerned about size as disk space is inexpensive and relative to ram capacity the difference between 250MB and 500MB is pretty much lost in a corner of a 2GB ram system.

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#82 Post by wiak »

rufwoof wrote:A alternative would be to use something like debootstrap, where the official debian can be built (very) quickly with ease ... and then add in puppy functionality (scripts .etc.).
...
Debdog Stretch is using a unofficial version of that debootstrap type build, but induces the risk that you're pulling in stuff from a third party unapproved repository
Yes, but this thread is not attempting to persuade Puppy stewards to give up Puppy woof-CE build mechanisms and rely on Debian for debootstrap and so on. That would no longer be Puppy, despite having same or similar characteristics or functionality. I want to see woof-CE itself actively developed such that there remains a unique Puppy distribution for the years to come. Otherwise we just forget about Puppy and use Debian in some shape or form and we forget about makepup (which is concerned with woof-CE) and just use the likes of mklive-stretch... But that's up to users to decide what they want to use really so best left for discussion elsewhere. Personally I'm growing more impressed with woof-CE, despite any issues, and we have to remember that it can be molded to build Puppy distributions based on many upstream distribution repositories, or for repositories of its own - very flexible actually and seems to basically work well - just needs some work on making it easy to update the build recipes IMO.

EDIT: And by the way, I've been using DebianDog based distributions for years and am well-versed in its plus and minus aspects (it can be run as a non-root user of course). It's a nice wee system, though yes its easy enough to mold standard Debian too - but... nice to keep having alternatives like Puppy hence me putting much more effort into Puppy/woof-CE at the moment and pleasantly surprised with what I'm finding. Tons of potential remain in Puppy building - DebianDog, just a Debian spin at the end of the day really... Something inspiring about working on a distribution that is NOT dependent on something like underlying debootstrap scripting system. Anyway, I'm in both camps, so not meaning anything 'political'.
wiak

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#83 Post by rufwoof »

wiak wrote:I want to see woof-CE itself actively developed such that there remains a unique Puppy distribution for the years to come. Otherwise we just forget about Puppy and use Debian in some shape or form and we forget about makepup (which is concerned with woof-CE) and just use the likes of mklive-stretch...
+1

The likes of Debdog and mklive-stretch neither add to Debian nor Woof-CE, would be better if they did either one or the other (Debian Live project could do with work, as might woof-ce). Instead they're neither one nor the other and just a standalone that as-is shouldn't be here nor on the Debian forums, but have its own forum as per other distros such as Tinycore. Now add the likes of mklive.... (and your own makepup) to Woof-CE such that its reviewed and controlled as a part of Puppy and that's a whole different situation. A Woof-CE mklive would be a considerable Puppy enhancement IMO that likely would be the first choice by many over makepup or standard 1, 2, 3 ... alternative type Puppy building.

belham2
Posts: 1715
Joined: Mon 15 Aug 2016, 22:47

#84 Post by belham2 »

rufwoof,

If it is really you----which is a large question indeed. Why is it such a big question? Several months back the "real" rufwoof came on here, to puppylinux, and out of the blue (after posting to this forum for years) asked to have all your threads permanently closed and also to have your murga id erased. You stated you'd been hacked in those threads. And that you wouldn't be posting here anymore.

Suddenly, these past 5-6 weeks, you've started posting again, and completely "unlike" the old rufwoof, have taken to sniping and attacks at various things & various people, almost coyly, sly-like---especiallyat some like Fred & his Ddogs, someone who the old rufwoof considered, given the body of posts over the years, a good friend.

And now this crap you are stating here, in the above thread. It is such narrow-minded, atypical crap that if confirms to me that you are not the original rufwoof. If it is you by some chance, you've must have had some massive accident, stroke, badluck or whatever to be posting like this & have been.

Murga is NOT a zero sum game, quit acting like it is and ever was (as Barry directly long ago said---he purposely wanted this forum to flower in all directions when he was running it---indeed, he is back himself flowering in new directions with quirkies & easy linux.

Whomever you are, please stop with the sniping posts (like snidely talking about William going for one of his "walkabouts"---that was real classy, man, real classy), quit acting like somebody that the old rufwoof used to attack for doing this very exact same thing. Don't act all euridite either and try to take the high-road---you know what you've been doing recently, pissing up and stirring up the pot where it is not welcome.

Murga forums, in general, benefit from everything that is going on here, especially the growth of debiandogs and also (whenever it comes again) the next growth phase of puppyland. It's just unreal....if I had the power, and Flash ok-ed it, I would nip this in the bud now and I'd ban you from this forum in a heartbeat. The constant sniping, chipping away at things and others, serves no purpose except your own.

Rational discussion, right here, right now, and in several instances over the past weeks, is not what you're doing and/or have been doing........what the he!! has happened to you? I remember the old rufwoof, I remember when he helped me many times, when he created his huge 2GB first creation and I downloaded it, us both chuckling at not only the size but some of the fun stuff in how it worked. Damn, I miss that person. And I wish the real rufwoof would come back and reclaim what is his instead of witnessing this stuff we do now from someone who is trying to act like him.

jd7654
Posts: 296
Joined: Mon 06 Apr 2015, 16:10

#85 Post by jd7654 »

wiak wrote:...Puppy forum members need to focus more on helping out with woof-CE in my humble opinion...
That's quite a statement. Could you elaborate a bit on how you think Puppy forum members can be conforming to your expectations?

Also, in your opinion, what is the purpose of woof-CE? Is it a Puppy distro builder/developer tool, or an end-user Puppy construction set type product in and of itself? And to which audience is your script targeting?

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#86 Post by rufwoof »

belham2 wrote:rufwoof,

If it is really you----which is a large question indeed. Why is it such a big question? Several months back the "real" rufwoof came on here, to puppylinux, and out of the blue (after posting to this forum for years) asked to have all your threads permanently closed and also to have your murga id erased. You stated you'd been hacked in those threads. And that you wouldn't be posting here anymore.

Suddenly, these past 5-6 weeks, you've started posting again, and completely "unlike" the old rufwoof, have taken to sniping and attacks at various things & various people, almost coyly, sly-like---especiallyat some like Fred & his Ddogs, someone who the old rufwoof considered, given the body of posts over the years, a good friend.

And now this crap you are stating here, in the above thread. It is such narrow-minded, atypical crap that if confirms to me that you are not the original rufwoof. If it is you by some chance, you've must have had some massive accident, stroke, badluck or whatever to be posting like this & have been.

Murga is NOT a zero sum game, quit acting like it is and ever was (as Barry directly long ago said---he purposely wanted this forum to flower in all directions when he was running it---indeed, he is back himself flowering in new directions with quirkies & easy linux.

Whomever you are, please stop with the sniping posts (like snidely talking about William going for one of his "walkabouts"---that was real classy, man, real classy), quit acting like somebody that the old rufwoof used to attack for doing this very exact same thing. Don't act all euridite either and try to take the high-road---you know what you've been doing recently, pissing up and stirring up the pot where it is not welcome.

Murga forums, in general, benefit from everything that is going on here, especially the growth of debiandogs and also (whenever it comes again) the next growth phase of puppyland. It's just unreal....if I had the power, and Flash ok-ed it, I would nip this in the bud now and I'd ban you from this forum in a heartbeat. The constant sniping, chipping away at things and others, serves no purpose except your own.

Rational discussion, right here, right now, and in several instances over the past weeks, is not what you're doing and/or have been doing........what the he!! has happened to you? I remember the old rufwoof, I remember when he helped me many times, when he created his huge 2GB first creation and I downloaded it, us both chuckling at not only the size but some of the fun stuff in how it worked. Damn, I miss that person. And I wish the real rufwoof would come back and reclaim what is his instead of witnessing this stuff we do now from someone who is trying to act like him.
I am polite and helpful when appropriate, head-on aggressive when appropriate. You want to look back at the insults that were thrown at me, especially by little dick, before spouting your own stir-up crap. You also have your facts wrong - I was not hacked in those threads. You may feel it acceptable for a non Puppy to be extensively discussed in the forums Puppy Projects section. I don't. Fine, accept that we disagree.

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#87 Post by rufwoof »

jd7654 wrote:what is the purpose of woof-CE? Is it a Puppy distro builder/developer tool, or an end-user Puppy construction set type product in and of itself?
Isn't the wikka in comparing to remastering suggestive of woof-ce being a end-user tool?

http://puppylinux.org/wikka/WoofCE
Woof CE is the build system for Puppy Linux. It allows a Puppy to be built from the ground up. Compared to remastering it allows much more flexibility and will include all the latest improvement.

Sailor Enceladus
Posts: 1543
Joined: Mon 22 Feb 2016, 19:43

Re: Success!

#88 Post by Sailor Enceladus »

davids45 wrote:Just the common problems in woof-CEs due to missing libs (e.g. libpng.so.12 in slacko)
Hi David,

Just curious, which programs in Slacko 14.2 require libpng.so.12? From what I've seen they all work ok with the included one, I think libpng.so.16.... or maybe 14. I did notice that libpng.so.12 was manually added to tahrpup or xenialpup or debianpup for something though. Is it for an older SFS or PET you've added to slacko maybe?

jd7654
Posts: 296
Joined: Mon 06 Apr 2015, 16:10

#89 Post by jd7654 »

rufwoof wrote:Isn't the wikka in comparing to remastering suggestive of woof-ce being a end-user tool?

http://puppylinux.org/wikka/WoofCE
Not necessarily. I think remastering is definitely an end-user tool, to easily customize their setup permanently to their liking in the base sfs.

What the wiki and other posts about woof-CE were trying to say about remastering, was for budding Puppy developers who were using remastering to customize a Puppy derivative or Puplet for use by others, to instead do it more properly using woof-CE.

To try and use woof-CE as an end-user tool as some sort of pseudo-distro-updater, that runs contrary to what Puppy is. A small, lightweight, easy to use, carefully crafted, self contained, complete OS. Will someone on a old computer or netbook be spending gobs and gobs of processing, storage and hours of time just to update their tiny 200MB Puppy with woof-CE? A full distro or rolling distro might be better in that case.

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#90 Post by wiak »

jd7654 wrote:
wiak wrote:...Puppy forum members need to focus more on helping out with woof-CE in my humble opinion...
That's quite a statement. Could you elaborate a bit on how you think Puppy forum members can be conforming to your expectations?

Also, in your opinion, what is the purpose of woof-CE? Is it a Puppy distro builder/developer tool, or an end-user Puppy construction set type product in and of itself? And to which audience is your script targeting?
woof-CE is a build system for creating Puppy Linux distributions using repositories from other distributions. Anyone can use it. makepup is just my own frontend, which anyone is welcome or not to use. I started using Puppy long time ago, but for a few years have used Dogs more and that's a great system as is the recent mklive-stretch build system. I don't want or need any one conforming to my personal opinions, but I do hope we will be careful to protect the development of Puppy Linux itself since it would be a shame to lose it. I have been surprised by the amount of makepup downloads so I guess quite a few are using it (and thus woof-CE) which is gratifying in terms of Puppy woof-CE future.

Anyway, I prefer this thread to focus on its purpose, which is makepup and woof-CE and not anything about the excellent Dogs of Fred, Toni and others (including the very few contributions of my own).

wiak

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#91 Post by wiak »

jd7654 wrote:
To try and use woof-CE as an end-user tool as some sort of pseudo-distro-updater, that runs contrary to what Puppy is. A small, lightweight, easy to use, carefully crafted, self contained, complete OS. Will someone on a old computer or netbook be spending gobs and gobs of processing, storage and hours of time just to update their tiny 200MB Puppy with woof-CE?
I'm sorry jd7654, but I don't agree with your first or last statement here at all. Personally I agree entirely with rufwoof about the usefulness of being able to keep a woof-CE-built puppy up-to-date (if that proves possible). And doing so, as we are demonstrating, uses very little bandwidth, or even processing speed or storage. An update seems to take less than 15min on my 9 year old laptop (and slow rural broadband with less than 500kB/s download speeds). One of the worries with Puppy is in fact the difficulty in keeping it up-to-date with security fixes (no apt-get update here) and surely that is important to most people? There is also the woof-CE frontend called woof-gui available by the way - originally written by BarryK for end-user use it appears and updated/maintained by jlst who does most of the woof-CE maintenance at the moment.

I don't agree with rufwoof's opinion about including mklive/debootstrap into Puppy (i.e. Debianising Puppy itself), at least not if that meant traditional Puppy was abandoned and it became just another Debian system. Could be a bit of both I suppose (maybe that's what rufwoof means). But it would be great to see mklive DebianDog developers also becoming interested in Puppy/woof-CE itself - seen some signs of that from Toni actually. Cross-sharing of utilities is useful too but its the build systems that are complicated and need a sufficiently-sized development team to maintain longterm.

Anyway, I would prefer if such discussions were held elsewhere because they are clogging up a thread whose purpose is development of makepup. By all means question its point or purpose in another thread or just ignore it all together. I doubt that its creation is doing any harm to you or anyone else.

wiak

jd7654
Posts: 296
Joined: Mon 06 Apr 2015, 16:10

#92 Post by jd7654 »

wiak wrote:... as we are demonstrating, uses very little bandwidth, or even processing speed or storage. An update seems to take less than 15min on my 9 year old laptop
Oh, but you're forgetting the 5GB of storage for complete woof-CE directory, and associated processing time. As an end user, why have a tiny 200MB OS if it needs 5GB extra storage to keep available just to be able to do updates? Well, you can update Puppy in less than 1 minute without woof-CE using far less storage in a personal file.

Anyway, I think your automation wrapper can be useful for woof-CE developers in some cases. But it's also good to consider the target audience and purpose before one begins at coding.

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#93 Post by wiak »

jd7654 wrote:
wiak wrote:... as we are demonstrating, uses very little bandwidth, or even processing speed or storage. An update seems to take less than 15min on my 9 year old laptop
Oh, but you're forgetting the 5GB of storage for complete woof-CE directory, and associated processing time. As an end user, why have a tiny 200MB OS if it needs 5GB extra storage to keep available just to be able to do updates?
You keep it on a what is extremely cheap 8GB usb flash stick if you have that storage problem - times have changed, no dialup nowadays and storage cheap as pie. It all works fine on my old Pentium M, 1GB RAM, broken harddrive machine and I don't mind who finds it useful or uses it or not. It's just a program I'm writing and posting up here for interested users.

In terms of bandwidth - next time you watch Netflix or any online movie you better switch off cos you are using far more than this when used dozens of times. The 4 or thereabouts GB is in any case only needed for decompression of archives really (which includes temp storage prior to mksquashfs and making iso etc, same as with mklive-stretch by the way), not for the archive storage itself, not that it matters, since it is certainly not a measure of the archive download size, which is hardly more than that for many a typical mid-sized iso albeit done file by file (my local repository archive is around 500MB including the 80M huge kernel file).

If you think I can manage to download huge files on my rubbish broadband connection (often as little as 200kB/s download rate and capped at 80GB max download per month) you are dreaming! It's not as if debiandog mklive scripts use much less (storage/archive/bandwidth-wise), and they seem to be pretty popular, with you, me and others.

Actually uploading/downloading pre-made isos is a bigger waste of bandwidth than using a build system like makepup. Puppy does not have any good update mechanism built in so basically the iso becomes insecure over time and only way out is to download a whole new version. Using makepup to simply automatically update any files that change is much less resource/bandwidth intensive. Yes, it is possible to keep personal eyes out for some packages needing security updates sometimes and fix that manually and remaster - but that's a completely useless method overall when dozens of packages get security updates constantly over time (and often, the very package that needs a critical update cannot be easily updated on a running system).

Anyway, moving on with my code now, useful or otherwise.

wiak

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#94 Post by wiak »

makepup ver 0.0.3 uploaded.

Change log extract:

## 0.0.3 Added -R/--REBUILD option, which forces rebuild (post-install scripts etc) of ALL_PACKAGES rather than default CHANGED_ONLY
# Fixed -k/--keep option so the branch doesn't need to be re-downloaded if this commandline switch is given

Usage as before:

Code: Select all

./makepup --help
for brief usage help.

1. By default, makepup will now only download and rebuild changed upstream distribution packages (based on what is already cached in the automatically build woof-CE local-repositories directory).

2. If you want to rebuild ALL_PACKAGES, use commandline argument -R (capital R)

3. If you want to re-download and rebuild all packages, simply delete your (automatically pre-created) woof-CE local-repositories directory and run:

Code: Select all

./makepup
(no commandline args required here if you don't want them)

4. If you want to do a rebuild without re-downloading an already existing woof-CE-<branch> directory, use commandline argument -k/--keep:

Code: Select all

./makepup -k
After an initial build has been made with makepup, subsequent rebuilds take less than 15 minutes on my Intel Core II Duo old laptop (2GHz CPU with 2 GB RAM). If you use the --keep (-k) argument at that stage the only downloads made are of the upstream package lists (to allow check for any package changes/updates) - the default makepup will automatically download any updated upstream distro packages and rebuild them, hence the process is very quick and simple. You can test this action by, for example, simply deleting a package or two in your (auto-built) woof-CE local-repositories pet or tgz directories; then re-run:

Code: Select all

./makepup -k

You can then watch (via the terminal output) as these deleted packages only are re-downloaded and rebuilt followed by the frugal install files (in sandbox3/build) and iso (in woof-output-<distroversion>) being rebuilt automatically.

wiak

User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

#95 Post by saintless »

The wget line doesn't work for me as it is. I suspect it is ISP depending issue:

Code: Select all

Enter y to continue, any other key to quit: y
--2017-09-02 11:29:44--  https://github.com/puppylinux-woof-CE/woof-CE/archive/rationalise.zip
Resolving github.com (github.com)... 192.30.253.112, 192.30.253.113
Connecting to github.com (github.com)|192.30.253.112|:443... connected.
ERROR: cannot verify github.com's certificate, issued by `/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended Validation Server CA':
  Unable to locally verify the issuer's authority.
To connect to github.com insecurely, use `--no-check-certificate'.
unzip:  cannot find or open rationalise.zip, rationalise.zip.zip or rationalise.zip.ZIP
Adding --no-check-certificate still fails because the download is missing .zip at the end:

Code: Select all

Enter y to continue, any other key to quit: y
--2017-09-02 11:41:43--  https://github.com/puppylinux-woof-CE/woof-CE/archive/rationalise.zip
Resolving github.com (github.com)... 192.30.253.112, 192.30.253.113
Connecting to github.com (github.com)|192.30.253.112|:443... connected.
WARNING: cannot verify github.com's certificate, issued by `/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended Validation Server CA':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/puppylinux-woof-CE/woof-CE/zip/rationalise [following]
--2017-09-02 11:41:43--  https://codeload.github.com/puppylinux-woof-CE/woof-CE/zip/rationalise
Resolving codeload.github.com (codeload.github.com)... 192.30.253.120, 192.30.253.121
Connecting to codeload.github.com (codeload.github.com)|192.30.253.120|:443... connected.
WARNING: cannot verify codeload.github.com's certificate, issued by `/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 11582400 (11M) [application/zip]
Saving to: `rationalise'

100%[======================================>] 11,582,400   773K/s   in 16s     

2017-09-02 11:42:00 (722 KB/s) - `rationalise' saved [11582400/11582400]

unzip:  cannot find or open rationalise.zip, rationalise.zip.zip or rationalise.zip.ZIP.

ls
makepup.conf  makepup003  makepup_extra.conf  rationalise
I had the same problem with sfs-get in DebianDog while modifying the script to download the modules from github.
This is what I did in sfs-get and works in your script for me:

Code: Select all

wget -c --no-check-certificate "https://github.com/puppylinux-woof-CE/woof-CE/archive/${woofclonefile}" -O ${woofclonefile}
Then it works:

Code: Select all

ls
makepup.conf  makepup_extra.conf
makepup003    rationalise.zip     woof-CE-rationalise
Maybe useful for others having the same problem.

Toni

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#96 Post by wiak »

Thanks Toni,

I once had that problem when using from another Puppy host and --no-check-certificate worked for me back then, but I didn't need to use the more complete line you suggest. I will leave main script as is for now, in case it is more secure, but will bear your comment in mind.

EDIT: I don't understand quite what you mean when you say the download was missing .zip at the end when wget used with --no-check-certificate though? I don't have that problem when tried that in the past.

Certainly, woofclonefile variable in makepup contains .zip at the end the way it has been created. So, for some reason, the ISP you are using removes the .zip from the downloaded file itself?

wiak

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#97 Post by wiak »

Okay, Toni, changed my mind and re-uploaded in case others have that problem you mentioned:

makepup ver 0.0.4 uploaded.

Change log extract:

## 0.0.3 Added -R/--REBUILD option, which forces rebuild (post-install scripts etc) of ALL_PACKAGES rather than default CHANGED_ONLY
# Fixed -k/--keep option so the branch doesn't need to be re-downloaded if this commandline switch is given
## 0.0.4 Per issue saintless had I used wget with --no-check-certificate and forced output file name with -O

Usage as before:

Code: Select all

./makepup --help
for brief usage help.

1. By default, makepup will now only download and rebuild changed upstream distribution packages (based on what is already cached in the automatically build woof-CE local-repositories directory).

2. If you want to rebuild ALL_PACKAGES, use commandline argument -R (capital R)

3. If you want to re-download and rebuild all packages, simply delete your (automatically pre-created) woof-CE local-repositories directory and run:

Code: Select all

./makepup
(no commandline args required here if you don't want them)

4. If you want to do a rebuild without re-downloading an already existing woof-CE-<branch> directory, use commandline argument -k/--keep:

Code: Select all

./makepup -k
After an initial build has been made with makepup, subsequent rebuilds take less than 15 minutes on my Intel Core II Duo old laptop (2GHz CPU with 2 GB RAM). If you use the --keep (-k) argument at that stage the only downloads made are of the upstream package lists (to allow check for any package changes/updates) - the default makepup will automatically download any updated upstream distro packages and rebuild them, hence the process is very quick and simple. You can test this action by, for example, simply deleting a package or two in your (auto-built) woof-CE local-repositories pet or tgz directories; then re-run:

Code: Select all

./makepup -k

You can then watch (via the terminal output) as these deleted packages only are re-downloaded and rebuilt followed by the frugal install files (in sandbox3/build) and iso (in woof-output-<distroversion>) being rebuilt automatically.

wiak

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#98 Post by rufwoof »

wiak wrote:EDIT: I don't understand quite what you mean when you say the download was missing .zip at the end when wget used with --no-check-certificate though? I don't have that problem when tried that in the past.
Not missing, but other key/values appended as well, something like a downloaded filename of rationalize.zip?id=95923_324853&st=34258902386. i.e a secure link (certificates) downloaded filename with key/values appended. Specifying the output filename with a -O parameter ensures its stored as only the desired filename "rationalise.zip" for instance.

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#99 Post by rufwoof »

wiak wrote:...Anyway, moving on with my code now, useful or otherwise
USEFUL !!!

Consider for instance Debian oldstable (currently Jessie) which is my own personal preferred choice of stable installation/primary boot. Works well on older kit as its been out for years, through development, testing and release for typically 2 to 4 years. As you know Debian's 'stable' doesn't mean "rock-solid", but rather "unchanging". oldstable as such is both unchanging (changes only occur for security fix/critical issues purposes) and rock-solid (having been out in general release for 2+ years). woof-ce set to point to that repository would be pulling down programs that were perhaps better suited to a typical puppy user (older kit) ... and pulling in recent puppy progs/libs/scripts. A single click mechanism such as makepup to build that is a very useful feature. More so if a dedicated woof-ce repository were used for such purpose rather than 'testing' or 'rationalize'. The sort of thing Pelo keeps banging on about, a choice that is similar to a release version that gets updated relatively infrequently for security issues in the main Debian files, with controlled puppy files/versions also pulled in if needed/desired (assuming someone or some mechanism were in place to manage that) and that a single click enabled a Puppy to be upgraded to the latest snapshot of that (security fixes etc.).

Having both makepup and mklive within woof-ce is a great enhancement to Puppy, even though those might be predominately used by end users rather than by developers.

User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

#100 Post by saintless »

wiak wrote:EDIT: I don't understand quite what you mean when you say the download was missing .zip at the end when wget used with --no-check-certificate though? I don't have that problem when tried that in the past.

Certainly, woofclonefile variable in makepup contains .zip at the end the way it has been created. So, for some reason, the ISP you are using removes the .zip from the downloaded file itself?
Yes, in this case the download removes .zip for some reason. Here is the output removing "-O ${woofclonefile}:

Code: Select all

root@debian:/initrd/mnt/dev_ro2/MK-PUP# ./makepup003

NOTE WELL!!! Unless option -k[--keep] is
included on the commandline, any existing
woofbranch clone directory will be deleted as 
will any previously downloaded woof-branch zip file.
(To save download bandwidth, any local-repositories
or woof-out_* directories are currently NOT deleted).

Enter y to continue, any other key to quit: y

Are you sure?!
Enter y to continue, any other key to quit: y
--2017-09-03 12:04:36--  https://github.com/puppylinux-woof-CE/woof-CE/archive/rationalise.zip
Resolving github.com (github.com)... 192.30.253.113, 192.30.253.112
Connecting to github.com (github.com)|192.30.253.113|:443... connected.
WARNING: cannot verify github.com's certificate, issued by `/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert 
SHA2 Extended Validation Server CA':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/puppylinux-woof-CE/woof-CE/zip/rationalise [following]
--2017-09-03 12:04:36--  https://codeload.github.com/puppylinux-woof-CE/woof-CE/zip/rationalise
Resolving codeload.github.com (codeload.github.com)... 192.30.253.120, 192.30.253.121
Connecting to codeload.github.com (codeload.github.com)|192.30.253.120|:443... connected.
WARNING: cannot verify codeload.github.com's certificate, issued by `/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=
DigiCert SHA2 High Assurance Server CA':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: `rationalise'

    [   <=>                                 ] 11,582,400   784K/s   in 15s     

2017-09-03 12:04:53 (751 KB/s) - `rationalise' saved [11582400]

unzip:  cannot find or open rationalise.zip, rationalise.zip.zip or rationalise.zip.ZIP.
root@debian:/initrd/mnt/dev_ro2/MK-PUP# ls -l
total 11368
-rw-r--r-- 1 root root      198 Sep  3 12:04 makepup.conf
-rwxr-xr-x 1 root root    14106 Sep  3 12:03 makepup003
-rwxr-xr-x 1 root root    14216 Sep  3 12:02 makepup004
-rw-r--r-- 1 root root      568 Sep  3 12:04 makepup_extra.conf
-rw-r--r-- 1 root root 11582400 Sep  3 12:04 rationalise
root@debian:/initrd/mnt/dev_ro2/MK-PUP# 
It is saved as rationalise only.

But not always. Some links to google drive and github can't be downloaded because very long file name
(maybe caused by the untrusted certificate issue). Here is example that doesn't download sfs.txt:

Code: Select all

root@debian:~# wget --no-check-certificate https://github.com/DebianDog/Jessie/releases/download/v2.0/sfs.txt
--2017-09-03 12:12:49--  https://github.com/DebianDog/Jessie/releases/download/v2.0/sfs.txt
Resolving github.com (github.com)... 192.30.253.112, 192.30.253.113
Connecting to github.com (github.com)|192.30.253.112|:443... connected.
WARNING: cannot verify github.com's certificate, issued by `/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert 
SHA2 Extended Validation Server CA':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/33422404/5ad1ebac-9e1e-11e6-9c9e-
c27e9a17c38a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20170902%2Fus-east-
1%2Fs3%2Faws4_request&X-Amz-Date=20170902T131249Z&X-Amz-Expires=300&X-Amz-Signature=
7fcb5110b41fb2b8585dc0a52bb0df8f52d7a3917cf0601d23ad4b52a7f2376b&X-Amz-SignedHeaders=
host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dsfs.txt&response-content-type=
application%2Foctet-stream[following]
--2017-09-03 12:12:49--  https://github-production-release-asset-2e65be.s3.amazonaws.com/33422404/5ad1ebac-9e1e-
11e6-9c9e-c27e9a17c38a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20170902%2Fus-east
-1%2Fs3%2Faws4_request&X-Amz-Date=20170902T131249Z&X-Amz-Expires=300&X-Amz-Signature=7fcb5110b41fb2b8585dc0a52bb
0df8f52d7a3917cf0601d23ad4b52a7f2376b&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment
%3B%20filename%3Dsfs.txt&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.
amazonaws.com)... 52.216.81.96 Connecting to github-production-release-asset-2e65be.s3.amazonaws.com
 (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.81.96|:443... connected.
WARNING: cannot verify github-production-release-asset-2e65be.s3.amazonaws.com's certificate, issued by 
`/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Baltimore CA-2 G2':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 303 [application/octet-stream]
5ad1ebac-9e1e-11e6-9c9e-c27e9a17c38a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=
AKIAIWNJYAX4CSVEH53A%2F20170902%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20170902T131249Z&X-Amz-Expires=
300&X-Amz-Signature=7fcb5110b41fb2b8585dc0a52bb0df8f52d7a3917cf0601d23ad4b52a7f2376b&X-Amz-SignedHeaders=
host&actor_id=0&response-content-disposition=attachment; filename=sfs.txt&response-content-type=
application%2Foctet-stream: File name too long

Cannot write to `5ad1ebac-9e1e-11e6-9c9e-c27e9a17c38a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=
AKIAIWNJYAX4CSVEH53A%2F20170902%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20170902T131249Z&X-Amz-Expires=
300&X-Amz-Signature=7fcb5110b41fb2b8585dc0a52bb0df8f52d7a3917cf0601d23ad4b52a7f2376b&X-Amz-SignedHeaders=
host&actor_id=0&response-content-disposition=
attachment; filename=sfs.txt&response-content-type=application%2Foctet-stream' (Success).
root@debian:~# 
Read the last messages above about file name too long.
And after adding "-O sfs.txt"

Code: Select all

root@debian:~# wget --no-check-certificate https://github.com/DebianDog/Jessie/releases/download/v2.0/sfs.txt -O sfs.txt
--2017-09-03 12:13:27--  https://github.com/DebianDog/Jessie/releases/download/v2.0/sfs.txt
Resolving github.com (github.com)... 192.30.253.112, 192.30.253.113
Connecting to github.com (github.com)|192.30.253.112|:443... connected.
WARNING: cannot verify github.com's certificate, issued by `/C=US/O=DigiCert Inc/OU=www.digicert.com/CN
=DigiCert SHA2 Extended Validation Server CA':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/33422404/5ad1ebac-9e1e-11e6-9c9e
-c27e9a17c38a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20170902%2Fus-east-
1%2Fs3%2Faws4_request&X-Amz-Date=20170902T131328Z&X-Amz-Expires=300&X-Amz-Signature=03cb82d7f238b1a5432687
2bbe9f3441518bb40ed9a65584b5210ef4aa03bf64&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=
attachment%3B%20filename%3Dsfs.txt&response-content-type=application%2Foctet-stream
 [following]
--2017-09-03 12:13:28--  https://github-production-release-asset-2e65be.s3.amazonaws.com/33422404/5ad1ebac-
9e1e-11e6-9c9e-c27e9a17c38a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20170
902%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20170902T131328Z&X-Amz-Expires=300&X-Amz-Signature=03cb82d7
f238b1a54326872bbe9f3441518bb40ed9a65584b5210ef4aa03bf64&X-Amz-SignedHeaders=host&actor_id=0&response-
content-disposition=attachment%3B%20filename%3Dsfs.txt&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.
amazonaws.com)... 52.216.81.88
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-
2e65be.s3.amazonaws.com)|52.216.81.88|:443... connected.
WARNING: cannot verify github-production-release-asset-2e65be.s3.amazonaws.com's certificate, issued by
 `/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Baltimore CA-2 G2':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 303 [application/octet-stream]
Saving to: `sfs.txt'

100%[======================================>] 303         --.-K/s   in 0s      

2017-09-03 12:13:29 (1.27 MB/s) - `sfs.txt' saved [303/303]

root@debian:~# ls -l
total 7
-rw-r--r-- 1 root root  20 Feb 11  2014 DirFileList.path
drwxr-xr-x 2 root root 108 Aug 21  2015 Startup
lrwxrwxrwx 1 root root  25 Mar 22  2016 desktop.jpg -> /opt/docs/pictures/02.jpg
-rw-r--r-- 1 root root  13 Mar 27  2014 mod-list
-rw-r--r-- 1 root root 303 May 25 12:38 sfs.txt
lrwxrwxrwx 1 root root  17 Mar 24  2016 startup -> /root/startup-jwm
-rwxr-xr-x 1 root root 663 Jul 18  2014 startup-icewm
-rwxr-xr-x 1 root root 616 Jul 18  2014 startup-jwm
root@debian:~# 
Toni

Post Reply