DebianDog - Jessie (21 June 2017)

[fredx181]

Hi rufwoof,

Can you check if the attached remaster script is alright for you? (I modified a little)
I added check for if porteus-boot is in use or not at the beginning, and at the end changed wmreboot to reboot (see fredx181 comments)
I like to add it as Menu entry with name something like "Quick-Remaster" but that's up to you how to name it.
Later I will check with you the changes required for it in linuxrc.
Maybe it's good at some point in the script to check if filesystem.module exists, if not, do something like: "echo filesystem.squashfs > filesystem.module", just a thought.

Fred

[rufwoof]

A neat feature of boot 3 is everything is self contained, the same initrd whether you're booting a full install or a frugal install. My LXDE 64 install for instance is installed to the first partition, along with grub4dos menu.lst and also has the Debian boot loader (grub) in /boot ... and that same partition is set to be the persistence (save folder) partition.

With everything in /live/filesystem.squashfs more usually, but I can fully extract that to / and then boot via grub (chained from menu.lst) and it acts as a full install (all changes immediately written to disk, kernel can be updated via apt-get upgrade ...etc.). Then afterwards I can reboot persistence (frugal), run a remaster and all of / (save folder) is moved back into filesystem.squashfs (except menu.lst bootloader ...etc files).

In short you can flip between being a 'full install' to/from being a frugal install using /live/filesystem.squashfs - with / as being the 'save folder'. And that caters for the kernel being upgraded/updated. In contrast, with boot 1, you have to lock (pin) the kernel/initrd.

The only downside is that, unlike boot 1, it doesn't support save to folder, only save to file or save to partition (where the partition can be the same one as where the filesystem and menu.lst are).

Point being ... you can target how involved (potentially driving you mad at one end, to very low effort at the other end) you might want to make DebianDog. At the lightest effort end could be just a couple of scripts, save2flash and single click remaster, applied to a boot 3 setup (as supplied by Debian), perhaps with the core desktop being Openbox/LXDE (as supplied by Debian). At the other end you might target DD as being three different boot styles, two different desktop setups (Openbox and Rox/Jwm) ... or as complicated as you like.

[rufwoof]

Can you check if the attached remaster script is alright for you?

Looks good to me Fred. reboot is a good choice, I edited my wmreboot and commented out the saving choices and splash to equal effect. The Porteus check ... you know better than me anyway, so I'd say its good to go.

My boot 3 remaster is similar in many ways, but it does its own clearing out of the save 'folder' (which is a partition in boot 3 case, and that in my case is shared alongside /live folder and menu.lst .....etc, so its a selective 'clean'). If it were in its own dedicated partition then it could all be wiped. One issue with that however is that depending upon the amount of changes, removing files can lead to a system lockup and having to hard shutdown (hold power button for 5 seconds). The Porteus version is cleaner in that respect (left to initrd to do the cleaning at the next reboot). With boot 3 and accepting kernel/initrd updates from Debian means that cleaning can't be coded into the initrd ... unless you pin that kernel/version.

Code: Select all

#!/bin/bash
BOOT=`cat /proc/cmdline | grep persistence-read-only`
if [ -z "${BOOT}" ]; then
 exit 1
fi

/usr/local/bin/flush2disk # Store latest changes
RetCode=$?
if [ $RetCode -eq 3 ]; then
 echo
 read -p "Continue with remaster (y/n) " -n 1 -r
 if [[ $REPLY =~ ^[Yy]$ ]]
 then
  echo
  echo Remastering without current session changes
 else
  echo
  echo Cancelling remaster
  sleep 5
  exit 3
 fi
fi
echo
sync

LABEL=`cat /proc/cmdline | awk 'BEGIN{FS="persistence-label="} {print $2}' | awk 'BEGIN{FS=" "} {print $1}'`
BASE=`mount -l | grep "\[$LABEL\]" | grep -m 1 /lib/live/mount/persistence | awk 'BEGIN{FS=" "} {print $3}'`
if [ -z "${BASE}" ]; then
 echo cannot identify installation type - exiting
 sleep 4
 exit
fi
cd $BASE/live
mkdir remaster
cd remaster
mkdir -p tmp1 tmpa
# read filesystem.module to determine which of the two toggles to use
CURRENT=`cat ${BASE}/live/filesystem.module | grep filesystem | grep squashfs`
mount -o loop ${BASE}/live/$CURRENT tmp1/
mount -t aufs -o br:$BASE:tmp1 none tmpa/
CURRENT=`echo $CURRENT | sed s/.squashfs//`
if [ ! -z `echo $CURRENT | grep 01` ]; then
 if [ -f $BASE/live/filesystem.squashfs ]; then
   rm $BASE/live/filesystem.squashfs
 fi
 mksquashfs2 tmpa $BASE/live/filesystem.squashfs -comp lzo -Xcompression-level 1 -e tce live Documents boot lost+found persistence.conf vmlinuz initrd.img grldr menu.lst sdb_mbr.bak debian-usb
 sed -i "s/filesystem01.squashfs/filesystem.squashfs/" $BASE/live/filesystem.module
else
 if [ -f ${BASE}/live/filesystem01.squashfs ]; then
   rm $BASE/live/filesystem01.squashfs
 fi
 mksquashfs2 tmpa $BASE/live/filesystem01.squashfs -comp lzo -Xcompression-level 1 -e tce live Documents boot lost+found persistence.conf vmlinuz initrd.img grldr menu.lst sdb_mbr.bak debian-usb
 sed -i "s/filesystem.squashfs/filesystem01.squashfs/" $BASE/live/filesystem.module
fi
sync
umount -f tmpa tmp1
rmdir tmpa tmp1
cd ..
rmdir remaster

cd $BASE
rm -rf root etc .wh* .gtk* lib usr var .Trash* bin dev lib32 lib64 live-build opt media mnt proc home run sbin srv sys tmp
sync

exit 2

BTW flush2disk is similar to save2flash i.e. with your modifications to use rsync instead of copy. I call both via 'start' scripts so that they can be run as su and flush2disk can generate a exit 3 outcome

Code: Select all

#!/bin/bash
read -p "Flush changes recorded in memory to disk : Are you sure? (y/n) " -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]
then
 echo
 sudo flush2disk.sh
 exit 0
else
 exit 3 # can be called by remaster so we include a error # here for that
fi
echo

[rufwoof]

For boot 3 this is a cleaner way to clean out the persistence files after a remaster ... add a systemD service

Create a executable script to do the persistence (save) cleaning, but only if /tmp/cleanpersistence file exists ... mine looks like :

/usr/local/bin/persistence

Code: Select all

#!/bin/bash

if [ -f /tmp/cleanpersistence ]; then
 LABEL=`cat /proc/cmdline | awk 'BEGIN{FS="persistence-label="} {print $2}' | awk 'BEGIN{FS=" "} {print $1}'`
 BASE=`mount -l | grep "\[$LABEL\]" | grep -m 1 /lib/live/mount/persistence | awk 'BEGIN{FS=" "} {print $3}'`
 if [ -z "${BASE}" ]; then
  exit
 fi
 cd $BASE
 rm -rf root etc .wh* .gtk* lib usr var .Trash* bin dev lib32 lib64 live-build opt media mnt proc home run sbin srv sys tmp
fi

Create a systemD service to link that into systemD

/lib/systemd/system/persistence.service

Code: Select all

[Unit]
Description=persistence clean if requested
DefaultDependencies=no
Before=shutdown.target reboot.target halt.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/persistence  

[Install]
WantedBy=halt.target reboot.target shutdown.target

... and enable it

systemctl enable persistence.service

you can check its status is enabled by scanning through

systemctl list-unit-files

that shows all files, or just look for the single file by name

root@debian:/home/user# systemctl list-unit-files persistence.service
UNIT FILE STATE
persistence.service enabled

1 unit files listed.
root@debian:/home/user#


so now all remaster.sh script has to do is create a /tmp/cleanpersistence flag file

touch /tmp/cleanpersistence

... and when the system reboots, halts, shutsdown the persistence.service will fire up /usr/local/bin/persistence ... which cleans out the persistence (save) files/folders

Not sure, but if you edit the /usr/local/bin/persistence script you may have to (???) deactivate and reactivate the persistence.service so that it re-sym-links in that new code

systemctl disable persistence.service
systemctl enable persistence.service

I haven't thoroughly tested it yet - but working well so far for me Leaving deletion of live running files right to the end of shutdown/halt/reboot (i.e. in single user mode rc.0 or rc.6) is less inclined to cause a system lock-up as was the case if the remaster script deleted those files whilst still in multi-user mode.

UPDATE : No luck. For very large updates that still can result in shutdown locking up (needing a hard shutdown).

[anikin]

Hi Everyone,

For boot 3 this is a cleaner way to clean out the persistence files after a remaster ... add a systemD service...

rufwoof,
Is there a hidden meaning in your remastering concept?
I'm thinking at the top of my mind and can't get the gist of it. Please, don't get me wrong, but remastering is a very basic thing. It boils down to the following:
1) You remaster to preserve the changes to the system without *any* save/persistence options.
2) Or, alternatively, you preserve the changes via save/persistence options.
It's that simple, a binary choice: either remaster, or use persistence/save, but don't mix them together. I remaster a lot and use Toni's remastering script, that he created a long time ago at my request (see the attachment). It is extremely simple, but works flawlessly not only on DD but any Debian/Ubuntu live. Fred has a GUI remastering tool, that also works nicely.

[rufwoof]

It's that simple, a binary choice: either remaster, or use persistence/save, but don't mix them together

I want the two mixed together. During single sessions save (or not), periodically merge all of the saved stuff into the main filesystem. On a live running system.

Which means that once the merge save folder (partition) content is saved into the filesystem it can be deleted. The problem is removing that content in a live system as those files are being used at the time. For porteus boot I have it set so that the system reboots and the initrd detects a flag file and cleans out the save folder content before its used - and where that initrd is in effect locked (kernel updates are pinned to not happen). With boot 3 the initrd is in effect inaccessible as Debian could replace that (updates) with versions that wouldn't include that 'delete save folder content if flag file exists'.

[saintless]

Hi all.

First posts messages in all threads are updated with this information:

One month from today (28.09.2016) I will remove all files from:
http://www.kazzascorner.com.au/saintless
Thank you very much for your support Kazza!

I will not update anymore:
http://www.smokey.01.com/saintless
Thank you very much for your support Smokey!

All DebianDog and MintPup files are uploaded at github. Links available in the first page post.

To fredx181 - the creator, developer, maintainer... the Heart and Soul of DebianDog project as I read from mcewanw:
Congratulations for dropping support and updates for all existing Wheezy and Jessie versions and derivatives.
Congratulations for taking others ideas and presenting them as yours. Like "umount /lib/live/mount/overlay" and "usr/sbin/chroot"
Congratulations for not respecting any license (GPL and non-free).
Congratulations for pretending so well you have nothing to do with my decision to leave the forum.

To mcewanw - the "DebianDog History Document project writer":
Thank you sfs - there wouldn't be porteus-boot working with Debian kernel without you, even if I cant find you as source in linuxrc.
Thank you jbv - your ideas and zerosize functions are everywhere in fredx181's scripts, even if I cant find you as source anywhere.
There are many more but we both know fredx181 doesn't like much to give credits or sources in his scripts.
No need to mention me in the future. You already made me look like a tester for his and yours scripts. Just keep it this way.
For your information only: Sickgut, jbv (FoxyRoxyLinux.com) and saintless are three different persons and DebianDog exists and works so well thanks to them.

To rufwoof:
Please, feel free to work on my (Very) far from polished Jwm version. It includes only few years of my life wasted. Just delete them and remaster.

To anikin:
https://github.com/MintPup/DebianDog-Wh ... dog-cli.sh

All the best and good luck with the community project.

Toni

[anikin]

Hi Toni, Fred, William,

Please guys, get immediately in touch with each other via pm, email, etc., to settle any misunderstanding. This is Toni's project. There's absolutely no question or second thought about it. Don't burn bridges, too much is at stake here.

[mcewanw]

Hi Toni, Fred, William,

Please guys, get immediately in touch with each other via pm, email, etc., to settle any misunderstanding. This is Toni's project. There's absolutely no question or second thought about it. Don't burn bridges, too much is at stake here.

Forget it anikin. Both Fred and I independently tried appealing to Toni via PMs. Unfortunately, he thinks we attacked him in some way back at that time he left the forum. That new post says more than I care to even think about again.

Toni certainly started this project, inspired by earlier work by Sickgut and JBV. I certainly did wonder, after Toni left, if he really was Sickgut, but so what - he isn't. Fine. I never said anything ever against Sickgut, who seems to have had great ideas, which alas never seemed to be taken up by main Puppy developers.

Does that mean Fred had nothing to do with the development of DebianDog? As far as I understood it from these earlier DD days, Fred and Toni seemed to be contributing continually in pretty much equal doses, but I couldn't myself care less which one of them, if any, claims overall 'ownership'. They both did a great job with that.

I have never had much myself to do with underlying DebianDog developments - just a few tiny things like chpupsocket or whatever it was called and a bit of work on debdog installer. I just 'tried' to help - my own long interest is just in programming silly wee utility apps and I'm not planning to do any more of that anymore anyway since I'm almost retirement age now, getting slower brain I feel, and want to enjoy my family life in preference to technical matters.

I'm surprised Toni is further upset with the new development work Fred had no choice but to basically now lead. The last I recall, Toni wrote thus:

Hi Fred, William.

I will let others judge which tone is aggressive reading the last pages.
Before this thread gets more and more away from MintPup I prefer to stop here.
I just removed myself from github DebianDog organisation members and it is up to you now what will happen with this organisation page.
All DD versions first pages are linked to github so it is easy to change any information you like. Fred has access to all dd-repositories. I don't mind if you ask Flash to give you access to edit the first pages posts in all threads started by me (I don't know if it is possible).

From this moment I will be active only in mintpup github page and I will probabaly fork DD-Jwm Wheezy and Jessie versions there later

.

I have of course read Toni's comments in his latest post, but truly have nothing I wish to further PM him about and nothing I wish to defend. I know Fred was quite distressed when Toni left the project - the things said back then were somehow depressing, so I just hope this latest doesn't distress him further. Since Sickgut left, JBV is not on this forum that I know of, and Toni left, so Fred had no choice really but to soldier on alone or to let the project die altogether - a lot of it was his work too, not just Toni's or Sickgut's or JBV's. No one creates anything in isolation, so thanks to Sickgut, JBV, Toni, and Fred for that much - I can't say which I would thank the most.

[fredx181]

Please guys, get immediately in touch with each other via pm, email, etc., to settle any misunderstanding. This is Toni's project. There's absolutely no question or second thought about it. Don't burn bridges, too much is at stake here.

Pff... "to settle any misunderstanding"
It's absolutely clear to me that Toni doesn't want to.

Fred

[rufwoof]

To clarify, my unpolished remark about the jwm version was first impression and its significant dis-similarity to puppy style jwm. When I looked deeper I saw the very clever stuff beneath that ... just that it wasn't immediately apparent and hence looked bare to me.

That aside, Toni you're documentation of a limit of 7 sfs's is indeed a loop allocation limit. I've increased that inside initrd (linuxrc, somewhere close to the end of that file) and loaded many more sfs's.

Also you can create/use /live/filesystem.module files under boot 3 to contain the sfs's you want loaded and in the order you want loaded, which can also include .dir folders

filesystem.squashfs
changes.dir
someother.squashfs

and only those will be loaded in that order, even if other .squashfs files are in /live
Remove filesystem.module and it reverts to loading /live/.squashfs files in alphanumeric sorted order.

Sad to see you still angry/upset. I had hoped you might return to share in what personally I find to be a friendly group to bounce ideas off each as I for one found you to be most knowledgeable and helpful. You do however come across as being rather sensitive. I wish you good luck whatever path you decide to take going forward.

[emil]

Dear group,
I seldom post here theses days, what I see makes me sad.
From every contact I had I have to say saintless is one of the most well meaning and helpful persons I ever met online, probably also in real life. He put a lot of work and energy in it, both technically and also socially.

I just want to say: Thank you toni.

posting from debian dog
with kindest regards
emil

[rufwoof]

In follow up to http://murga-linux.com/puppy/viewtopic. ... 841#925841 as attaching a systemd.service to do the clearing out of the 'save' after a remaster can still lead to having to hard-reboot/shutdown, I've dropped that idea and gone with a software induced hard-reboot. That could lead to fsck's at the reboot due to a unclean shutdown, however typically you shouldn't be running other programs during a remaster and after remaster finishes removing files and sync'ing, the system is in a relatively stable state anyway such that hard-reboots more often are ok (given that most things are running in ram anyway).

So the end of my remaster script that cleans out the 'save' space (which is a partition, and the same partition as used by grub4dos and /live) now looks like :

Code: Select all

cd $BASE
rm -rf root etc .wh* .gtk* lib usr var .Trash* bin dev lib32 lib64 live-build opt media mnt proc home run sbin srv sys tmp
sync
echo 1 > /proc/sys/kernel/sysrq
echo b > /proc/sysrq-trigger     # force hard reboot
# echo o > /proc/sysrq-trigger     # force hard shutdown

[backi]

Hi i am feeling just as a guest here .Don`t know the "Drama " behind the curtain .

I am thankful for what all you guys have done ,lots of work...Don`t really know who are all the Masterminds behind this Project : Thanks to you all .

Tony (saintless) always was friendly and helpful when asking for solution .....so are all the other guys ,who are coming here together, sharing their intelligence, work and bread with each other ,can`t complain about anyone .

Can` t see anybody here doing ego trips......I enjoy much being here ...also miss Tony - Saintless genius .

But i find , it is not a fair attitude from Tony,to make accusations (which i can not prove if right or wrong )and not to give fredx or any other any a chance to justify them self , by cutting off any communication .Just my personal thoughts.

I am not an insider here and no doctor.I ask myself ,why does Tony feel so humiliated ?
Tony.. i beg you please ,ask yourself....isn`t this a bit a paranoic overreaction ?
Makes me really sad .

May there be peace and love and perfection throughout all creation, o God.
(John Coltrane)

[anikin]

Forget it anikin. Both Fred and I independently tried appealing to Toni via PMs. Unfortunately, he thinks we attacked him in some way back at that time he left the forum...

... "to settle any misunderstanding"
It's absolutely clear to me that Toni doesn't want to...

It's hard to disagree with you guys, but the worst verbal exchange we've ever seen here was between Toni and yours truly. I regret it, I really do, as well as I'm sure Toni does. That was in the past, I'm talking about the future. The future of this project without Toni is possible. However, without him it will be a flawed, unfair project. I don't think a lot of people will subscribe to it. The only option left is to find a way to restore justice and bring Toni back. Here's one aspect, I'd like you pay attention to: when Toni mentioned competition in that exchange, you failed to see the bigger picture. There was no competition in that particular exchange. But there was and still is competition between the original DD project and DD Porteus. As paradoxical as it may sound, Toni made a huge mistake by allowing Porteus Boot into his project. In the end Porteus Boot just sucked out all the oxygen out of the whole project. That's what, you guys are failing to see. That's what needs to be reconciled. Mind you *reconciled* - not rejected one in favor of the other. Let's go back to square one.

[fredx181]

Don`t know the "Drama " behind the curtain

You are not the only one.
I think nobody knows, except Toni himself.
Can only guess what he means with:
"pretending so well you have nothing to do with my decision to leave the forum", because he didn't say what "I have to do with it".
I only know what happened in the mintpup thread (the discussion about memory usage reported compared in mintpup and xenialdog).
Of course I wasn't the nicest person at some point in that discussion... but I don't think that's his reason only to leave the forum.

Thanks for your support.

Fred

[fredx181]

The only option left is to find a way to restore justice and bring Toni back. Here's one aspect, I'd like you pay attention to: when Toni mentioned competition in that exchange, you failed to see the bigger picture. There was no competition in that particular exchange. But there was and still is competition between the original DD project and DD Porteus. As paradoxical as it may sound, Toni made a huge mistake by allowing Porteus Boot into his project. In the end Porteus Boot just sucked out all the oxygen out of the whole project. That's what, you guys are failing to see. That's what needs to be reconciled. Mind you *reconciled* - not rejected one in favor of the other. Let's go back to square one.

You might have a point, suppose you're right, I think that Toni first needs to open communication again.
Anyway I can live with a DebianDog without porteus-boot.
Probably it's too late, too much has happened.

Fred

[backi]

Yes.... remember this fucking ram usage thing .

In Xenial Dog... Htop showed ram usage on my machine quite high ...so i asked if anybody else could confirm .
Suddenly things went out of control .

Never understood why Tony felt so offended.....
Since that point i feel somehow confused and guilty .
I shouldn`t ....But i do .

Let me steal some Lyrics from Pink Floyd :

How I wish, how I wish you were here.
We're just some lost souls
Swimming in a fish bowl
Year after year
Running over the same old ground.
What have we found
The same old fears.

TONY.... Wish you were here.

I wish Tony would come back ....and we all become friends forever.

Let there be Peace on Earth

[fredx181]

Since that point i feel somehow confused and guilty

Don't!!! period.

Fred

[fredx181]

OK, let's continue with the "community project"
(well, to be honest I really am confused about the future of DD, but can't resist to reply on something I find interesting )

rufwoof,
Is there a hidden meaning in your remastering concept?
I'm thinking at the top of my mind and can't get the gist of it. Please, don't get me wrong, but remastering is a very basic thing. It boils down to the following:
1) You remaster to preserve the changes to the system without *any* save/persistence options.
2) Or, alternatively, you preserve the changes via save/persistence options.
It's that simple, a binary choice: either remaster, or use persistence/save, but don't mix them together. I remaster a lot and use Toni's remastering script, that he created a long time ago at my request (see the attachment). It is extremely simple, but works flawlessly not only on DD but any Debian/Ubuntu live. Fred has a GUI remastering tool, that also works nicely.

I think it's a nice concept that rufwoof has made.
Suppose you choose to save to file/folder several times and at some point you want to make remaster with the changes included, then the advantages are:
- very quick, no copying done, just mountiing using aufs to overlay main filesystem.squashfs + changes (and LZ4 compression is very fast).
- the main filesystem.squashfs will instantly be "replaced" (automatically add new entry in filesystem.module file), no manual copying/replacing required.
- The changes will be deleted (after reboot) so no hassle to do that manually (after all, the changes are already included in the remaster, so should be removed from existing save file/folder).

As it is now it only supports changes folder deletion but I'm working on it to support also deletion of savefile contents (e.g. changes.dat) after remaster.

Fred