Banksy 3 - aka "The Personator"

For talk and support relating specifically to Puppy derivatives
Message
Author
User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#46 Post by greengeek »

Sylvander wrote: It is somewhat slow, but I figured that was because it must load the files from a slow optical disk/drive.
<Yes, optical media loading is slow. If it was loading over a gig of data you would really notice it even more. I was thinking that a cdrw may be more like a DVD in terms of larger data capacity but you have confirmed that the actual content on the cdrw is well less than normal CD capacity anyway.

This suggests that the missing 1.5GB reported by partview is either erroneous or caused by something like the file expansion process after loading CD data, or maybe caused by some artifact of the unusual way banksy is layered. Maybe both. Either way it looks like having a swap partition is necessary during a personalisation process that involves significant addition of data/programs. Less critical once running from the completed personal CD - but I still need to look closer at personal storage calculation and usage.
.
366MB is much smaller than the 8.5GB - 7.1 GB = 1.4 GB of free storage that disappears during the making of the ISO.
There are good reasons why a lot of storage is consumed during that process - the /tmp/banksydistillor directory holds all of the added data ready to build into the new personal sfs, then it is formed into the new sfs which is also in /tmp, then that sfs is copied into an "isobuild" directory along with other necessary files (all of this also in /tmp) then that isobuild directory is compiled into the new iso (yet another large file sitting in /tmp). So you can see a heap of data accumulates in /tmp during the process.

I think I can smarten the process somewhat to make it more thrifty on RAM. More testing to follow. Till then I suggest the USB swap partition trick may be the key.

Also - installing the banksy files to HDD definitely speeds booting (although I know you are not comfortable with that structure...)

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#47 Post by Sylvander »

greengeek wrote:installing the banksy files to HDD definitely speeds booting (although I know you are not comfortable with that structure...)
I like:
a. Banksy running totally in RAM with no use of any writable drive.
b. The fact that the Banksy CD-RW is read-only.

I don't like:
c. Any writable storage media being used during normal use of Banksy.
e.g. Swap partition. [Might a swap partition be infected, and so infect a subsequent Banksy session?]
Might a full install of Banksy [to an internal HDD partition] be infected?

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#48 Post by greengeek »

Sylvander wrote:I like:
a. Banksy running totally in RAM with no use of any writable drive.
b. The fact that the Banksy CD-RW is read-only.
I agree.
I don't like:
c. Any writable storage media being used during normal use of Banksy.
e.g. Swap partition.
Also agree
[Might a swap partition be infected, and so infect a subsequent Banksy session?]
I have read that this is impossible because any code in the swap area becomes useless after shutting down a session. However, swap is writable so I'd have to say that my guess is that there is more risk with it than without it. (Although we have to bear in mind that RAM itself is writable so in that case maybe there is not much difference - unless there is some attack vector that can "collect" tainted data from the previous swap session. If that is possible then I would venture to suggest such complex hacker skills would most likely reside in the hands of nation states such as was revealed by the stuxnet virus - in which case I have no show of protecting my system against that level of penetration)

Having said that I would be happy to take the risk of using the usb swap just for the time it takes to get the personalisation/burn done if the amount of data being added required the extra space during the processing. (I hope to reduce the required space in the next impersonator script).
Might a full install of Banksy [to an internal HDD partition] be infected?
I consider it highly unlikely because the two sfs in use are both readonly (in contrast with normal puppies where the personal storage file or folder is writable). In order to corrupt/infect the banksy code it would be necessary to unsquash, modify, resquash, then overwrite one of the sfs files. I can't imagine that being very likely except by local hands-on operation.

I would rate the use of Wine as a greater risk than use of swap or HDD install - especially if permitting the mono and gecko downloads. I don't have any grounds for saying that - it's just a gut feeling. Do the mono and gecko downloads ask you any questions or explain what they are for? Not as far as I could tell. That in itself would concern me. I prefer to carry the most minimalistic payload possible to get the job done - less code, less exposure. That's my mantra anyway.

Your testing has brought to my attention that my system always has 10GB of swap partition available - I never thought about that being a security risk till now. I may not sleep well tonight :-)
.

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#49 Post by Sylvander »

1. "I have read that this is impossible because any code in the swap area becomes useless after shutting down a session."
I'd be happy if I knew that any swap partition in use was zero-filled as part of Banksy shut down, but I guess that's impractical.
My lack of knowledge leads me to imagine stuff remaining in the swap partition [left there by (Banksy or) some other Puppy] being brought to life during the latest Banksy session.

2. "we have to bear in mind that RAM itself is writable so in that case maybe there is not much difference"
But surely there IS a difference.
I believe the contents of SWAP is persistent, whereas the content of RAM is completely lost after a relatively short time [not persistent].

3. "I would be happy to take the risk of using the usb swap just for the time it takes to get the personalisation/burn done"
same here.

4. "I hope to reduce the required space in the next impersonator script"
I like it.

5. " In order to corrupt/infect the banksy code it would be necessary to unsquash, modify, resquash, then overwrite one of the sfs files. I can't imagine that being very likely except by local hands-on operation."
Nice to hear.

6. "I would rate the use of Wine as a greater risk than use of swap or HDD install"
Yes, I'm not happy that I use WINE, but I hate the prospect of attempting to [transfer the vault contents, and] switch to using a Puppy alternative that is also portable [and works really well] like "Acerose Password Vault" [APV].

7. "Do the mono and gecko downloads ask you any questions or explain what they are for? Not as far as I could tell."
Same here.
MONO
GECKO

8. "Your testing has brought to my attention that my system always has 10GB of swap partition available - I never thought about that being a security risk till now."
I hope I'm not raising false fears; it's just something that troubles me.
Hence, I'll avoid using a swap partitition until I come to know it poses no risk.

version2013
Posts: 503
Joined: Mon 09 Sep 2013, 00:00
Location: Florida, USA
Contact:

#50 Post by version2013 »

Sylvander wrote: 7. "Do the mono and gecko downloads ask you any questions or explain what they are for? Not as far as I could tell."
Same here.
MONO
GECKO
More links:
Mono
Gecko

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#51 Post by greengeek »

Hi Sylvander and version2013 - thanks for the mono and gecko links. I'm uncomfortable about the mono being an interface for .NET although I can see that some Wine programs will need it. Gecko worries me less, but I think I would only load this if i knew it was definitely required for a specific program.
Sylvander wrote:My lack of knowledge leads me to imagine stuff remaining in the swap partition [left there by (Banksy or) some other Puppy] being brought to life during the latest Banksy session.
I can't rule that out unfortunately. Who knows what hackers are capable of. All I can say is I am unaware of any hijack problems (except for one browser hijack on a dodgy site) during the time I have been using banksy (but hey, who would know if any of my data had been spied on??)
[The browser hijack told me my HDD had been encrypted and that I needed to pay a ransom. The browser and system was fully locked. However after a forced powerdown all was perfect]

There are three issues I am looking at to improve the performance of banksy during the personator process:

1) Add it to my longterm list to graft the PAE kernel version of Slacko 5.6 into banksy to allow addressing of more memory

2) Try variations on the "swappiness" setting as mentioned by partsman here. This won't be of any help to those who absolutely don't want to use swap, but may improve memory usage significantly and will probably allow significantly greater amounts of data to be added in the banksy personalisation process before swap becomes a requirement. I just gave it a quick try and found my RAM usage went up from 192MB (out of 2GB) to 1.5GB. It's a bit scary that my system was only using 192MB of my available RAM. (Mind you I don't know if I can trust my conky figures yet - I need to find good quality metrics to evaluate this stuff).
@Sylvander - I am interested in the bargraph figures shown by htop during any future personalisation attempts that you make. I have always wondered why my htop RAM usage figures were so low - I convinced myself that the code must be very efficient but now I realise I had huge swap all the time and RAM usage was affected by a low "swappiness" figure that seems to be inherent in Slacko. More testing required.

3) Make ongoing improvements to the impersonator script and the way it accumulates and handles data in /tmp. I seem to recall that sfs can be squashed on the fly somehow - maybe that would permit more efficient handling of personalisation data.
.
Last edited by greengeek on Sun 17 Apr 2016, 17:42, edited 2 times in total.

ASD

#52 Post by ASD »

In banksy 3 (as with many Pups) the first directory listed is "archive" which has a unique property.

Although the note inside the archive directory does not seem directly relevant to Banksy 3, it might be worth a try:

e.g. moving a large Wine pet (and/or other pet/sfs test files) to the archive directory and then clicking the "impersonator" button. Then watching for interesting effects (if any) during impersonation and upon a reboot.

A few days ago I tried Banksy 3 and all went well until I clicked the impersonator button and the process failed. Whilst Banksy 3 seems to have some nice features, it's not for me.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#53 Post by greengeek »

ASD wrote:In banksy 3 (as with many Pups) the first directory listed is "archive" which has a unique property.

Although the note inside the archive directory does not seem directly relevant to Banksy 3, it might be worth a try:
Thanks ASD - I had not actually read the text file in /archive until you mentioned it but it seems relevant to a multisession CD/DVD whch i have hardly ever used, and which I think I have fully disabled in banksy.
A few days ago I tried Banksy 3 and all went well until I clicked the impersonator button and the process failed.
I probably should have made some cautionary notes on the relevant posts to the effect that banksy "impersonator" version is an experimental effort mostly for Sylvanders testing. The normal banksy 3 "personator" available in the first post of the thread requires significantly less resources than "impersonator" as it doesn't try to grab as much data.

Having said that I do still need to improve that original version to make it more efficient and I realise it is not to everyones tastes. Thanks for trying it and for your comments about /archive.
cheers!

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#54 Post by Sylvander »

greengeek wrote:@Sylvander - I am interested in the bargraph figures shown by htop during any future personalisation attempts that you make. I have always wondered why my htop RAM usage figures were so low
Erm...
I'm in the middle of a 3rd attempt at personalizing and using "impersonator", and I've just realized I don't really know exactly what you mean by the above. [What are the "bargraph figures"?]
Right now [some personalizations completed, but "impersonator" not yet running] htop shows mem = 118/3019MB.
2 CPU cores: jumping around between 4% & 14%
swp: 0/0MB [connected a Flash Drive, and made a swap partition during the session, is it not being used?]

Guess I'll abort this session, and wait 'till you explain in detail what you need.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#55 Post by greengeek »

Sylvander wrote:I've just realized I don't really know exactly what you mean by the above. [What are the "bargraph figures"?]
There are a number of ways to identify how much of the available RAM is being used and I was referring to the top left hand corner of the Htop screen (Menu, System, Htop system process viewer) as I thought that might be a good way to view how close the system was to consuming all of it's available RAM (and therefore at risk of needing "swap" or else running out of memory). Pic attached.
htop shows mem = 118/3019MB
Yes, you're looking at the graphs I was meaning. I am surprised that it only shows 118MB in use - given that the base banksy sfs is over 200MB and your personal sfs will add extra MB over that..
swp: 0/0MB [connected a Flash Drive, and made a swap partition during the session, is it not being used?]
Swap is only available if it is discovered during booting (or if you manually control it with the swapon or swapoff command)

Just for comparison purposes I will attach below an Htop bargraph which suggests I am using 276MB out of 1984MB available RAM. This is weird as this particular session is using a HUGE personalised banksy (I added HEAPS of extra browsers, word processors etc etc) and I have opened multiple windows to run many programmes at once. I really expected it to show a much higher figure. Maybe the Htop bargraphs are not to be trusted.

By contrast I have also included a screenshot of the "free" command which offers what I would consider a more accurate view of my RAM usage. That pic suggests I am using about 1.6GB of RAM which is more in line with my expectations given the amount of stuff I have loaded and running.

You mention 3 attempts at using the impersonator - is it doing the business so far or is it giving you trouble?

I think I would make two caveats with the current impersonator version:

1) If you don't have a swap partition you will need to avoid adding excessive personalisations as the total RAM required by the impersonator is roughly 5 times the final size of the iso (eg if banksy is 220MB and you add 200MB personalisations you will need at least 420 x 5 = 2.1GB RAM for the process). I am currently working on improving this.

2) If you are able to add a swap partition for the duration of the impersonator process it will give you much greater ability to add personalisations. I run a 10GB HDD swap partition. I reckon a 4GB or 8GB usb stick would be a suitable alternative although it pays to remember that usb port speed and usb stick write speeds are significantly less than HDD drives.
.
Attachments
htop_bargraphs.jpg
(12.15 KiB) Downloaded 902 times
free_command.jpg
(11.38 KiB) Downloaded 880 times

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#56 Post by Sylvander »

1. Repeat of attempt-3.
Completed all of my customizations, then took the screenshots displayed below.
I'll [close all open windows, then] complete using "impersonator" to make the ISO and burn the CD-RW, then reboot using the new CD and report back.

2. The new Banksy3 CD-RW disk booted just fine, and works as it aught it seems.
Attachments
00.png
(3.62 KiB) Downloaded 877 times
01.png
(6.53 KiB) Downloaded 865 times

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#57 Post by greengeek »

Sylvander wrote: The new Banksy3 CD-RW disk booted just fine, and works as it aught it seems.
Hey thanks for all your testing and feedback - much appreciated. I'm glad you've been getting a useable outcome - that mirrors my own trials in the sense that impersonator works really well as long as there is adequate working memory.

If you are keen for more testing I have made a newer version of b3impersonator which is more efficient on RAM usage. I think it might now be possible for your original trial (which included the mono and gecko downloads but did not have a swap partition) to complete successfully within the 3019MB RAM that banksy addresses with its nonPAE kernel.
(If you are sick of testing thats fine :-) )

Downloads here:

EDIT 2 : please don't use this version. Better version available two posts further down
EDIT : This version works perfectly burning to CD but I made a booboo with the DVD burning script. New version to be uploaded soon. *****
http://www.mediafire.com/download/bf8in ... ic_RC2.iso
http://www.mediafire.com/download/1275u ... so-md5.txt
EDIT : This version works perfectly burning to CD but I made a booboo with the DVD burning script. New version to be uploaded soon. *****
EDIT 2 : please don't use this version. Better version available two posts further down
.
Last edited by greengeek on Sun 17 Apr 2016, 17:48, edited 3 times in total.

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#58 Post by Sylvander »

1. Attempt number-4 in progress.
a. I've burned the CD-RW of the latest rc2.
No swap partition was used.
Personal storage = 1.5G
Free = 842M [lowest value]
That completed OK.
About to reboot, and will report back.

b. Booted OK, and working as it aught.
Easy and simple to do.
"Just works". :D
SUPERB!

c. Won't be able go give further replies until Sat 16th April 2016.
Attachments
00.png
(3.59 KiB) Downloaded 845 times
01.png
(4.62 KiB) Downloaded 846 times

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#59 Post by greengeek »

I have uploaded a new version of the banksy3 impersonator which has the DVD burn script corrected and has a couple of conveniences added:

- a taskbar shortcut for the /root folder (makes it quick and easy to get access to the filesystem)

- a taskbar shortcut for xvkbd virtual keyboard which can be a more secure way to enter data rather than using keyboard keystrokes.

- ability to use shift+ctrl+c to copy data from the terminal
- ability to use shift+ctrl+v to paste data into the terminal

(Original download links for reference only:
http://www.mediafire.com/download/mfd68 ... ic_RC5.iso
http://www.mediafire.com/download/4uuo8 ... so-md5.txt)

EDIT : Newest version available here:
http://www.mediafire.com/file/jvfr3v9un ... 8.iso/file
adds the following:
- Has updated impersonator script which adds the ability to concatenate personalisations: eg it can append latest personalisations to a previously created personal sfs or if desired can directly burn a new CD/DVD which contains the previous CD/DVD's personalisations plus any new personalisations added in the current session.
- includes newer version of Grub4dos that better handles boot setup of frugal installations.
- Updated sns script displays networks in order of signal strength.

Note for new users :
- Impersonator is still in testing. The normal functionality all works well but the actual "impersonator" script that collects ALL of your personalisations requires quite a lot of RAM to manipulate the data into the new iso. You may also (temporarily) need to have a swap partition available (2GB - 10GB depending how much data you want to add). My laptop has 2GB of RAM and a 10GB swap partition on HDD. The swap partition only starts being used when I add quite a lot of extra data or sizeable programs into the running system prior to running the impersonator script.

- Many PCs that are now in the "older" bracket would have 2GB or more RAM so that is really the target market for banksy. I also run it on a 1GB netbook but that required me to add a usb swap partition temporarily and it slowed the personalisation process dramatically. Still worked though.
Last edited by greengeek on Sat 26 May 2018, 20:26, edited 7 times in total.

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#60 Post by Sylvander »

Tried the usual with RC5, and everything worked just fine.
No swap partition was used.
Personal storage = 1.5G
Free = 841M [lowest value]

After burning the CD-RW...
All the windows closed automatically when I clicked the button to acknowledge completion. NICE! :D

Booted OK, and working as it aught.

Brown Mouse
Posts: 564
Joined: Tue 09 Jun 2009, 21:06

#61 Post by Brown Mouse »

Is there any easy method of gaining access to other partitions on the hdd/removable media?

TIA

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#62 Post by greengeek »

Brown Mouse wrote:Is there any easy method of gaining access to other partitions on the hdd/removable media?
Depends on how you define easy. You can click on the terminal icon (the black "screen" icon 2 places to the right from the main menu button) and enter the following command:

pmounter

This will bring up the pmount dialog and you can click to mount the partition you want. (I find this method easy but others may consider it an annoyance).

There are a couple of considerations:

- All I did was to rename pmount to pmounter so that a newbie or child could not access pmount from the menus. Just a safety precaution.

- Because "pmount" has been renamed some of the functions of the pmount dialog do not work as they originally did (eg: "Refresh" does not work). If there are any issues I simply close the pmount dialog and type "pmounter" in terminal again.

- The pmount dialog does not reflect any devices that have been plugged in after the dialog was started. Need to close the dialog and enter "pmounter" in terminal again to start a new pmount session that shows the newly added device (does not affect the status of whatever was already mounted).

- Banksy3 uses a "readonly" version of the ntfs mounter. This is a precaution to stop newbies/children from harming their Windows partition. If you need to write to an ntfs partition I have a .pet which allows you to temporarily enable/disable writing. Let me know if you need it.
.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#63 Post by greengeek »

Sylvander wrote:Booted OK, and working as it aught.
Thanks again for testing. Much obliged good sir! (and thanks for all your posts over the years concerning modifcations that disable savefile updates at shutdown time - they are what prompted my original interest in saving personalisations in a non-corruptible way. I really love running banksy knowing I never have to worry about a savefile again 8) ).

Credited your valuable input in the text blurb inside the RC5 /opt/b3personator/b3burnimp script :)
cheers!

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#64 Post by Sylvander »

greengeek wrote:I really love running banksy knowing I never have to worry about a savefile again
Same here. :D
Most of the time.
Except when it's necessary to make changes.

It's good to have a special Puppy that never changes.

What would happen if I wanted to make a further change to [my personalized?] Banksy3?

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#65 Post by greengeek »

Dang it. I knew you were going to ask that. :)
I will do some testing and have a think.

Post Reply