Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Tue 12 Nov 2019, 23:07
All times are UTC - 4
 Forum index » Advanced Topics » Puppy Projects
Fatdog64-802/801/800 Final [21 May 2019]
Moderators: Flash, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 16 of 27 [399 Posts]   Goto page: Previous 1, 2, 3, ..., 14, 15, 16, 17, 18, ..., 25, 26, 27 Next
Author Message
rufwoof


Joined: 24 Feb 2014
Posts: 3611

PostPosted: Thu 16 May 2019, 10:38    Post subject:  

A note of caution if you do use Fatdog multi-session usb. There's been a long term Puppy issue with layering multiple sfs's, for as long as I can remember, where in some cases it messes up. Typically with dot folders. For instance I'm linking /root folders of .ssh, ..ssh, .calcurse and ..calcurse (I'm storing my ssh keys and calendar files inside a encrypted folder on my main HDD) to outside of Fatdog space (sym linking from usb based /root to hdd based versions of those folders).

After saves, reboots etc. those symlinks however restore, themselves, such that they're no longer sym linked out to the hdd based versions, but become actual physical folders under /root again.

That's not a Fatdog specific issue, although Fatdog is affected as I believe that Fatdog uses elements of woof and it's a Puppy/Woof issue. i.e. likely due to how woof's layering (cleaning/arranging .wh files) has a bug when it comes to dot folder names.

_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3611

PostPosted: Sun 19 May 2019, 11:01    Post subject: 801 bugs  

Configuration : usb grub4dos (legacy) frugal boot using multi-session saves back to usb

=====
Bug 1

Menu, System, Fatdog Help - Install to a flash drive (alternative method) [file uefi-flashdrive2.md]
Quote:
The steps
Step 1
Before mounting: Make sure your flash drive has a FAT32 partition, and it is large enough (300M or more) to hold copies of the Fatdog files. Some UEFI firmware also accepts FAT16 but the official standard requires FAT32. Most flash drives today ship with FAT32 factory-formatted so you usually don't even need to do this, but it is always good to check and confirm.

... needs revising to 500MB.

For info (to report that gpt/uefi secure boot and multi-session usb works fine) ...

I followed the steps to create a GPT UEFI usb boot and used a 512MB sized fat partition, and all of the files copied to that just barely fitted.

After creating a second partition (ext3) to use as a multi-session save area on the USB it all booted/worked fine. UEFI is not a boot method that I am familiar with and I somewhat stumbled through getting that running (somewhat randomly selected the keys and choice of fatdog keys to get it to pass security as that seemed the most obvious, at least to me).

=====
Bug 2

At the very end of boot-options-linux.md help file section, the first link is a dead link

=====
Bug 3

pkeys=uk kernel boot parameter doesn't set the console (ctrl-alt-F3 for instance) keyboard to uk layout (remains at the default us layout). Creating a /etc/keymap (that otherwise doesn't exist) containing 'uk' fixes the console keyboard layout to be UK layout without having to specify a pkeys=uk boot parameter.

_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
fatdog

Joined: 16 Apr 2013
Posts: 99

PostPosted: Mon 20 May 2019, 22:34    Post subject:  

Thanks rufwoof. Bug 1,2,3 will be fixed in next release.

PS: we have uploaded kernel 4.19.44 with zombieload mitigation. You can update your running kernel using fatdog-updater from Control Panel. Otherwise we will release 802 soon with this kernel version included, soon.

PS2: the mitigations slow down the CPU. If you don't like the mitigations (which are enabled by default), you can pass kernel parameter "mitigations=off" which disable __ALL__ mitigations including spectre etc and will restore the speed, but, you know, you will be __vulnerable__. You can also disable various parts of the mitigations which you think is irrelevant - https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt has all the documentations.

_________________
-= The Fatdog Team (kirk, jamesbond, SFR and step) =-
Contributed Fatdog64 packages thread
This account is used for announcements only. Send PM directly to members' handle.
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3611

PostPosted: Tue 21 May 2019, 05:25    Post subject:  

I'd appreciate consideration by the fatdog team for a change request.

Extension of save2session (multi-session saves) to include validating that the device is ready/available (and perhaps additional tests to ensure that there is sufficient space on that device for the save).


Currently for my own purposes I've hard coded near the top of /usr/sbin/save2session
Code:
[ ! -d /mnt/sda1 ] && mkdir -p /mnt/sda1
mount /dev/sda1 /mnt/sda1
if [ ! -f /mnt/sda1/menu.lst ]; then
   umount /mnt/sda1 >/dev/null 2>&1
   xmessage "usb unavailable exiting" &
   exit
fi
umount /mnt/sda1

... which in my case ensures that the usb I use for booting/multi-saving is available. Otherwise the code as-is, if the usb isn't connected, sees no prior save and starts forming a totally new (first save) set. That code is obviously hard coded specific to my particular installation/setup, where the usb booted shows as device sda and includes my grub4dos bootloader menu.lst file.

Fatdogs usb booted multi-session save choice caters for that usb being unplugged after booting, which I do as a standard practice (I even have a notification message being run as part of ~/Startup to remind me to unplug the usb), and where save files are also stored on that usb - such that the usb needs to be re-attached prior to running a "Save Session". Separately to that I store my ssh keys in one encrypted folder (rox right click option) on HDD, and have another encrypted folder for my calcurse (diary), and yet another encrypted folder for DATA (everything else). Generally I boot/use and shutdown without saving, excepting if configuration changes are needed when I boot/re-configure/save/reboot. So I start with the exact same clean session at each reboot, including a 'brand new' browser session (which does mean having to store data, and browser bookmarks separately (I maintain a local html file of all my bookmarks)).

Running that way means there is no need for the Spectre etc. measures to be active, I can run with mitigations=off kernel boot parameter. In practice real world actual exploits of Spectre etc. are rare, but have high public awareness of existence. Greater real world risks tending to be less widely publicised. For instance Google's browsers
Code:
Google on Tuesday updated Chrome to version 74, an update that patched 39 security vulnerabilities and added support for websites that want to honor users' requests to limit stomach-churning motion effects.

The search company paid out $26,837 in bug bounties to 17 researchers who reported some of the vulnerabilities quashed in Chrome 74. Five of the flaws were ranked "High," the second-most-serious category in Google's four-step rating system.
Such holes are far more likely to be actually exploited, and where exploits look to install a crackers control persistently (across reboots). Booting 'clean' each time with the boot/system code loaded from removable media (usb) mitigates such threats. Leaving just online accounts userid/passwords at risk. Generally to mitigate main concern online account userid/passwords, such as online banking, if you boot clean and go nowhere else before or after other than to your banks website and reboot again afterwards then that userid/password will be protected. For other web sites, such as murga-linux.com/puppy or wherever, you just have to accept as everyone does, that those userid/passwords are at risk of being accessed by others at any time (more often userid/passwords aren't obtained via cracking individual systems, but instead via cracking the actual web site and securing thousands or in some cases millions of userids/passwords).

Of all the Puppy's/derivatives, Fatdog is the king of multi-session (that enables booting and saving from/to removable, that is physically disconnected (isolated) once booted), but as per the above change request could be improved further still Smile

_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
fatdog

Joined: 16 Apr 2013
Posts: 99

PostPosted: Tue 21 May 2019, 06:53    Post subject:  

Fatdog64 802 is released. Please see first post.
_________________
-= The Fatdog Team (kirk, jamesbond, SFR and step) =-
Contributed Fatdog64 packages thread
This account is used for announcements only. Send PM directly to members' handle.
Back to top
View user's profile Send private message 
chiefengineer

Joined: 25 Mar 2013
Posts: 61

PostPosted: Thu 23 May 2019, 16:49    Post subject: Thank-you for this great iso
Subject description: Everything works but the battery
 

Let me share two things:

1)I was so happy with the performance of my Lenovo Ideapad with this release I bought second. I discovered hibernation couple with the fact that bluetooth is on the same chip disable the clicker (not the mouse) so I worked around those.

2)My surprise was the second machine I bought was Intel (not AMD) and the battery goes undetected (this is also true in Kali which I run). It could be a defect so I am testing this blindly...but it appears to be a known issue even with Windows.

I am running a traditional legacy bios boot which I dd'd to a 3.1 usb key
and created a third save partition. It flies.

I have also disable/re-enabled some cryptic thermal controls in the bios to no avail. Battery seems to work/recharge so far, including an unplug beep---just no detection...it appear the power button blinks when it is running low.

Any ideas?
Back to top
View user's profile Send private message 
don570


Joined: 10 Mar 2010
Posts: 5370
Location: Ontario

PostPosted: Thu 23 May 2019, 17:58    Post subject: fatdog64 802  

I installed fatdog64 802 ---> frugal install on windows XP ntfs disk
I found UUID with 'blkid' command so I could use grub4 menu.lst file
I saved to a folder rather than a file.

First report:
Blender works. I started SAMBA server with the control panel.

Then I went to /root/share and clicked on network icon.
I was able to connect with another linux computer using YASSM
__________________________________________________
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3611

PostPosted: Mon 27 May 2019, 08:29    Post subject: initrd compression experiment  

Opened initrd and rebuilt both the kernel modules sfs and fd64 sfs within that using no compression (mksquashfs -noI -noX -noD -noF parameters).

Reformed the initrd (cpio) which was now around 1.4GB in size.

Applied xz -e9 --check=crc32 initrd ... i.e. extreme compression and the resulting initrd.xz was 333MB (compared to 412MB for the default fatdog 8.2 initrd). So alongside a 6MB, a frugal booted Fatdog 8.2 can be as little as 340MB.

However, it looks to me that no matter what compression is applied within fd64.sfs it still takes up the same amount of ram when actually loaded/booted. It's as though the sfs is read, extracted and then stored into ram using whatever compression the kernel is set to use for storing the contents in ram (inodes, dentry compression). Booting that extreme compressed initrd.xz and it initially copied into ram quicker (being smaller), but then stalls as its loaded into ram (being high compression that takes quite a while on my setup, around 75 seconds). So no real overall speed advantage, and uses a similar amount of ram overall. No real benefit excepting perhaps for the likes of PXE booting when the initial reading of the initrd might be very slow, and slow enough to offset that 75 seconds of reading into ram delay.

Delving deeper, and using the benchmarks stated in https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO, for a compress once, decompress many situation such as frugal booting, the 'best' choice (when we care little about how long it takes to compress, and are seeking the fastest decompression), then from that link we can measure the function of compression ratio x decompression speed. Based on those figures, lz4 stands out considerably, followed by lzop (lzo) level 9 compression, followed by gzip. Compression ratio x decompression speed values of (lower = better)

70.5 lzma
29.52 lzop
64.2 gzip
10.68 lz4
78.96 xz

We also however need to consider the time to initially read/load that (bios level reading/copying tends to be relatively slow), where smaller = faster. On my kit for instance (usb frugal boot) bios reads the initrd at around a 18.5MB/sec rate. So a initrd with lz4 Xhc compressed kernel modules and fd64.sfs, that weighs in a 611MB for instance takes around 33 seconds to read/load. Whereas a 411MB initrd (as per the default fatdog 8.2) takes 22 seconds (but then takes around 8 times longer to read into ram than when using lz4). On other kit (reading from DVD), the BIOS load time might be considerably longer.

No overall conclusion, other than to me it looks like the better choices are to either use xz as per how Fatdog currently does, or lz4. For me, when using lz4 the desktop/system does appear to be snappier after initially being booted.

_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
williams2

Joined: 14 Dec 2018
Posts: 190

PostPosted: Mon 27 May 2019, 15:09    Post subject:  

Quote:
it still takes up the same amount of ram when actually loaded/booted. It's as though the sfs is read, extracted and then stored into ram

An sfs is like a zip or a tar.gz file. There are files compressed or not in the sfs. The sfs file is a certain size.
For example, my bionicpup64 sfs file is 331 MB. (right-clicking the sfs file in /initrd/mnt/tmpfs and clicking count)
The files inside the sfs file are 1355 M (right clicking /initrd/pup_ro2 and clicking count)
This is the size the files are when and if they are accessed / used. If you execute or read a file it will be decompressed automatically, ready to use. Files that are not executed or read stay compressed in the sfs file.

If the sfs file is copied to the ram drive, it will take up space in the ram drive. In my case, 331 MB. If it is on a hard drive or flash drive and not copied to the ram drive, it will not take up space in the ram drive.

So you can boot Puppy without copying the sfs to the ram drive, and you would save 331 MB. The sfs still is taking space in the hard drive or flash drive. It would be slower to access than if it were in the ram drive. The contents of the sfs file are not extracted unless you execute or read a file that is in the sfs. If you execute a script in the sfs, it will automatically be extracted and decompressed and copied from the sfs to a buffer in ram, and copied (decompressed) to a cache in ram and (decompressed) to somewhere in ram where it can be executed. Only files in the sfs that are actually executed and/or read will be copied from the sfs to ram.

That is, mounting an sfs does not copy anything to ram. There is some overhead, like a device buffer. Only files that are actually accessed will be copied to ram.

If you use rox or du to examine the size of the files in the sfs, it will show you the decompressed sizes. It will not decompress the files unless they are actually used (accessed).
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3611

PostPosted: Mon 27 May 2019, 15:22    Post subject:  

Thanks williams2.

In my case, fd64.sfs inside of initrd, whether that fd64.sfs is not compressed, or highly compressed, it still results in the same amount of htop mem once the system has booted. That's with the ram boot option i.e. copied to ram. Which led me to conclude that the underline process involves reading the fd64.sfs from within init ram space, and installing it into user ram space and where its not a direct copy but a 'loading' (transformation) in how its stored in user ram space.

_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
williams2

Joined: 14 Dec 2018
Posts: 190

PostPosted: Mon 27 May 2019, 18:14    Post subject:  

Quote:
In my case, fd64.sfs inside of initrd

Shouldn't matter. Booting FatDog721 (I haven't installed 8 yet):

Code:
# losetup
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE
/dev/loop1         0      0         0  0 /fd64.sfs (deleted)
/dev/loop2         0      0         0  0 /aufs/devsave/fd64save.ext4
/dev/loop0         0      0         1  1 /kernel-modules.sfs (deleted)
#


The layered aufs file system seems to be:

/kernel-modules.sfs (at the bottom)
/fd64.sfs
/aufs/devsave/fd64save.ext4 (at the top)

rox shows /aufs/pup_ro is 299 M (right click, properties)
rox shows the contents of pup_ro if they were all uncompressed is 1251 M (right click, count)

In this case, the sfs files in the initrd were mounted and then seem to have been deleted. Because the sfs files are mounted and therefore still in use, the name is removed from the directory and so is invisible, but the inode is still in the directory and the sfs files are still taking up space.
fd2.png
 Description   rox count
 Filesize   12.59 KB
 Viewed   604 Time(s)

fd2.png

fd1.png
 Description   rox properties
 Filesize   27.61 KB
 Viewed   610 Time(s)

fd1.png

Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3611

PostPosted: Mon 27 May 2019, 18:55    Post subject:  

I'm booting multi-session, saving back to the same usb booted from (and with usb removed after bootup), and (Fatdog 8.2) losetup shows
Code:
# losetup
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE                     DIO LOG-SEC
/dev/loop1         0      0         0  0 /fd64.sfs (deleted)             0     512
/dev/loop0         0      0         1  1 /kernel-modules.sfs (deleted)   0     512
#

/aufs/pup_ro properties shows 339MB with right click count : Total: 1442 M (29670 files, 3749 directories)
Quote:
In this case, the sfs files in the initrd were mounted and then seem to have been deleted. Because the sfs files are mounted and therefore still in use, the name is removed from the directory and so is invisible, but the inode is still in the directory and the sfs files are still taking up space.

Seemingly deleted to new processes, but still accessible to any child processes i.e. not released and still accessible until any process or its children close it - but deleted to any new process that tried to open it.

Confused as to why a non compressed fd64.sfs inside initrd that is over a GB in size, results in the same htop mem being shown as when a compressed fd64.sfs inside initrd.
s.png
 Description   
 Filesize   44.71 KB
 Viewed   590 Time(s)

s.png


_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
williams2

Joined: 14 Dec 2018
Posts: 190

PostPosted: Mon 27 May 2019, 19:40    Post subject:  

Quote:
why a non compressed fd64.sfs inside initrd that is over a GB in size, results in the same htop mem being shown as when a compressed fd64.sfs

du of files in the sfs file system will always show the uncompressed sizes. So it would not matter what compression is used, it will always show the same uncompressed sizes.

I don't know about htop. I would think it should show more free ram if the sfs files are compressed. I don't know why it wouldn't. I think "free" gives me a better idea of what is going on compared to htop.
Code:
# free -wh
              total        used        free      shared     buffers       cache   available
Mem:           3.5G        512M        1.6G        755M        181M        1.2G        1.8G
Swap:            0B          0B          0B
#

htop says I'm using 1.24 G
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3611

PostPosted: Mon 27 May 2019, 20:35    Post subject:  

I have similar figures to yours for 'free'

htop shows a compound memory value, sum of used + buffers + cached, colour coded green, blue, orange respectively.


(clickable thumbnail)

Side note : see how in that screenshot I've moved the multi-sessions Save Session desktop icon to be next to the main drives icons (I used event manager to set the left offset to be deeper to leave space for the Save Session icon (bottom 34, left 54 in my case)). Quite a nice location for it IMO.

_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
williams2

Joined: 14 Dec 2018
Posts: 190

PostPosted: Mon 27 May 2019, 20:48    Post subject:  

Maybe FatDog is making the ram drive (the rw file system) a fixed percentage of the total ram. In which case it would not matter what size the sfs files are.

Just a thought.
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 16 of 27 [399 Posts]   Goto page: Previous 1, 2, 3, ..., 14, 15, 16, 17, 18, ..., 25, 26, 27 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Puppy Projects
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0950s ][ Queries: 13 (0.0104s) ][ GZIP on ]