This Save-session file problem exist with ALL modern PUPs

Please post any bugs you have found
Message
Author
gcmartin

This Save-session file problem exist with ALL modern PUPs

#1 Post by gcmartin »

Most importantly, let me say
"Happy Holidays to ALL of YOU!"

This bug exist in most/all modern WOOF built PUPs
Note: Although this has been observed in several PUPs, I am reporting it from a recent Carolina view.

Scenario
Here's a problem that Carolina is plagued by too. I have a "Compressed NTFS file system that I am testing Carolina from on a PC that is used to dual boot various OSes. This is rather a first for me, because I rarely use this method for running PUPs. But, I got started as a result of trying to help someone else in the forum.

The Boot manager is a Carolina installed GRUB4DOS to the MBR on the HDD. The menu.lst entry points to /LinuxBoots/Carolina (SDA5) folder where the contents of the Carolina ISO exist. On initial booting/pfix=ram all is well as its has always been. There is no GRUB4DOS error in getting the system booting and running properly accessing all HDD/USB partitions that exist. The funning system has NO ERRORs for anything it is doing. After desktop tailoring, I take the system thru Shutdown as I have done using Live media for years...ONLY THIS time, I select the option to store the save-session data to a file that is created in /LinuxBoots/Carolina folder.

So I now have a booting Carolina system with a Carolina file created of my session activities. This file is save by Carolina in the boot Carolina folder, as one would expect.

On reboot the system fails starting at the point when it attempts to reference the save file. It seems to find the file, but the file is not what the boot process wants to see.

Problem at boot
Normal boot messages leading up to the following:
loading drivers to access disk drives ..... done
searching for Puppy files ... pupsave zdrive adrive ... done
loading personal file /LinuxBoots/Carolina/linasave-005_01.4fs (sda5) ...
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
Dumping last lines of /tmp/bootinit.log
mount: mounting /dev/loop1 on /pup_rw failed: Invalid argument
e2fsck: 1.41.9 (22-Aug-2009)
e2fsck: Invalid argument while trying to open /dev/loop1
mount: mounting /dev/loop1 on /pup_rw failed: Invalid argument
Dumping last lines of kernel.log
<6>[ 6.825671] aufs 3.2-20120326
<6>[ 6.828294] fuse init (API version 7.17)
<3>[ 11.066059] Ext4-fs (loop1):unable to read superblock
<3>[ 11.080042] Ext4-fs (loop1):unable to read superblock
Pausing for 60 seconds


.... system lockup follows ...
-------------------------------------

Problem at desktop skipping save-session
Starting the system via pfix=ram. At desktop I executed the following to access the save file and got the same error message for the save file:

Code: Select all

# losetup /dev/loop1 /mnt/sda5/LinuxBoots/Carolina/linasave-005_01.4fs
# e2fsck -b 8193 /dev/loop1
e2fsck 1.41.14 (22-Dec-2010)
e2fsck: Invalid argument while trying to open /dev/loop1

The superblock could not be read or does not describe a correct ext2
filesystem.  If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>
Question
Is there something wrong with Shutdown processing in Puppy Linux when it goes to create a save-session file from the running system to the Compressed NTFS?

Here to help
Last edited by gcmartin on Wed 26 Dec 2012, 00:36, edited 1 time in total.

linuxcbon
Posts: 1312
Joined: Thu 09 Aug 2007, 22:54

#2 Post by linuxcbon »


User avatar
James C
Posts: 6618
Joined: Thu 26 Mar 2009, 05:12
Location: Kentucky

#3 Post by James C »

From forum member ICPUG's site:

http://www.icpug.org.uk/national/linnwin/step1-xp.htm
This is because NTFS-3G, the Linux driver to work with NTFS partitions, will not be able to write files to NTFS with compression partitions. Therefore, what you can do will be limited to read only of NTFS compressed partitions. NTFS-3G works perfectly happily with the NTFS no compression system and you will be able to read from and write to such partitions.

User avatar
rcrsn51
Posts: 13096
Joined: Tue 05 Sep 2006, 13:50
Location: Stratford, Ontario

#4 Post by rcrsn51 »

There is a bug in various 5.3 vintage Puppies, like Carolina, concerning NTFS save files. It has been fixed in the latest builds of 5.4 Puppies.

As a test, I made a compressed NTFS flash drive and used it for the save file of Slacko 5.4. It worked fine.

gcmartin

#5 Post by gcmartin »

James C wrote:From forum member ICPUG's site:

http://www.icpug.org.uk/national/linnwin/step1-xp.htm
This is because NTFS-3G, the Linux driver to work with NTFS partitions, will not be able to write files to NTFS with compression partitions. Therefore, what you can do will be limited to read only of NTFS compressed partitions. NTFS-3G works perfectly happily with the NTFS no compression system and you will be able to read from and write to such partitions.
Thanks James C. But the reality is that the system has absolutely no problems with any filesystem operations during normal system running.

None, yet the save-session is where the problem is introduced. Reading, writing, editing, creating, ... no problems in operations. This normal desktop accessing of compressed file systems has been working for quite some time. The reason I may have missed this in the past is that as everyone knows), I "almost never" install/write PUPs (frugal or main) on any HDD/USB. The problem is not normal system use. This occurs when the system is to create its save-session file. There are no known or reported other times of errors using the partition(s) on any PCs.

Thanks @Rcrsn51. The problem reported here was observed last week in recent BarryK's Precise. (Here hoping BarryK see this to move the resolution into his distros as well.)

Thanks though and Happy holidays.
Last edited by gcmartin on Wed 26 Dec 2012, 00:53, edited 1 time in total.

gcmartin

This may be the actual problem ...

#6 Post by gcmartin »

Yeah. There is something that is happening in Shutdown's save-session processing that does NOT occur during normal system operations.

Since all other system operations work flawlessly, the problem "MAY" be in the tool used to either format the file for ext use or in something in how the system is mounting the file for copying the system changes.

I have a question which I hope someone entertains. Why is frugal save-session file different from what happens on Live media? I can create artificial reasons for the difference, personally, but, in practice seems that both should use the same technology; because, "the Live media reasoning is really a sound reasoning in how it is structured for recovery". This answer has nothing to do with whether its on Live media or frugally on USB/HDD. It only asks about the processing of save-sessions on reboots. This area may benefit from a minor change to reduce save-session processing to a single process no matter if its Live media or frugal. Hope the developer(s) see some benefit.

In any event, my question in the paragraph above has NOTHING to do with the current problem where PUPPY shutdown is have trouble creating-writing to its Shutdown file in the filesystem.

Hope this helps
Last edited by gcmartin on Wed 26 Dec 2012, 01:34, edited 4 times in total.

User avatar
rcrsn51
Posts: 13096
Joined: Tue 05 Sep 2006, 13:50
Location: Stratford, Ontario

#7 Post by rcrsn51 »

Correction. My NTFS compressed save file would boot OK but didn't save anything. So I took compression off and it started working.

User avatar
James C
Posts: 6618
Joined: Thu 26 Mar 2009, 05:12
Location: Kentucky

#8 Post by James C »

From forum member amigo :

http://www.murga-linux.com/puppy/viewto ... 814#566814
I meant that there is no intention, claim or pretension to support MS compressed filesystems. It's not a bug because they tell you that compressed FS are not supported and should not be used by their NTFS/FAT drivers.

gcmartin

this is not an NTFS bug. Something overlooked, maybe..

#9 Post by gcmartin »

James C wrote:From forum member amigo :

http://www.murga-linux.com/puppy/viewto ... 814#566814
I meant that there is no intention, claim or pretension to support MS compressed filesystems. It's not a bug because they tell you that compressed FS are not supported and should not be used by their NTFS/FAT drivers.
Thanks @James C. I'm not sure that that is the issue here.

We have working filesystem operations. Its been working for years. We aren't making any grave departures here. We are reporting that in the normal workings of the system, we discovered a bug in PUPPY....NOT a bug in NTFS.

We are NOT trying to fix anything MS. This is a simple Puppy shutdown processing issue in its filesystem operations at shutdown time.

Thanks though for your comments.

User avatar
rcrsn51
Posts: 13096
Joined: Tue 05 Sep 2006, 13:50
Location: Stratford, Ontario

#10 Post by rcrsn51 »

This issue was discussed at length during the Slacko 5.4 development cycle. Here is the gist of it.

1. Somewhere in the 5.3 cycle, some code was added to rc.shutdown to facilitate the unmounting of network shares. That code called the umount script.

2. However, the umount script had a bug that would cause it, under some conditions, to also unmount any mounted NTFS partitions.

3. So if you had selected an NTFS partition as the target for a save file, it would be accidentally unmounted before the internal filesystem could be written into the save file. If you have one of these non-working save files, you can verify this by click-mounting it and observing that it is empty.

4. The solution, which is now in woof, was to replace "umount" with "umount-FULL" in rc.shutdown.

The question of whether compressed NTFS save files work is a separate issue.

gcmartin

#11 Post by gcmartin »

Remembering the discussion.

From an external view, it seems that during processing at shutdown the system is able to create the file to support the save-session.

Thus several things could be happening from; remounting the partition where the save-session is to be created; to the formatting of the file for Puppy use, to the logic being interfered because of a missing error message somewhere.

I do not have the skills to review the code for why this is showing up at shutdown, all the while, up until this point the system has behaved expectantly.

Wish I could contribute more or be more of a help.

User avatar
rcrsn51
Posts: 13096
Joined: Tue 05 Sep 2006, 13:50
Location: Stratford, Ontario

#12 Post by rcrsn51 »

gcmartin wrote:Wish I could contribute more or be more of a help.
1. Do a frugal install of either the latest Slacko 5.4 or Precise 5.4.

2. Make a save file in your compressed NTFS partition. Does it boot? Does it save content?

3. Repeat with an uncompressed NTFS partition.

gcmartin

Shutdown cannont save-session to NTFS. Normal use is OK

#13 Post by gcmartin »

Report:using Carolina5 and Slacko54 and both recently released versions of Precise from developers

Test1
  • Save-session (ext2) to another NTFS partition on Shutdown.
  • Save-session and reboot... WORKS!.
Test2
  • Copied the boot folder from the Compressed partition to the non-compressed NTFS partition.
  • Updated GRUB4DOS to reflect the new boot location.
  • Booted that new entry using Test1's ext2 Save-session... does not WORK!
Note: The PC now has Carolina (frugal) in 2 folders: Compressed NTFS and normal NTFS.

Summary
  • the PC boots pfix=ram, no matter which partition is used
  • the PC boots with the EXT2 save-session on the non-compressed NTFS partition, again, no matter which partition is used
  • The PC fails to create a useable save-session file on the compressed NTFS
Attachments
Carolina folder contents.png
Folder contents of boot folder
(34.92 KiB) Downloaded 1121 times

gcmartin

More evidence of problem happening at save-session Shutdown

#14 Post by gcmartin »

Here's more info from tests done with Carolina, for it seems to help hint at where to look for the save-session problems in Woof build PUPs.

Scenario
Boot Carolina with Live media. Tailored desk and requested reboot. The PupSave Wizard took me thru several screens in preparation for Shutdown.

This time, at the end, it prepared a folder to be used for Carolina and save-session.

Review of the Summary screen present (see below) and the folder contents (see below) Carolina seems poised to complete Shutdown.
But, even though the save-session file (pre-created by Carolina on the NTFS partition) is NOT a zero length, when the system attempts access to the "linasave" the save-session file in the Carolina folder, the boot fails with error messages reflecting attempt at access a "zero-length" file.

Hope this gives clear, repeatable evidence of where in Puppy shutdown processing this problem exisst so that it can be resolved for ALL Puppy distros which attempts save-session to NTFS..There appears to be something that Puppy is NOT honoring at this stage of shutdown processing that is normally respected when the system is running normally.

Here to help
Attachments
Carolina save-session creation on NTFS.png
Summary screen prior to shutdown. Following folder also created at this time.
(29.68 KiB) Downloaded 1131 times
Carolina Shutdown Folder.png
Carolina creates this folder with its contents to support physical Shutdown
(54.5 KiB) Downloaded 1053 times

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#15 Post by jamesbond »

I just did a quick check on Fatdog, it seems to work correctly on compressed NTFS (creating and using).

Check the version of ntfs-3g used inside initrd (that is, part of woof). Mine is ntfs-3g 2012.1.15 integrated fuse 27.

cheers!
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

User avatar
01micko
Posts: 8741
Joined: Sat 11 Oct 2008, 13:39
Location: qld
Contact:

#16 Post by 01micko »

jamesbond wrote:Check the version of ntfs-3g used inside initrd (that is, part of woof)
It's old:

Code: Select all

 ./ntfs-3g --help

ntfs-3g 2010.1.16 integrated FUSE 27 - Third Generation NTFS Driver
		XATTRS are on, POSIX ACLS are off

Copyright (C) 2005-2007 Yura Pakhuchiy
Copyright (C) 2006-2009 Szabolcs Szakacsits
Copyright (C) 2007-2009 Jean-Pierre Andre
Copyright (C) 2009 Erik Larsson

Usage:    ntfs-3g [-o option[,...]] <device|image_file> <mount_point>

Options:  ro (read-only mount), remove_hiberfile, uid=, gid=,
          umask=, fmask=, dmask=, streams_interface=.
          Please see the details in the manual (type: man ntfs-3g).

Example: ntfs-3g /dev/sda1 /mnt/windows

Ntfs-3g news, support and information:  http://ntfs-3g.org
It's a static compile by Barry. I have no idea how to update it. Little help?
Puppy Linux Blog - contact me for access

User avatar
pemasu
Posts: 5474
Joined: Wed 08 Jul 2009, 12:26
Location: Finland

#17 Post by pemasu »

Barry`s post how he did it.
http://bkhome.org/blog/?viewDetailed=01316

User avatar
01micko
Posts: 8741
Joined: Sat 11 Oct 2008, 13:39
Location: qld
Contact:

#18 Post by 01micko »

Ha, was just about to search that, downloading the uClibc filesystem now, but it's old! (2005). We'll see if it works, I'll grab the latest ntfs-3g sources. (edit, found an image from 2008, had to expand it to make room for the compile resize2fs root_fs_i386.ext2 150M then mount it, copy the ntfs-3g source, chroot, compile)

- Later...


ok, I got it compiled, the binary is 420k stripped, Barry's is 391k, so that's not too bad considering there would be natural bloat from bugfixes, extra features etc. I didn't test it as I don't have any compressed ntfs drives.

To test, unpack and add the binary to your PATH (suggest /root/my-applicatins/bin or /usr/local/bin) and move (or rename) the original (usually /bin/ntfs-3g) then try to mount an compressed ntfs partition

Code: Select all

mkdir -p /mnt/ntfs-test 
mount -t ntfs-3g /dev/sda2  /mnt/ntfs-test 
replacing sda2 with whatever your target partition is. That should mount it rw. (That's the code on the ntfs-3g site).

Don't forget to unmount when you've finished testing.. umount /mnt/ntfs-test

If it works you can just click on your initrd.gz in a frugal and it should unpack in /root. Replace the ntfs-3g file in /bin then repack the initrd.gz. Remaster. See what happens!

NO WARRANTY

EDIT: if it works, there is no need to recompile in your distro as it's completely static. Wouldn't matter what distro you built it in.. (x86 32 bit of course)

Cheers
Puppy Linux Blog - contact me for access

User avatar
L18L
Posts: 3479
Joined: Sat 19 Jun 2010, 18:56
Location: www.eussenheim.de/

This Save-session file problem exist with ALL modern PUPs

#19 Post by L18L »

gcmartin wrote:Here's more info from tests done with Carolina, for it seems to help hint at where to look for the save-session problems in Woof build PUPs. ...
about pupsaveconfig BarryK wrote: Excellent work, but I never put that into Woof, as I wanted to do it a bit differently.
see http://bkhome.org/blog/?viewDetailed=02489

So please forget the term "ALL modern PUPs" :lol:

(I prefer the official ones :D )

User avatar
rcrsn51
Posts: 13096
Joined: Tue 05 Sep 2006, 13:50
Location: Stratford, Ontario

Re: More evidence of problem happening at save-session Shutdown

#20 Post by rcrsn51 »

gcmartin wrote:Hope this gives clear, repeatable evidence ..
Not really. You haven't stated whether you are still dealing with COMPRESSED NTFS partitions.

As a test, I booted Carolina 005 off its Live CD.

I applied the rc.shutdown patch as described in the Slacko 5.4 thread.

I made a save file on an UNcompressed NTFS flash drive.

I rebooted. The save file was detected. Everything worked correctly.

Post Reply