Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sat 27 Dec 2014, 16:03
All times are UTC - 4
 Forum index » House Training » Users ( For the regulars )
Pupsave on USB when using separate data partition
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 2 [22 Posts]   Goto page: 1, 2 Next
Author Message
greengeek

Joined: 20 Jul 2010
Posts: 2762
Location: New Zealand

PostPosted: Sat 11 Aug 2012, 14:39    Post subject:  Pupsave on USB when using separate data partition
Subject description: Potential risk of data loss?
 

I am seeking clarification of my understanding of exactly what happens to the data when a pupsave is being used on a usb flash installation.

Example:
Imagine a usb stick which has a boot partition and a separate data partition. The pupsave is in a folder on the boot partition, but I keep my data in the second partition for safety.

It's my understanding that in a default usb flash installation the pupsave will update only occasionally in order to minimise write cycles to the flash memory on the stick (unlike an HDD install where the savefile is continuously updated). This means that there are periods of time where my "fresh" data is just sitting in RAM and not yet permanently saved into the pupsave filesystem - right?

So if I had a powercut at that moment my fresh data would be lost - right?

But what about this case: Imagine that I am using Abiword to type up a long project, and I occasionally save the file to my DATA partition. Does the Abiword save dialog ACTUALLY save immediately to the data partition (which is also a flash memory just like the boot partition) or does Abiword save into the RAM filesystem, and wait for the pupsave routine to reach it's appropriate timeout and then the data gets saved to the appropriate permanent locations.

I am hoping that any data I try to save direct to my data partition ACTUALLY goes there immediately. But does it?

This is of particular interest to me as I generally turn off my pupsave function and only do a pupsave at shutdown time (if I'm happy with the changes I've been making) - believing that all the data I saved to the other partition was immediately and permanently saved.
Back to top
View user's profile Send private message 
Karl Godt


Joined: 20 Jun 2010
Posts: 3991
Location: Kiel,Germany

PostPosted: Sun 12 Aug 2012, 08:49    Post subject:  

That's a good question .

If in PUPMODE=13 on USB the mountpoint "/initrd/pup_rw" is tmpfs filesystem i think.

The save-file.[2-4]fs should have it's own mountpoint in /initrd/ directory, in PUPMODE=13 it is "/initrd/pup_ro1" i think.
Back to top
View user's profile Send private message Visit poster's website 
Dave_G


Joined: 21 Jul 2011
Posts: 459

PostPosted: Sun 12 Aug 2012, 09:00    Post subject:  

That is a good question so I decided to try it out on Puppy4.3.1 frugal install
on USB flash drive.

Created a .abw (Abiword document) and saved it to /root.
Switched power off to computer.
When I restarted, the document was no longer there.

Interestingly, when I restarted the computer, Xwin did not complain about
an improper power down even though the file .XLOADED was in /etc before power down.
This can only mean that nothing was saved.
Back to top
View user's profile Send private message 
Karl Godt


Joined: 20 Jun 2010
Posts: 3991
Location: Kiel,Germany

PostPosted: Sun 12 Aug 2012, 09:13    Post subject:  

The relevant code for saving from temporary filesystem layer is in `which snapmergepuppy' :

Code:
SNAP="/initrd/pup_rw"
cd $SNAP || exit 1

if [ "$SAVE_LAYER" ];then #defined in PUPSTATE
 BASE="/initrd${SAVE_LAYER}"
else
 BASE="/initrd/pup_ro1"
fi


Note : PUPSTATE is /etc/rc.d/PUPSTATE configuration file .
Back to top
View user's profile Send private message Visit poster's website 
Dave_G


Joined: 21 Jul 2011
Posts: 459

PostPosted: Sun 12 Aug 2012, 09:34    Post subject:  

So I guess the moral of the story is, if one needs stuff saved permanently,
enable the periodic saving feature or use a UPS.

Considering how cheap USB flash drives are these days, as long as one
backs up the save file say once a week to another medium it does not matter
if the USB flash drive is written to say every 30 minutes or so and only lasts
around 6 months.

On my main Puppy machine, I have my save interval set to every hour and I have
been using the same USB flash drive for the last 8 months and the machine is operational every day for around 8 to 10 hours a day.
Back to top
View user's profile Send private message 
rcrsn51


Joined: 05 Sep 2006
Posts: 9402
Location: Stratford, Ontario

PostPosted: Sun 12 Aug 2012, 10:18    Post subject:  

Just to be clear:

1. You are running an application like Abiword.

2. You are working on a file that is located in a different partition, like sdb2.

3. You click File > Save.

4. You have a power failure.

The file might not be saved, but only because it was still in cached memory. If you opened a terminal window and did a "sync", I'm pretty sure that it would survive the power failure.
Back to top
View user's profile Send private message 
RSH


Joined: 05 Sep 2011
Posts: 2420
Location: Germany

PostPosted: Sun 12 Aug 2012, 10:21    Post subject:  

Hi.

I do save each and every document i do create with the LazY Puppy outside the running system.

I do use mostly a 8 gb usb stick. Putting all the LazY Puppy files to this usb stick gives me about 4 GB free space for data.

On my home workstation i do not use any save file to 100% - i did never lost any file (except the usb stick crashes).

On my usb stick i do have a directory, that contains several sub directories for different data types like office docs, pictures, scripts, music and some more. On the root partition of the usb stick i do have a file that contains the names of theses directories and sub directories - so, they are known by the system (no matter how they are named or re-named).

Code:
# LazY Puppy 2.0.2 MMS Pro external directories
# 2012-07-24 RSH for LazY Puppy
#------------------------------------------------------------------------------

# main directory of files
mydirsdir="Dateien"

# sub directories of main directory
myguidesdir="Anleitungen"
myaqualungdir="Aqualung"
mybashscriptsdir="Bash"
myimagesdir="Bilder"
myhtmldir="HTML"
myiconsdir="Icons"
mylazypuppydir="LazYPuppy"
mymusicdir="Musik"
myofficedir="Office"
mytransferdir="Transfer"
myuploaddir="Upload"
myvideosdir="Videos"

# files in office directory
mynotes="Notizblock.txt"
mylpmmstodolist="LazYPuppy-Bugs-ToDo.txt"

# Script Directories
myscripts="Skripte"
mymousesettings="MouseSettings"

# Software and Installation Directories
mysoftware="Software"

#------------------------------------------------------------------------------
# End of LazY Puppy 2.0.2 MMS Pro external directories
#------------------------------------------------------------------------------


LazY Puppy (the downloadable version as well) contains a file in /usr/sbin/ceelab/lp2_sys_data named LP2_BootDir, which is been created automatically at every boot up. This file contains different paths of LazY Puppy.


Code:
LazY_Puppy_Boot_Partition_Link="/mnt/sdd1"
LazY_Puppy_Boot_Partition="/mnt/sdd1"
LazY_Puppy_Boot_Dir_Link="/mnt/sdd1/LazY"
LazY_Puppy_Boot_Dir="/mnt/sdd1/LazY"
LP2BDL="/mnt/sdd1/LazY"
LP2BD="/mnt/sdd1/LazY"
LP2MNT="+mnt+sdd1+LazY"
LP2MNTDIR="+mnt+sdd1+LazY"
LP2MNTPRTLNK="+mnt+sdd1"
LP2MNTPRT="+mnt+sdd1"


Using these two files (i mean the data) i can build every needed path and use it from inside the system to run scripts from outside the system etc.pp.

I do use it f.e. to make symlinks for aqualung media library, config and playlists.

Code:
#!/bin/sh
# Create needed symlinks to external files and directories in /mnt/home
#------------------------------------------------------------------------------

yaf-splash -deco "Links erzeugen" -icon gtk-info -fontsize large -bg orange -text "Erzeuge Links für: Aqualung Audioplayer" &
X3PID=$!

# make file that contains the boot directory
/usr/sbin/ceelab/tools/get_boot_dir

sleep .5

# include needed files to build the paths
. /usr/sbin/ceelab/lp2_sys_data/LP2_BootDir
. $LazY_Puppy_Boot_Partition_Link/lped

# make symlink for Aqualung Files Diretory
AQUALUNGDIR="$LazY_Puppy_Boot_Partition_Link/$mydirsdir/$myaqualungdir"
if [ -d /root/.aqualung ]; then
   echo "removing /root/.aqualung"
   rm -r /root/.aqualung
fi
echo "make symlink to $AQUALUNGDIR/.aqualung in /root"
ln -s $AQUALUNGDIR/.aqualung /root/.aqualung
if [ /root/.aqualung ]; then
   echo "symlink /root/.aqualung created"
fi
# end of symlink for Aqualung

sleep .5

kill $X3PID

exit 0


If i want to increase the functions of this file or to add some new features, i can do it without the need to remaster the LazY Puppy. I just go the the menu and do click on its entry. The script is always been found as long as the usb stick is mounted - even if i would change the directory.

RSH

EDIT:

rcrsn51 has posted while i did wrote my post.

rcrsn51 wrote:
If you opened a terminal window and did a "sync", I'm pretty sure that it would survive the power failure.

LazY Puppy has a menu entry for that (menu LazY Puppy - LP2 Sync) Very Happy Laughing Cool

_________________
LazY Puppy
RSH's DNA
SARA B.
Back to top
View user's profile Send private message 
Karl Godt


Joined: 20 Jun 2010
Posts: 3991
Location: Kiel,Germany

PostPosted: Sun 12 Aug 2012, 10:38    Post subject:  

Dave_G wrote:
So I guess the moral of the story is, if one needs stuff saved permanently,
enable the periodic saving feature or use a UPS.

Considering how cheap USB flash drives are these days, as long as one
backs up the save file say once a week to another medium it does not matter
if the USB flash drive is written to say every 30 minutes or so and only lasts
around 6 months.

On my main Puppy machine, I have my save interval set to every hour and I have
been using the same USB flash drive for the last 8 months and the machine is operational every day for around 8 to 10 hours a day.


I think you can use the "Save" icon on the desktop that should be created by /etc/rc.d/rc.sysinit i think .
Or run
Code:
snapmergepuppy

manually via cli
Or create a PUPSnap.desktop file in /usr/share/applications/ that executes snapmergepuppy .
Or navigate with the abiword fileselector to "/mnt/home" or "/initrd/mnt/dev_save"(if in pfix=ram mode) and save the blank sheet first so you only need to hit the save button on abiword and it should bypass the tmpfs layer . In fact every writable mount point in the /initrd/* directory can be used to bypass the tempfs buffer.

(I run full installs and don't have all these problems) Wink
Back to top
View user's profile Send private message Visit poster's website 
rcrsn51


Joined: 05 Sep 2006
Posts: 9402
Location: Stratford, Ontario

PostPosted: Sun 12 Aug 2012, 11:01    Post subject:  

Since talk is cheap Wink , I decided to test this.

1. I made a flash drive with two partitions.

2. I started an Abiword document on sdb2.

3. If I did a Save, then immediately killed the power, the document was lost.

4. If I did a Save followed by a "sync", then killed the power, the document was saved correctly. The blinking lights on the flash drive are the clue.
Back to top
View user's profile Send private message 
Dave_G


Joined: 21 Jul 2011
Posts: 459

PostPosted: Sun 12 Aug 2012, 11:05    Post subject:  

Puppy creates (amongst others) a mount point for sdb1 (on my system)
for the USB flash drive as read-write then loop1 also as rw for the save file
and others such as loop0 and loop2 depending if other sfs files have also
been loaded such as devx.

Since sdb1 is accessible as /mnt/home then if we save directly there it is permanent.
Tried it by simulating a power failure.

To make it more organised, do as RSH wrote and make several directories
such as docs ,pics, music and so on.

The big advantage as RSH has said is we no longer limited by the size of the
save file but only by the size of sdb1 itself.

The disadvantage is making back ups will involve multiple directories and files.
Back to top
View user's profile Send private message 
RSH


Joined: 05 Sep 2011
Posts: 2420
Location: Germany

PostPosted: Sun 12 Aug 2012, 11:51    Post subject:  

Hi, me again.

I also did make a openbox pipe menu to get easily access to the externally saved files via the openbox menu. I just do install this once after boot up - if i want to use it.

Example script /etc/xdg/openbox/scripts/images_extern.sh
Code:
# Raster Image documents --> pipe menu for openbox
# RSH #

. /usr/sbin/ceelab/lp2_sys_data/LP2_BootDir
. $LazY_Puppy_Boot_Partition_Link/lped

IMGSEXTDIR="$LazY_Puppy_Boot_Partition_Link/$mydirsdir/$myimagesdir"

# Begin pipe menu for openbox
echo "<openbox_pipe_menu>"

for IMGSEXT in ` ls "$IMGSEXTDIR/" | egrep ".bmp|.jpg|.jpeg|.png|.tiff|.tif|.xpm|.BMP|.JPG|.JPEG|.PNG|.TIFF|.TIF|.XPM" `
do
   echo "<item label=\"$IMGSEXT\">
      <action name=\"Execute\">
         <execute>
            /etc/xdg/openbox/scripts/select_img_action \"$IMGSEXTDIR/$IMGSEXT\"
         </execute>
      </action>
   </item>"
done

echo "</openbox_pipe_menu>"

exit 0

# End


example script (what to do with the image) /etc/xdg/openbox/scripts/select_img_action
Code:
#!/bin/sh
#RSH

bild=$1
bnf=`basename $bild`

Xdialog --title "Bilddatei" --icon "/usr/share/pixmaps/artbrush-b.xpm" --timeout 60 --ok-label "Ansehen" --cancel-label "Bearbeiten" --yesno "Wie möchten Sie mit der Bilddatei \n\n'$bnf' \n\nverfahren? Bitte wählen Sie..." 0 0

case $? in
   0) defaultimageviewer $bild ;;
   1) defaultimageeditor $bild ;;
   255) exit 255 ;;
esac

exit 0


and exapmle html code (insert in file) /root/.config/openbox/menu_top.xml
Code:
<menu execute="/etc/xdg/openbox/scripts/images_extern.sh" id="allimgs-ext-menu" label="Pixelgrafik"/>


Please take my apologize (not appointment Laughing ) for the German Language parts in the scripts. I am just to lazy to translate them by now, because i did make this only for my private use. This is not available in the downloadable versions of LazY Puppy - but can be created though.

I hope this is interesting, somehow...

RSH

_________________
LazY Puppy
RSH's DNA
SARA B.
Back to top
View user's profile Send private message 
Karl Godt


Joined: 20 Jun 2010
Posts: 3991
Location: Kiel,Germany

PostPosted: Tue 14 Aug 2012, 11:50    Post subject:  

Quote:
Description


The sync() function shall cause all information in memory that updates file systems to be scheduled for writing out to all file systems.

The writing, although scheduled, is not necessarily complete upon return from sync().
Return Value

The sync() function shall not return a value.


from man 3 sync
Back to top
View user's profile Send private message Visit poster's website 
greengeek

Joined: 20 Jul 2010
Posts: 2762
Location: New Zealand

PostPosted: Tue 14 Aug 2012, 14:50    Post subject:  

rcrsn51 wrote:
The blinking lights on the flash drive are the clue.
Yes, good point. One of the reasons I started worrying about this issue is that I have recently been trialling puppy installs on micro-SD (Tflash) cards, so that I can carry several of them in my wallet. Unfortunately the adapter/controller they slide into does not have an LED so I feel like I'm flying blind with regard to data integrity.

It would be really nice if it was possible to have a function that brought up a gtk dialog describing the file save progress (especially when saving to a separate partition which is what I do for the data I really want to keep)

Quote:
The writing, although scheduled, is not necessarily complete upon return from sync().
Hmmm, now I feel even less secure Smile

Quote:
Please take my apologize (not appointment Very Happy ) for the German Language parts in the scripts.
Thanks RSH. I do not yet understand what is the purpose of the openbox script - I need to look at this in more detail.

Quote:
Or navigate with the abiword fileselector to "/mnt/home" or "/initrd/mnt/dev_save"(if in pfix=ram mode) and save the blank sheet first so you only need to hit the save button on abiword and it should bypass the tmpfs layer . In fact every writable mount point in the /initrd/* directory can be used to bypass the tempfs buffer.
Interesting. I will read up a bit more about the tempfs structure.
Back to top
View user's profile Send private message 
rcrsn51


Joined: 05 Sep 2006
Posts: 9402
Location: Stratford, Ontario

PostPosted: Tue 14 Aug 2012, 15:49    Post subject:  

greengeek wrote:
Quote:
The writing, although scheduled, is not necessarily complete upon return from sync().
Hmmm, now I feel even less secure Smile

This just means that there might be a short gap between when the sync command ends and the data is completely written.
Back to top
View user's profile Send private message 
greengeek

Joined: 20 Jul 2010
Posts: 2762
Location: New Zealand

PostPosted: Tue 14 Aug 2012, 16:21    Post subject:  

rcrsn51 wrote:
This just means that there might be a short gap between when the sync command ends and the data is completely written.
I'm finding that when I save a big file it sometimes takes way longer on some flash cards than others. There seems to be a huge variation in write speed between brands/models. Without an LED it's guesswork. Might have to stay away from micro-SDs. Smile
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 2 [22 Posts]   Goto page: 1, 2 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » Users ( For the regulars )
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1190s ][ Queries: 13 (0.0100s) ][ GZIP on ]