Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Mon 22 Sep 2014, 10:23
All times are UTC - 4
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Utilities
snap2 rotating snapshot backups for Puppy
Post new topic   Reply to topic View previous topic :: View next topic
Page 5 of 11 [158 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 Next
Author Message
Jim1911

Joined: 19 May 2008
Posts: 2457
Location: Texas, USA

PostPosted: Mon 08 Feb 2010, 16:12    Post subject:  

Hi lstandish,

Just tried your new snap2-4 (version 4.0beta). Impressive, however, I got more files than I expected on the snapshot backup.

I mirrored a 26.54GB partition with 18.69GB used to a 31.25GB partition. The mirror operation worked fine. Then I made one file change of a few KB to the source partition and then selected 'Snapshot Backup Now' using the GUI. It created a recent.1 directory in which I expected to just find the changed information of only a few KB, however it created another entire partition backup in the recent.1 directory, therefore the entire 31.25GB drive partition was filled. I then deleted the recent.1 backup and the drive now shows 18.73GB used for the mirror which seems about right.

Am I misunderstanding what should take place with the snapshot? Confused

Your dirdiff Graphical Directory Comparision utility is really nice too.

Thank you for all the hard work that you've put into this. Looking forward to further tests.
Jim Very Happy

Last edited by Jim1911 on Mon 08 Feb 2010, 16:25; edited 3 times in total
Back to top
View user's profile Send private message 
Sylvander

Joined: 15 Dec 2008
Posts: 3444
Location: West Lothian, Scotland, UK

PostPosted: Mon 08 Feb 2010, 16:14    Post subject:  

With reference to my previous post, here's an example:
00.jpg
 Description   
 Filesize   16.4 KB
 Viewed   798 Time(s)

00.jpg

Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Mon 08 Feb 2010, 22:32    Post subject:  

Jim1911,

Each snapshot backup is supposed to look like a full backup. It is a "snapshot" of the state of your files at a given date. However, the storage space used for each snapshot is essentially only that of changed and modified files, thanks to the use of hard links for all files that are unchanged between snapshot backups.

Many file size reporting utilities are fooled by hard links and will incorrectly report the space used as though each snapshot were a separate copy. I suspect this is what happened in your case.

If this happened, you will find that the log report of the second backup (recent.1 - look on the second tab of the snap2 GUI for log display) only shows that one file was backed up. (There may have been a few others too that the system automatically updated or created.)

Also, running 'du -cbs backuproot' in a terminal (where 'backuproot' is the top level directory for your backup storage) will show the correct amount of storage space used by your backups.

To illustrate how space reporting utilities are fooled, below is a screenshot of my Debian 'disk properties' for my 8 gig USB flash drive, which I used for months as snapshot backup storage. It appears to hold 30.2 gigs! However, look at the (correct) "Free disk space" line at the bottom.

Sylvander,
Yes, recent.1 must ALWAYS be newer than recent.2 (etc.). If you mess with the date stamps so that this is no longer true, backups can no longer be done.

I'll try to find a more descriptive button name for the logfile/reporting/backup deletion utilities. Thanks very much for your suggestions and feedback!
my8gigUSBdrive.jpg
 Description   
 Filesize   38.73 KB
 Viewed   791 Time(s)

my8gigUSBdrive.jpg


_________________
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
http://standish.home3.org/snap2
Back to top
View user's profile Send private message 
Sylvander

Joined: 15 Dec 2008
Posts: 3444
Location: West Lothian, Scotland, UK

PostPosted: Wed 10 Feb 2010, 06:42    Post subject:  

Don't know if I mentioned or not, but...

Prompted by me using your snap2 program to make snapshots...
It then occurred to me that I needed to make an image of the whole of the internal 80GB HDD.

That Pudd image was 43GB, and I had to delete some of the snapshots to make space.
So I bought a new Allcam SATA to USB external enclosure, plus a 1TB SATA HDD.

Everything is up and working now.
Made new snap2 snapshots of sda1&sda5, sda3, sda6, sda7.
All very easy/routine to do.

Now making images using the FREE "Seagate Disk Wizard - Bootable Media" [CD-RW].
This has USB drivers, so my USB mouse is functioning and there's a cursor, and it can see the destination partition on the external USB HDD.
It took ONLY TWO HOURS to image my 80GB internal HDD.
I'd prefer to use some program within Puppy to do the job, but...
Pudd took 20 hours to do the same job.
Plus there were difficulties due to mounted partitions [Linux swap & pupsave], so I had to delete the Linux swap, and use the...
puppy pfix=ram command.
Much easier just to use the Seagate disk.

My FREE "Acronis True Image 11" "Emergency Disk" wasn't able to see any USB devices, so is useless to me.
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Thu 11 Feb 2010, 01:14    Post subject:  

Version 4.0 does not show deleted files correctly (it also shows changed files). This is fixed in version 4.1, which I will upload soon (and announce here).
_________________
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
http://standish.home3.org/snap2
Back to top
View user's profile Send private message 
Jim1911

Joined: 19 May 2008
Posts: 2457
Location: Texas, USA

PostPosted: Thu 11 Feb 2010, 10:31    Post subject:  

lstandish wrote:
Also, running 'du -cbs backuproot' in a terminal (where 'backuproot' is the top level directory for your backup storage) will show the correct amount of storage space used by your backups.
Sorry, took so long to reply, but as you can see from the output below, there is a duplicate mirror being created by the snapshot created by clicking on "Snapshot Backup Now". The only difference should be an OpenOffice3.2-sfs4_431.sfs which was downloaded and added to the mirror location. (your great little dirdiff utility confirms the difference) Bottom entry is after deletion of the recent.1 backup.
Code:
# cd /mnt/sdc2/
# ls
kids  mirror  MU  recent.1  upup432
# du -cbs
61012170345     .
61012170345     total
# du -cbs
36961235181     .
36961235181     total
#

I'll delete the contents and start over when your next release is posted. Thank you for all the hard work that you are devoting to this project.
Cheers,
Jim
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Thu 11 Feb 2010, 10:56    Post subject:  

Hi Jim,

You configured and ran both a mirror-type backup AND a snapshot-type backup. These are 2 different backup types. They are configured separately and are run by clicking on separate buttons. If you make a mirror backup and also a snapshot backup you will have 2 separate copies taking up full disk space in each, as you observed.

Use a mirror-type backup when you only need a SINGLE copy of your files. Use snapshot backups when you want to be able to obtain past versions of files. Do not use both for the same source directories.

If you have made a mirror backup, make a few changes to the source, and run it again, only the changed/added files will be backed up - it will be very fast.

If you have made a snapshot backup (which on first run results in a directory called recent.1), make a few changes to the source, and then do another snapshot backup, you will get what appears to be a whole new copy of your files, but in fact only the changed/added files will actually take up disk space - it will ALSO be very fast.

After the second run you will have, then, a 'recent.1' and a 'recent.2'. recent.1 will be the most recent snapshot. recent.2 will be the previous one.

recent.1 is always an exact 'snapshot' of the source files at the time of the last backup. For that reason, it will be the same as a mirror-type backup specifying the same source directories. However, unlike a mirror backup, with snapshot backups you also have previous snapshots of your files, allowing access to files you have deleted from the source, and previous versions of files.

On dirdiff, sorry for the confusion: I am not the author of that. I just made the Puppy package.

The most recent version of snap2 (4.0) will also give information on files deleted from one snapshot to the next, but if you want to try that, you should wait for me to release version 4.1, since 4.0 is also reporting changed files.

_________________
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
http://standish.home3.org/snap2
Back to top
View user's profile Send private message 
Jim1911

Joined: 19 May 2008
Posts: 2457
Location: Texas, USA

PostPosted: Thu 11 Feb 2010, 11:46    Post subject:  

Quote:
Am I misunderstanding what should take place with the snapshot? Confused

As stated in my first post, I was, thanks for clarifying. Suggest that you include a prominent "CAUTION: Before proceeding make sure that your source and destination partitions are mounted." on your opening screen. Thanks for packaging dirdiff, it's a very handy little utility.

Looking forward to your next release,
Jim
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Thu 11 Feb 2010, 12:45    Post subject:  

I agree some sort of prominent warning is necessary. For the moment I have added the following to the "3 Easy Steps" info at the top of the GUI:

"On the DIRECTORIES TO BACK UP tab, add folders to EITHER 'Snapshot Backup Paths' OR 'Mirror Backup Paths' (Do not use both for the same source folders)"

I am open to suggestions on how to clear up the snapshot vs mirror backup confusion. Of course, I try to avoid overwhelming the user with wordy explanations.

Thanks very much for your feedback on this. All feedback helps a developer in the struggle for user-friendliness.

_________________
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
http://standish.home3.org/snap2
Back to top
View user's profile Send private message 
8-bit


Joined: 03 Apr 2007
Posts: 3368
Location: Oregon

PostPosted: Thu 11 Feb 2010, 14:04    Post subject:  

What would it take to add some code to check if the destination partition is mounted?
I ask this because one can write to /mnt/[partition or drive] of an unmounted partition as /mnt/[partition or drive] is just a directory used when mounting a partition or drive.
In the case of the destination being unmounted, the backup will appear to work, but it will not go to the destination as expected.
Do I make any sense?
Back to top
View user's profile Send private message 
Jim1911

Joined: 19 May 2008
Posts: 2457
Location: Texas, USA

PostPosted: Thu 11 Feb 2010, 14:30    Post subject:  

Hi lstandish,
Quote:
I am open to suggestions on how to clear up the snapshot vs mirror backup confusion. Of course, I try to avoid overwhelming the user with wordy explanations.

Consider just adding some of the comments that you provided me above to your nice Help file, like the bold faced markups below:

BACKUP METHODS

This program does backups of sets of source directories using rsync. There are
2 types of backups:
1. mirror
2. snapshot
NOTE: Select only one type of backup for each source directory.

'MIRROR' TYPE BACKUPS:
A mirror backup is just a single copy of the files in the source directories. Selecting "Mirror Backup Now" the first time will create a mirror backup of the source directory. Subsequent mirror backups will create a new mirror backup that reflects only the current state of the source directory. . . . .

'SNAPSHOT' TYPE BACKUPS:
This method is used to keep a series of backups, to allow recovery of files to
any of several different points in time. However, unlike a mirror backup, with snapshot backups you also have previous snapshots of your files, allowing access to files you have deleted from the source, and previous versions of files. A snapshot backup on first run results in a directory called recent.1. Subsequent snapshots will result in a 'recent.1' and a 'recent.2'. Recent.1 will be the most recent snapshot. recent.2 will be the previous one. . . . .
Cheers,
Jim
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Thu 11 Feb 2010, 14:39    Post subject:  

OK, I'll use most of your wording, but I think saying that "...subsequent mirror backups will create a new mirror backup..." might be confusing. There is in fact only one mirror backup which is updated each time the backup is done. snapshots, on the other hand, could each be said to be a "new backup."

Anyway, I'll try to fix up the docs and keep it simple and clear. Thanks for the suggestions!
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Thu 11 Feb 2010, 18:16    Post subject:  

8-bit, you wrote,
Quote:
What would it take to add some code to check if the destination partition is mounted?
I ask this because one can write to /mnt/[partition or drive] of an unmounted partition as /mnt/[partition or drive] is just a directory used when mounting a partition or drive.
In the case of the destination being unmounted, the backup will appear to work, but it will not go to the destination as expected.
Do I make any sense?

I used to have exactly the problem you describe when I backed up to my USB drive. That's why I created the option "Create missing backup storage dir" on the ADVANCED OPTIONS tab. Uncheck it to avoid the problem you described. (Hint: hover your mouse over the advanced options to see the intended use.)

_________________
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
http://standish.home3.org/snap2
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Thu 11 Feb 2010, 22:35    Post subject: snap2 4.1beta released  

snap2 4.1beta has been released.
Release notes: http://www.linuxbackups.org/node/36

_________________
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
http://standish.home3.org/snap2
Back to top
View user's profile Send private message 
Sylvander

Joined: 15 Dec 2008
Posts: 3444
Location: West Lothian, Scotland, UK

PostPosted: Fri 12 Feb 2010, 05:36    Post subject:  

@lstandish

Trying snap2 4.1beta:
1. It seems to be working well.

2. When I make recent.2 and the report and log show no differences, and I check and confirm that using dirdiff...
I use "View backup logs->Delete snapshot" [I don't want to keep a snapshot with no differences to the previous snapshot], and when done, recent.2 isn't renamed back to its previous name of recent.1 Confused
Wouldn't it be best to have that done?
I can do it manually, but wouldn't it be best done automatically by snap2?

3. "Report Deleted Files" is working well. Very Happy

4. Don't see the "year/month/date-hour:min" being added to the recent.x folder name.
You said you were going to do that.
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 5 of 11 [158 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Utilities
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1030s ][ Queries: 12 (0.0081s) ][ GZIP on ]