Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Tue 15 Oct 2019, 00:17
All times are UTC - 4
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Utilities
snap2 rotating snapshot backups for Puppy
Post new topic   Reply to topic View previous topic :: View next topic
Page 6 of 11 [159 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 Next
Author Message
Jim1911

Joined: 19 May 2008
Posts: 2460
Location: Texas, USA

PostPosted: Fri 12 Feb 2010, 17:44    Post subject:  

Hi lstandish,

It's working well.

One issue, there still isn't a prominent caution ie., BEFORE PROCEEDING, MAKE SURE THAT THE SOURCE AND DESTINATIONS ARE MOUNTED, otherwise backups will be deleted. I tested this and sure enough, with a partition unmounted, a previous Mirror backup Is deleted very quickly. This is a necessary reminder for the first time user and a good reminder for forgetful old timers like myself.

You have really improved the content of the help file.

Thank you for this fine utility program,
Jim
Back to top
View user's profile Send private message 
8-bit


Joined: 03 Apr 2007
Posts: 3425
Location: Oregon

PostPosted: Fri 12 Feb 2010, 18:12    Post subject:  

Jim1911,
I have some thing for you to try.
Say your destination is set as /mnt/sda3.
Also let us say you forgot to mount /dev/sda3
After doing a backup/snapsot, and before mounting /dev/sda3
check /mnt/sda3 by opening that directory and see if the backup file is there.
That is something I was trying to express in a previous post.
That before a backup is done, the destination should be checked to see if it is mounted.
I have as a test copied files to an unmounted drive, NOT USING /dev/[drive/partition] but instead, using /mnt/[drive/partition].

The files copied fine to the unmounted /mnt/drive directory.
But they would disappear on mounting that drive.
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Fri 12 Feb 2010, 18:56    Post subject:  

Sylvander,
Part of the idea of a snapshot backup is to document the state of the files on a given date. If you delete the snapshot just because it is the same as a previous snapshot, you lose that documentation. The only reason to delete it would be, I think, to save disk space. Yet, the overhead (extra disk space) used by an identical snapshot is very small (only the size of the hard links).

In sum, I don't recommend deletion of identical snapshots, unless you are very low on backup storage space. snap2's built in support for deleting snapshots is intended mainly to be used when a backup was interrupted (and therefore incomplete).

I don't think there is any advantage to having snap2 try to rename snapshots to fill up 'holes' left by snapshot deletions.

Sorry, I have not yet had time to get to study what will be involved to include the date-time stamp in the snapshot directory name.

Jim1911,
Hmmm, I never considered the scenario of the *source* partition not being mounted! To fix that, snap2 would have to be able to tell when a source directory was really deleted, and when the source partition is only not mounted. I'll try to make snap2 detect the unmounted partition case, or if that is not practical, I'll add a warning about unmounted source partitions to the docs.

8-bit,
I see your problem: if /mnt/sda3 is specified as the backup storage location, the directory /mnt/sda3 will exist whether or not the partition is mounted. That means the snap2 setting to *not* allow creation of a missing backup storage directory will have no effect, and backup will proceed.

To prevent snap2 from proceeding with a backup when the destination partition is not mounted, specify a backup storage folder (on the BACKUP STORAGE tab) like this one: /mnt/sda3/mybackups Now, when sda3 is not mounted, /mnt/sda3/mybackups will not exist, and if the 'Advanced' setting to allow creation of missing backup storage folder is unchecked, then snap2 will not do the backup.

(You can move your backups in /mnt/sda3/ to /mnt/sda3/mybackups/ without having to actually move data, so it will be fast. If you do this remember to update the backup storage directory in snap2.)

_________________
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
http://standish.home3.org/snap2
Back to top
View user's profile Send private message 
Sylvander

Joined: 15 Dec 2008
Posts: 4422
Location: West Lothian, Scotland, UK

PostPosted: Sat 13 Feb 2010, 06:40    Post subject:  

1. "Part of the idea of a snapshot backup is to document the state of the files on a given date"
Can't think why I'd be interested in "the state of the files on a given date". Confused
That's not something that's useful to me.
What I really want is a series of snapshots of [the contents of partitions? in] DIFFERENT STATES, so I can perhaps restore one of those to get back to that state...
Or perhaps to recover a lost file, or [less likely] one or more files in a particular state.

2. "The only reason to delete it would be, I think, to save disk space. Yet, the overhead (extra disk space) used by an identical snapshot is very small"
(a) Imagine making a great list of snapshots, that are all [or perhaps only nearly all] IDENTICAL.
But you don't know [unless you expend time and effort], which are identical to which.
For my purposes, those identical snapshots are only clutter getting in the way.
[Perhaps there could be a tick-box to not save a snapshot that is identical to the previous snapshot? = CHOICE]
It's little consolation to know that at least they don't use much "real" space.
Which raises another important point as follows below:

(b) If I'm looking for a suitable snapshot to restore the way [an operating system? on] a partition used to be, when it worked a certain way [rather well?]...
I need something...
Some information about the snapshot...
That tells me what I need to know.
Does snap2 allow a user to make notes and associate them with a snapshot?
[Can't check here; this OS doesn't have snap2 installed]

3. "I don't think there is any advantage to having snap2 try to rename snapshots to fill up 'holes' left by snapshot deletions"
I like to keep things orderly.
If the folder names were not = snap.1, snap.3, snap.7, but rather a series of dates which were in date order, then it wouldn't look "untidy".
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Sat 13 Feb 2010, 11:23    Post subject:  

I guess we have different ideas about snapshot backups. I will rethink this, but for now I should should state my premises. I think my premises about snapshot backups represent "conventional wisdom" on this. (To a large extent, snap2 was developed based on these premises.)

In the first place, I think backups should be run on a regular basis, not sporadically. snap2 includes a backup scheduler to make automated, regular backups easy to do.

My cron-triggered automatic backup job is is run at 9:00 AM, and I don't mind that often only a few files are changed between one snapshot and the next. Sometimes no important files at all are changed, but I don't think that makes the backup unnecessary.

When I examine my snapshot backups, it is important to me to see that backups were run every day. If there is a day when no backup was run, I know that something "went wrong" that day. In my case, that usually happens when my computer is not on at 9:00 AM.

If I need to recover a file and I want the latest version, I look in yesterday's snapshot. The fact that yesterday's snapshot is there is my assurance that I am really getting the latest version of a file. (This is the "documentation" I spoke of.)

If I were to delete the last three days' snapshots because they did not contain important changes, then at file recovery time, how do I know that I'm getting the LATEST version? I would have to use a 3 or 4 day old snapshot and hope that it is really represents yesterday's state of my files, even though the snapshot date stamp is 3 or 4 days old.

For the same reason, I want to see that my "weekly" snapshots have date stamps about one week apart, etc.

The new log file viewer makes it very fast and easy to browse snapshots and see exactly what files were backed-up each time.

There is at the present time no way to add comments to a snapshot backup. I might add that, but as you can imagine it does not fit very well into the idea of regular, automated snapshot backups.

I will try to accommodate as far as possible and practical different ways to use snap2. I'm not saying that my premises on snapshot backups are the only "right" or useful ones. For example, snap2 was originally intended to back up data files, not an entire OS. I'll try to keep Operating System partition backup in mind.

_________________
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
http://standish.home3.org/snap2
Back to top
View user's profile Send private message 
Jim1911

Joined: 19 May 2008
Posts: 2460
Location: Texas, USA

PostPosted: Sat 13 Feb 2010, 11:48    Post subject:  

Hi Istandish,
Quote:
Hmmm, I never considered the scenario of the *source* partition not being mounted! To fix that, snap2 would have to be able to tell when a source directory was really deleted, and when the source partition is only not mounted. I'll try to make snap2 detect the unmounted partition case, or if that is not practical, I'll add a warning about unmounted source partitions to the docs.
Your idea of automatic detection is a much better choice than a warning. In my case, I have over 12 partitions and usually the only one mounted is the one that I'm working on, so it is very important that the partitions be mounted before running a backup. It would be great to have snap2 mount the necessary partitions, especially in the case of automated backups. I hope this won't be too difficult to achieve.
Cheers,
Jim
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Sat 13 Feb 2010, 12:03    Post subject:  

I spent hours yesterday coding to detect when a directory is on a mounted partition, and I had to give up. I got it working for cases where the mount point is listed in fstab, but USB drives have mount points that just "appear" out of nowhere. The only way I can think of would be to allow the user to specify a device file and mount point for each source directory - or for each "backup set". However, I have not wanted to limit source directories for a backup set to a single partition.

(I don't think there is a problem with unmounted destination directory, if the "Allow creation of missing destination dir" option is unchecked.)
Back to top
View user's profile Send private message 
Jim1911

Joined: 19 May 2008
Posts: 2460
Location: Texas, USA

PostPosted: Sat 13 Feb 2010, 13:10    Post subject:  

lstandish wrote:
The only way I can think of would be to allow the user to specify a device file and mount point for each source directory - or for each "backup set". However, I have not wanted to limit source directories for a backup set to a single partition.
This would work well when setting up automated backups.

I've just tried snap2 on dpup484b4. It appears to install fine, however, it gives an error when trying a backup "Process exited abnormally with status 0. Press any key to close tab." Previously, I have been running it on Barry's 432 experimental version which has worked great.

Jim Smile
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Sat 13 Feb 2010, 13:41    Post subject:  

I have code written now to take an optional mount point and device file on the same line as the source directory, lke this:

# source directories
/mnt/sde1 DEVICE=/dev/sde1 MOUNTPT=/mnt/sde1

It will do the mount if necessary. After backup, a previously unmounted source partition is unmounted again. Does this look OK?

_________________
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
http://standish.home3.org/snap2
Back to top
View user's profile Send private message 
Jim1911

Joined: 19 May 2008
Posts: 2460
Location: Texas, USA

PostPosted: Sat 13 Feb 2010, 13:49    Post subject:  

lstandish wrote:
It will do the mount if necessary. After backup, a previously unmounted source partition is unmounted again. Does this look OK?
Great! Very Happy
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Sat 13 Feb 2010, 13:52    Post subject:  

On the dpup484b4 error, make sure rsync is available (in $PATH), and that /tmp is writable. Does the snap2 GUI open OK, failing only upon "...Backup Now"?
Back to top
View user's profile Send private message 
Jim1911

Joined: 19 May 2008
Posts: 2460
Location: Texas, USA

PostPosted: Sat 13 Feb 2010, 14:23    Post subject:  

lstandish wrote:
On the dpup484b4 error, make sure rsync is available (in $PATH), and that /tmp is writable. Does the snap2 GUI open OK, failing only upon "...Backup Now"?
Yes, it opens fine and source and destinations set up. That was just FYIO, no need to pursue trying to get it to work on dpup since a new beta for dpup should be out soon. It works good on pups 431 and 432, I'll try it on Lighthouse which is based on 431.
Jim
Back to top
View user's profile Send private message 
Sylvander

Joined: 15 Dec 2008
Posts: 4422
Location: West Lothian, Scotland, UK

PostPosted: Sun 14 Feb 2010, 03:32    Post subject:  

@lstandish
Regarding your post above at 11:23 on Jan 13th...
1. Consider the following "thought experiment".

(a) Imagine I had configured snap2 to automatically make a snapshot every hour, on the hour.
24 snapshots a day, every day of every year.
That's a lot of snapshots, right?

(b) Now lets imagine that most of those snapshots happen to fall into groups of identical snapshots.
Perhaps there were differences only once a week, or once a month.
That seems to me to rather a lot of clutter.
Fairly pointlessly listing lots of identical snapshots.

(c) Now what if:
When a snapshot was being made...
If it was discovered to be identical to the previous snapshot...
Some alternative automatic action was taken by snap2. [But only if a tick-box was ticked]
Say the snapshot was aborted...
And perhaps some logfile was made that listed that the snapshot process had been completed, and found to be identical to such-and-such a snapshot.

(d) And all following snapshots that were found to be identical were similarly logged.

(e) Then when a snapshot was found to be no-longer-identical...
It would be completed and saved.

(f) So you end up with a series of snapshots that are DIFFERENT to each other.
Because they include at least ONE item that is not identical.
This would let the user instantly see WHICH snapshots were different to each other...
And also which snapshots were run, and found to be identical to those, so aborted.

(g) It would minimize the number of snapshots listed/saved.
Yet still allow the user to know which snapshot applied to a certain point in time.
And a user like myself, who wasn't interested in such things would also be happy.

2. You may consider this a crazy idea that would make snap2 much too complex. Sad
But it would certainly meet my needs even better than now.
I'd be going "WOW, that's cool"! Very Happy So FLEXIBLE! Cool
Back to top
View user's profile Send private message 
8-bit


Joined: 03 Apr 2007
Posts: 3425
Location: Oregon

PostPosted: Sun 14 Feb 2010, 12:10    Post subject:
Subject description: Detecting mounted destination
 

lstandish wrote:
I spent hours yesterday coding to detect when a directory is on a mounted partition, and I had to give up. I got it working for cases where the mount point is listed in fstab, but USB drives have mount points that just "appear" out of nowhere. The only way I can think of would be to allow the user to specify a device file and mount point for each source directory - or for each "backup set". However, I have not wanted to limit source directories for a backup set to a single partition.

(I don't think there is a problem with unmounted destination directory, if the "Allow creation of missing destination dir" option is unchecked.)


Look at the code in "Hot Backup for frugal installs found Here.
It may shed some light on how to detect if source or destination is mounted.
The detection code works there.
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Sun 14 Feb 2010, 13:23    Post subject:  

Hi Sylvander,
Thank you for your thoughts on duplicate snapshots. What you suggest is possible, but after giving this some thought I realized that keeping identical snapshots around has some unexpected advantages, due to the discarding of snapshots that happens as snapshots are "rotated."

Snap2 does not keep every snapshot backup. It only keeps, for example, 'recent' snapshots up to the configured maximum. After that, it "thins them out". After the 'recent' snapshots reach the maximum, the oldest one is only 'promoted' to being a 'daily' snapshot if it is at least 24 hours newer than the newest daily snapshot. If not, it is discarded, no matter how much unique data it may have.

In a similar way, once you have reached the configured maximum of 'daily' backups, the oldest one is either 'promoted' to being a weekly backup (if it is 7 days newer than the newest weekly snapshot), or discarded. If you run snap2 once per day, only one out of 7 snapshots will survive to become a 'weekly' snapshot.

So, no matter how many times per day you run snap2, the limits you have set on recent, daily, weekly, and monthly backups will put a ceiling on the total number of snaphots that can exist. The out-of-the-box values for each of these add up to a total of 26 snapshot backups that can exist at one time. This is of course configurable, but I don't think many people would need to keep more than about 50 snapshots.

Now to explain the advantage of keeping duplicate snapshots. Suppose we have these snapshots:

daily.6 (2010-02-11)
daily.7 (2010-02-10)
daily.8 (2010-02-09)
weekly.1 (2010-02-04)

Daily.6 has no changes with respect to daily.7. But daily.7 has changes with respect to the daily.8.

Now, suppose we have snap2 configured to keep 8 daily snapshots. So the next time snap2 is run, daily.8 will be discarded, since it is too close in time to weekly.1. The next snapshot that is eligible to become a 'weekly' is daily.6

So, daily.8 and daily.7 are doomed to disappear in only 2 more days (assuming backups are done daily). But daily 6, which also holds the unique data in daily.8, will be promoted to 'weekly' and will survive for several more weeks.

Now, you would have deleted daily.6 because it had no unique data with respect to daily.7. By doing the manual deletion, you will have greatly shortened the lifespan of the data, due the way snapshot promotion/deletion works.

_________________
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
http://standish.home3.org/snap2
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 6 of 11 [159 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Utilities
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1080s ][ Queries: 12 (0.0356s) ][ GZIP on ]