snap2 rotating snapshot backups for Puppy

Miscellaneous tools
Message
Author
keo01
Posts: 1
Joined: Thu 25 Feb 2010, 21:20

#106 Post by keo01 »

hello lstandish, your software seems to be what I was looking for...

I've one question, my idea is to "synchronize" files beetween desktop and laptop trough a backup media (USB, file server, etc).

The way I want to do this, is:
I make a change to a file (or a group of files) in PC1.
Backup it under request (it can be done with your program, right?) to a backup media (usb flash drive, NAS, etc).

Then when I want to make a change in the latest version of a file (either in PC1 or PC2 or elsewhere) , check the backup and get the latest version.
Save the changes and backup it again, so when I want work in the last version I can repeat the process.

Could this be possible? I don't have clear that I can backup the same file from two (or more) computers to only one snapshot(I mean, that don't be one snapshot for this file for PC1, and other snapshot for this file for PC2, etc).

Thanks.

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

#107 Post by lstandish »

Sorry, I was dealing with some urgent sys-admin work, and haven't visited this forum for a while. For some reason I did not get (or see) the emailed notice of a new post here.

To Sylvander and keo01:
I think that both of you are needing to back up from more than one OS to the same backup media. In my previous posts, I said I thought this was possible, but now I think that you will not be able to do this, at least not with the current snap2 version. It may already work for mirror backups, which do not involve hard links, but I'm not sure.

For snapshot type backups, which rely on hard links, I think any attempt to back up from 2 different operating systems to the same backup media will fail due to the fact the destination mount point directory on each OS will have different attributes. I believe I know of a workaround for this, already built into rsync, but I am too busy right now to investigate.

I'll post here later when I have had time to study this and perhaps make a new snap2 release that can ignore the source/destination mount directories.
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
[url]http://standish.home3.org/snap2[/url]

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#108 Post by Sylvander »

@lstandish

1. Whew!
Nice to know this phenomenon is real; not just my mind playing tricks.

Since last reporting, I've been making all snapshots from within Lighthouse-4.43f
Great Puppy, GREAT snap2.

The layout of snap2 seems a little strange at first encounter...
All those buttons and tabs together in one view...
But as I become accustomed to it, it's easy to use, and things are easy to get to because they're all available in that one view.

And now, of course, a snapshot typically takes only a minute or few to complete. :D 8)

2. Only improvement I can think of is to be able to also do restores from within snap2.

At the moment I would use Xfe.
Delete the whole of the contents of a partition...
Then "COPY-to..." from the snapshot folder holding [the backup copy of] all of the contents of that partition...
Back to the partition.

That involves a LOT of un-necessary deletions and restores...
Of folders and files that are identical in both locations.

I imagine if snap2 were doing that job it would only copy over where there were differences.

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

#109 Post by lstandish »

Glad snap2 is working out for you!

I have no plans to give snap2 file manager abilities, to use it for restores. For that, a 2-pane file manager works great. Just browse to the folders/files you want in one pane, and copy them to the destination in the other pane. I use mc (midnight commander) for this, available for Puppy (see attachment - this is a old package - not a pet - with an install script but it works). It has loads of cool features, and even allows you to have a remote server connection in one pane and the local filesystem in the other pane, to browse files, mark them, and restore from a remote server.

As I said, I hope to soon fix snap2 to optionally ignore mount points for both source and destination, to avoid trouble with using it between different OS's.
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
[url]http://standish.home3.org/snap2[/url]

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

snap2 v4.2 released

#110 Post by lstandish »

It has been a relatively long time since the last release of snap2, which was 4.1. I have been extremely busy with my small business, which involves harvesting and processing a wild tropical forest fruit called "carao".

In my last post, I mentioned my intention to allow snap2 to auto-mount partitions that are not mounted at backup time, and I made extensive code changes to allow auto-mount of both source and target partitions, for backup to both local and remote backup media.

Unfortunately, I ran into some issues with this, and ran out of free time. I therefore decided to put the auto-mount development on "hold" (sorry, Jim1911).

Meanwhile, I have some tips on how to use snap2 when backing up from more than one machine to the same backup storage. I understand that this is what Sylvander has tried to do. The following explanation is based on my theoretical understanding of how rsync works. That is, it is untested.

Suppose you specify a source path of /mnt/sda1 with backup storage as /mnt/mybackups. If you try to back up /mnt/sda1 from a distinct Puppy installation compared to the one used for the original backup, rsync will probably see a changed time stamp on the /mnt and /mnt/sda1 directories (which are distinct directories for each Puppy filesystem), and will therefore create a new backup copy rather than linking identical files together with hard links.

To avoid this for this example, tell rsync to ignore the /mnt/sda1 portion of the source path. This is done by adding a /./ to the end of the path. The source path becomes /mnt/sda1/./

In this case, rsync should not pay attention to the timestamp of directory /mnt nor directory /mnt/sda1. Moreover, it will not recreate this portion of the directory structure on the backup storage drive. Therefore, if the files under /mnt/sda1 are identical to files at the time of the previous backup, they should be linked together with hard links, even if backups are done from different installations of Puppy.
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
[url]http://standish.home3.org/snap2[/url]

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#111 Post by Sylvander »

@lstandish
Thanks for the tip, but I've been avoiding the problem by always using the same Puppy to make the snapshots.
I think I'll just continue doing that. :D

p.s. The more I use snap2, the more impressed I become. 8)

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

#112 Post by lstandish »

Thanks for the praise for my program, Sylvander. :)

I want to point out that the "engine under the hood" is the awesome rsync program. rsync is the product of the effort of many people, but the current maintainer of rsync is Wayne Davison. If you want to send him a thank-you note for his work on rsync, you can find his contact info here

In order for snap2 to get into the official Puppy release, users will have to recommend it to Barry, I guess.
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
[url]http://standish.home3.org/snap2[/url]

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#113 Post by Sylvander »

I've been praising/mentioning/suggesting snap2 both here and at www.pcguide.com/vb

I've been waiting for the most opportune moment to post a thread on it...
Probably here, with a link and explanation at the PC-Guide forums.

Here's my 1st mention of snap2 & dirdiff on my "Puppy Linux" thread in the Linux sub-forum at the PC-Guide forums.

I was the 1st to mention Puppy Linux there.
You'll notice there's now quit some interest.
I've been subject to sustained criticism/ridicule/attack [by 2 mod's] for what 1 individual called my "rabid" enthusiasm for Puppy.
Another accused me of believing and giving the impression that Puppy is the cure for all the ills known to mankind!

Quite false of course.
I'm simply enthusiastic for all of the wonderful things that Puppy [and programs like snap2] can do. :D 8)
I dream of the day when snap2 can restore as well as backup. :wink:

My inclination would of course be to have snap2 natively included in all future official Puppies.

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

#114 Post by lstandish »

Note to all users of snap2:

Version 4.3+ is now (as-of April 16, 2010) classed STABLE/TESTED.

Note that when upgrading snap2, all previous configuration files are carried over to the new version. That is, an upgrade will not cause you to lose your backup configuration.

Version 3.x should be considered obsolete.
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
[url]http://standish.home3.org/snap2[/url]

User avatar
abushcrafter
Posts: 1418
Joined: Fri 30 Oct 2009, 16:57
Location: England
Contact:

#115 Post by abushcrafter »

lstandish wrote:Sylvander, I found a great graphical directory comparison utility for you (called dirdiff), and make a pet package out of it (attached). You can compare up to 5 directories at once. It needs the Puppy tk/tcl libraries. Since they are listed as dependencies, they should be installed automatically.

Here's the Debian package description:
"Dirdiff can handle up to 5 trees. It displays a main window with a
list of the files which are different between the trees, with colored
squares to indicate the relative ages of the versions. A menu allows you to display the differences between any two of the versions in another window. Another menu allows you to copy the file from one tree to another."

I think I'll put it in the snap2 download area. (For Puppy 4.3.1)
Thanks just what I needed :D.

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#116 Post by BarryK »

Snap2 will be in the next release of Quirky:

http://bkhome.org/blog/?viewDetailed=01572
[url]https://bkhome.org/news/[/url]

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#117 Post by Sylvander »

SUPERB! :D

snap2 is now my number-1 backup program.

And I've used a fair number over the years, and done a lot of backing-up.

The FREE "Seagate Disk Wizard - Bootable Media" is my number-2 backup method [it makes images of chosen partitions or whole drives, and (unlike a running Puppy) is unaffected by a (locked) Linux swap partition on the drive to be imaged].
This is a FREE rebranded version of "Acronis True Image".

There was a small problem in using snap2:
May now be fixed: don't know.
It wasn't possible to use just any snap2 on any Puppy, and have them all make use of the same snapshots.
The different snap2's treated another snap2's snapshots as though they didn't exist.
Hence it's necessary to always use the same Puppy with it's installed snap2 to make ALL snapshots.

User avatar
alienpup
Posts: 9
Joined: Sun 18 Oct 2009, 16:18
Location: Arizona, USA

snap2 compression

#118 Post by alienpup »

lstandish wrote:[Version 4.3+ is now (as-of April 16, 2010) classed STABLE/TESTED.
Snap2 is wonderful. My other favorite backup program is storeBackup. Dependancies make it rather impractical for use in Puppy, but storeBackup does offer the option of by-file compression. I realize the implications of this but do you forsee snap2 offering a like option one day? Compression can cut storage requirements in half.

Thanks

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

#119 Post by lstandish »

Sorry, alienpup, I didn't see your post until today.

Compression of backups will not ever, I think, be offered by snap2, for several reasons.

1. snap2 is based on rsync. One of the greatest things about rsync is its "delta" algorithm. When backing up a previously-backed-up file over a network to remote backup storage, rsync running on the local machine collaborates with an rsync instance running on the backup server to determine which portions of files need to be backed up, and uploads only those differences ("deltas"). That can't be done if files are compressed. However, rsync does compress what it transmits, again to reduce bandwidth.

2. snap2 uses hard links to duplicate identical copies of files in backup storage. This takes up almost no space. Since in most cases only a small number of files are modified or added with each new backup, this results in a tremendous savings of disk space on the backup server, making compression unnecessary in order to save disk space.

3. One of snap2's features is easy examination and recovery of a previous version of a file from the backup storage area. One can browse to a given version of a file in the backups, examine it, compare it to the current working copy (using cool file comparison utilities), and based on this examination copy it back over to the source directory, or anywhere else. In order to handle backups this way, they cannot be compressed.

By the way, I used to be a fan of compressed backups, until I discovered the huge disk savings given by "hard link magic." For example, I back up my desktop PC (in Costa Rica) to my remote backup server (in USA). Each of my home directory backups has a total size of about 1.4 gigs. I have 40 of these "full backups" on my remote server. But, the total space used on the backup server is only about 2.8 gigs! And each backup, over my modest 1 Mbit broadband connection, usually only takes a couple of minutes.
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
[url]http://standish.home3.org/snap2[/url]

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#120 Post by Sylvander »

Problem! :(

1. Installed snap2-4.16.pet [and dirdiff-2.1-4fixed2.pet] within Boxpup-431 run off a CD-RW with pupsave.2fs on a 1.3GB ext3 partition on an 8GB Flash Drive.
Installed using pet files downloaded to and held on a FAT32 partition filling a [separate/different] 1GB Flash Drive.
Rebooted and during shut-down CHOSE [previously made necessary coding changes] to do the the only save of the session.
No saves DURING the session.

2. dirdiff appears OK in the [Utility] menu, but snap2 items don't appear.

3. Both snap2.desktop & snap2snapshot.desktop exist within the /usr/share/applications folder.

4. Ran obmenu-refesh in a terminal [the "refresh menu" item in the menu seemed to do nothing], and it reported it generated "/root/.config/openbox/menu.xml".
Took a look inside that, and there are no entries for the snap2 pair.

5. snap2 runs using a terminal command.
dirdiff doesn't run using either a terminal command, or the entry in the menu.
Both snap2 & dirdiff appear as installed in the PPM.

6. Any ideas?

User avatar
edoc
Posts: 4729
Joined: Sun 07 Aug 2005, 20:16
Location: Southeast Georgia, USA
Contact:

#121 Post by edoc »

Has snap2 been tested with Lucid 5.1 or fatdog64, please?

I can really use this valuable app but don't want to create the mess I am trying to avoid by rushing in blindly.
[b]Thanks! David[/b]
[i]Home page: [/i][url]http://nevils-station.com[/url]
[i]Don't google[/i] [b]Search![/b] [url]http://duckduckgo.com[/url]
TahrPup64 & Lighthouse64-b602 & JL64-603

Jim1911
Posts: 2460
Joined: Mon 19 May 2008, 20:39
Location: Texas, USA

#122 Post by Jim1911 »

edoc wrote:Has snap2 been tested with Lucid 5.1 or fatdog64, please?
It works fine with Lucid, don't know about fatdog64. You'll find it in Lucid under More Pets in QuickPet. However, you need to download lstandish's latest since it has had a few fixes.

User avatar
edoc
Posts: 4729
Joined: Sun 07 Aug 2005, 20:16
Location: Southeast Georgia, USA
Contact:

#123 Post by edoc »

OK, thanks! Just downloaded 4.16.

Will try to give it a try a little later - have a project I must complete first.
[b]Thanks! David[/b]
[i]Home page: [/i][url]http://nevils-station.com[/url]
[i]Don't google[/i] [b]Search![/b] [url]http://duckduckgo.com[/url]
TahrPup64 & Lighthouse64-b602 & JL64-603

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

#124 Post by lstandish »

Thanks, Jim, for answering this.

I worked very hard to prepare my snap2 for inclusion in Debian Linux, and when it was ready I realized that gtkdialog is not to be included in the next Debian. This has really "taken the wind out of my sails," since Debian is my "daily use" distro. However I expect snap2 will survive in puppy as long as gtkdialog is included.

I just installed the latest snap2 (2-4.16) in Puppy 5.1, and it appears the snap2 GUI menu entry is not created - sorry! I will try to fix this soon and make a new release.

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

Menu problem fixed

#125 Post by lstandish »

The lack of menu entries is now fixed in snap2-4.17.
Download http://files.lstandish.com/

(The problem was in syntax in the .desktop files for the 2 menu entries. '[$e]' can be used in KDE but not in Puppy.)
--
Lloyd
snap2 rotating snapshot backups for Puppy/Debian Lenny/Ubuntu
The convenience of full backups with the speed and disk economy of incremental backups
[url]http://standish.home3.org/snap2[/url]

Post Reply