Scripts to take backup of modified files only.

How to do things, solutions, recipes, tutorials
Post Reply
Message
Author
snayak
Posts: 422
Joined: Wed 14 Sep 2011, 05:49

Scripts to take backup of modified files only.

#1 Post by snayak »

Dear All,

After trying to take backup of modified files only(http://www.murga-linux.com/puppy/viewtopic.php?t=82975), I came up with two small scripts. Here I am sharing the scripts, please use it if you find it good. Please suggest improvements.

This script automatically remembers upto what date you have taken the backup.

Now it will start taking your backup since 01 Jan 2013. Then it remembers the date on which you have taken the backup. Then onwards simply run this script, it will take backup of files modified since last backup!


backup.sh
#!/bin/sh

last_N_days=30
exception1="dont_backup.txt"

src_dir=""
dst_dir=""
backup_log=""

usage()
{
echo Usage: backup.sh src-dir dst-path
echo Example: backup.sh /mnt/sda5 /mnt/sdb1
echo Modify Pattern for \"files to be excluded from back-up\" in backup.sh file.
}

echo hello

if [ $# != 2 ]
then
usage
else

backup_log=`pwd`
backup_log=$backup_log/backup.log
echo > $backup_log
echo check $backup_log for files excluded from backup

this_script=`realpath "$0"`
src_dir=$1
dst_dir=$2

echo you have given $1 as source directory
echo you have given $2 as destination directory

if [ ! -d $src_dir ]
then
echo source directory $src_dir does not exist
exit
fi

if [ ! -d $dst_dir ]
then
echo destination directory $dst_dir does not exist
exit
fi

cd $src_dir
src_dir=`pwd`
cd - > /dev/null

cd $dst_dir
dst_dir=`pwd`
cd - > /dev/null

echo complete path for source directory is $src_dir
echo complete path for destination directory is $dst_dir

src_dir_to_copy=`echo $src_dir | awk -F/ '{print $NF}'`
echo backup will copy complete $src_dir_to_copy/ directory to $dst_dir

time_stamp=`cat $this_script | tail -1 | awk '{print $2}'`

if [ "$time_stamp" = "" ]
then
echo Run it now!
echo \#Upto `date +%Y%m%d%H%M` >> $this_script
exit
else
echo taking backup from $time_stamp
fi

touch -t $time_stamp $this_script

cd `dirname "$src_dir"`
#find $src_dir_to_copy -type f -ctime -$last_N_days | while read file_name;
find $src_dir_to_copy -type f -cnewer $this_script | while read file_name;
do
file_excluded=`echo $file_name | grep $exception1`
if [ $file_excluded ]
then
echo $file_name >> $backup_log
else
new_path_for_file=$dst_dir/$file_name
new_dir_for_file=`dirname "$new_path_for_file"`
echo directory $new_dir_for_file will be created if does not exist
mkdir -p "$new_dir_for_file"
echo $file_name file will be copied in to it
cp "$file_name" "$new_dir_for_file"
fi
done
cd - > /dev/null

echo ========Files excluded from backup========
cat ./backup.log
echo ========Files excluded from backup========

(echo "g/^#Upto 201/d"; echo 'wq') | ex -s $this_script
echo \#Upto `date +%Y%m%d%H%M` >> $this_script

fi

echo done


#=====keep_this_line=====
#Upto 201301010000
restore.sh
#!/bin/sh

src_dir=""
dst_dir=""

usage()
{
echo Usage: restore.sh src-dir dst-path
echo Example: restore.sh /mnt/sdb1/sda5 /mnt/sda6
}

echo hello

if [ $# != 2 ]
then
usage
else

src_dir=$1
dst_dir=$2

echo you have given $1 as source directory
echo you have given $2 as destination directory

if [ ! -d $src_dir ]
then
echo source directory $src_dir does not exist
exit
fi

if [ ! -d $dst_dir ]
then
echo destination directory $dst_dir does not exist
exit
fi

cd $src_dir
src_dir=`pwd`
cd - > /dev/null

cd $dst_dir
dst_dir=`pwd`
cd - > /dev/null

echo complete path for source directory is $src_dir
echo complete path for destination directory is $dst_dir

src_dir_to_copy=`echo $src_dir | awk -F/ '{print $NF}'`
echo restore will copy complete $src_dir_to_copy/ directory to $dst_dir

echo directory "$dst_dir/$src_dir_to_copy" will be created if does not exist
mkdir -p "$dst_dir/$src_dir_to_copy"
cp -v -r $src_dir/* $dst_dir/$src_dir_to_copy

fi

echo done

Sincerely,
Srinivas Nayak
Last edited by snayak on Wed 06 Feb 2013, 08:21, edited 4 times in total.
[Precise 571 on AMD Athlon XP 2000+ with 512MB RAM]
[Fatdog 720 on Intel Pentium B960 with 4GB RAM]

[url]http://srinivas-nayak.blogspot.com/[/url]

Bruce B

#2 Post by Bruce B »

Do you have all the necessary escape provisions? This is particularly important when distributing the scripts. We should presume the user doesn't understand the innards of the script.

src_dir=$1

Example: if $1 is innacurate or not entered what happens?

I do like this.

[ ! $1 ] && echo missing argument && exit
src_dir=$1
[ ! -d $src_dir ] && echo directory $src_dir doesn\'t exist && exit

snayak
Posts: 422
Joined: Wed 14 Sep 2011, 05:49

#3 Post by snayak »

Dear Bruce,

Many thanks for your suggestion. I got your point.
I thought,

cd $src_dir

shall fail in case src_dir is something wrong and the user shall be able to see the error message. I have everywhere provided echo to display whatever is going inside script.

Seeing your comments, to gain a bit more knowledge, I checked these two pages for understanding "[ expression ]".

1. http://en.wikibooks.org/wiki/Bash_Shell ... conditions

2. http://www.gnu.org/software/bash/manual ... xpressions

What is the difference between "[ expression ]" and "[[ expression ]]" ?

In link2, the first line reads "Conditional expressions are used by the [[ compound command and the test and [ builtin commands." What does it mean?

Sincerely,
Srinivas Nayak
[Precise 571 on AMD Athlon XP 2000+ with 512MB RAM]
[Fatdog 720 on Intel Pentium B960 with 4GB RAM]

[url]http://srinivas-nayak.blogspot.com/[/url]

snayak
Posts: 422
Joined: Wed 14 Sep 2011, 05:49

#4 Post by snayak »

Scripts updated to automatically remember the last backup date and time. So next time when you run this script, it will take backup of files modified since last backup!
[Precise 571 on AMD Athlon XP 2000+ with 512MB RAM]
[Fatdog 720 on Intel Pentium B960 with 4GB RAM]

[url]http://srinivas-nayak.blogspot.com/[/url]

snayak
Posts: 422
Joined: Wed 14 Sep 2011, 05:49

#5 Post by snayak »

Script updated to use absolute path of its own.

Learnings:
1. 'realpath' command shows the absolute path of a file.
2. When a script is in /bin, and the script is executed, $0 holds its full path, like '/bin/a.sh'
[Precise 571 on AMD Athlon XP 2000+ with 512MB RAM]
[Fatdog 720 on Intel Pentium B960 with 4GB RAM]

[url]http://srinivas-nayak.blogspot.com/[/url]

snayak
Posts: 422
Joined: Wed 14 Sep 2011, 05:49

#6 Post by snayak »

backup script modified to backup files downloaded since last backup.

Learning:

"find /mnt/sda5 -cnewer /tmp/b.txt" searches all files whose "inode information changed" since the "last modification time" of /tmp/b.txt.
[Precise 571 on AMD Athlon XP 2000+ with 512MB RAM]
[Fatdog 720 on Intel Pentium B960 with 4GB RAM]

[url]http://srinivas-nayak.blogspot.com/[/url]

heenry
Posts: 1
Joined: Wed 06 Mar 2013, 13:40

#7 Post by heenry »

I'm wondering about the escape provisions also as this is a fundamental part of scriipt distribution.

Post Reply