Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sun 25 Feb 2018, 20:09
All times are UTC - 4
 Forum index » House Training » HOWTO ( Solutions )
Moving Directories in Linux w/ Rsync
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [4 Posts]  
Author Message
s243a

Joined: 02 Sep 2014
Posts: 828

PostPosted: Sun 07 Jan 2018, 06:29    Post subject:  Moving Directories in Linux w/ Rsync  

If you try to move files using rox by dragging from one folder to another and it fails for some reason; for example:
1. I/O error (Maybe you have a flaky USB connection or drive)
2. Too Much Ram being used
3. Insufficient space in the destination drive.
4. Insufficient permissions
5. Power outage
6. etc.

You could end up with a mess where some of your files were transfered and others wern't. Worse yet ROX is horrible at merging directors and I'm not sure if it identifies when a file is only partially transfered.

The solution that I'm trying as we speak uses rsync to do this task instead of Rox but we'll do a few tricks to make it more GUI-Like.

I've created three scripts:

The first script is used to save the path to the source directory that we want to move files from. This file -- located in the /tmp directory -- will be read later to get the path.


/usr/sbin/s243a_Fm
Code:

#!/bin/sh
OLD_IFS=IFS
IFS=""
WD="/tmp/s243a"
#https://stackoverflow.com/questions/793858/how-to-mkdir-only-if-a-dir-does-not-already-exist
[ -d $WD ] || mkdir -p $WD
echo $(readlink -f "$1") > "$WD/fm.txt"
IFS="$OLD_IFS"


To call this script you can either navigate to the directory in ROX and either pres the tilda key "~" to get a console or alternatively make the script clickable by rightclicking on the director, selecting "open with", then selecting "customize"

This will open the following directory in Rox

/root/.config/rox.sourceforge.net/OpenWith

Now symbolically link this script to the directory by draging the script into this directory and selecting "link relative"

Next we create a similar script for the destination directory:
/usr/sbin/s243a_To
Code:

#!/bin/sh
OLD_IFS=IFS
IFS=""
WD="/tmp/s243a"
#https://stackoverflow.com/questions/793858/how-to-mkdir-only-if-a-dir-does-not-already-exist
[ -d $WD ] || mkdir -p $WD
echo $(readlink -f "$1") > "$WD/to.txt"
IFS="$OLD_IFS"

and also make it clickable.

Finally, having saved the source path and destinations path to a file we can use rsync to move the files between these two locations. The script that we will use to do this will look like:
Code:

#!/bin/sh
OLD_IFS=IFS
IFS=""
WD="/tmp/s243a"
TO="`cat $WD/to.txt`"
FM="`cat $WD/fm.txt`"
#
# https://unix.stackexchange.com/questions/78375/move-files-and-delete-directories-with-rsync
#
rsync -avE --size-only --remove-source-files "$FM" "$TO" && \
find "$FM"  -depth -type d  -empty -exec rmdir "{}" \;
IFS="$OLD_IFS"

We can also make this script clickable.

However, it is better to run it in the console so that you can watch the progress:

From any path in the console simply type
Quote:

s243a_Rsync


and then the transfer will begin. Some notes on the options.

The -a is the archive option. It will preserver information about the file, such as timestamps, ownership and permissions. It will also do recursion.

I added the E option because I'm not sure if the -a option preserves execution permissions.

I added --size-only because I was thinking of the case where someone like me tried using rox to move a directory and it and failed. My theory was that if the size was different then the transfer wasn't completed and in this case we always wanted to take the original file if the size differed.

The --remove-source-files option removes files after they are synced but will not delete directories. The following line after rsync (i.e. the find command)
Code:

find "$FM"  -depth -type d  -empty -exec rmdir "{}" \;


is only executed if rsync succeeds. This command will delete only empty directories from the source folder.

I suppose that one might want to do this anyway regardless of whether or not rsync succeeds but only after they are sure that all the files are moved. Perhaps we could create a fourth script to delete empty directories and also make it clickable.

I would like to add more options to this scipt and also add a gui.

I'd probably try GTK dialog for the GUI. It would work as follows:

When you type
Code:

s243a_Rsync

A dialog opens up with the source and destination paths shown as previously set by the other two scripts. By each path there would be a button that you could press to select a different path. There would also be a checkboxes so that you could change some of the rsync options. For instance maybe I want to always skip a file if it exists in the destination folder. I might do this if I transfered some of the files to another folder and was trying to merge them back in but I didn't know if all of these files in this other folder were complete.

Another thing that we might want to try is a delete as we go option.

We could do this as follows:

With the verbose option selected in rsync it outputs to the console the name of each file that it moves (or syncs).

We could pipe this output into a script which watches these names and waits for a directory name change. Once a directory change is detected it would then empty the directory.
Back to top
View user's profile Send private message 
musher0


Joined: 04 Jan 2009
Posts: 11706
Location: Gatineau (Qc), Canada

PostPosted: Sun 07 Jan 2018, 14:25    Post subject:  

Interesting lesson about rsync. Thanks.
_________________
musher0
~~~~~~~~~~
"Logical entities must not be multiplied beyond necessity." | |
« Il ne faut pas multiplier les entités logiques sans nécessité. » (Ockham)
Back to top
View user's profile Send private message 
slavvo67

Joined: 12 Oct 2012
Posts: 1502
Location: The other Mr. 305

PostPosted: Wed 17 Jan 2018, 20:11    Post subject:  

I didn't use your examples but I did successfully move a full WoofQ with your Rsync idea. The cp command just didn't bring everything over properly (links, etc.).

Thanks,

Slavvo67
Back to top
View user's profile Send private message 
Flash
Official Dog Handler


Joined: 04 May 2005
Posts: 12772
Location: Arizona USA

PostPosted: Wed 17 Jan 2018, 22:22    Post subject:  

Forum member don570 made a .pet called Copy-fast that uses rsync. I use Copy-fast all the time to move directories containing audio books to my mp3 player from flash drives.
Here's a topic about Copy-fast in the forum.
copy-fast-2.2.pet
Description 
pet

 Download 
Filename  copy-fast-2.2.pet 
Filesize  7.99 KB 
Downloaded  18 Time(s) 
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [4 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » HOWTO ( Solutions )
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0765s ][ Queries: 14 (0.0182s) ][ GZIP on ]