Thumb drive slow if multiple file transfer?

Using applications, configuring, problems
Post Reply
Message
Author
firak
Posts: 175
Joined: Mon 14 Apr 2008, 14:07

Thumb drive slow if multiple file transfer?

#1 Post by firak »

About writing speed of usb sped drives.

I notice that the time increase exponentially when I run moare than one file transfer instance (rox) .
I did not measure exactly, but this is my impression.
I have a slow pen drive 4 MB/sec writing.

Writing on it 1G - it takes approx 5 min.
Transferring simultaneously 3 instance of 1G, it takes aprox 30 minutes.

It is better to transfer files one by one?
How can i do it, in one command, if files are form different folders?
There is any trick suggested for rox or tux commander?


(I run a atom based netbook with macpup528 )

User avatar
8-bit
Posts: 3406
Joined: Wed 04 Apr 2007, 03:37
Location: Oregon

#2 Post by 8-bit »

The file transfers are buffered.
When you do a copy, buffering of one file using available memory as you say takes a certain amount of time.
The buffering is done once at the beginning of the copy.
With more than one file, the buffering is not done again for additional files on a multiple file copy.
At least that is the way I understand it.
Also, when I copy a file, I also open a terminal and type "sync".
When the prompt "#" appears again, I know the file has been copied completely.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#3 Post by amigo »

First of all, using ROX to copy/move files is *way* slower than the plain CLI commands -mostly because of having to update the progress bar and display the file and folder names. You can set ROX to ignore the file names and only print the dir names, which will speed things up a bit.

But, even from the CLI, multiple operations at once will slow things down a lot. The USB bus is really slow compared to IDE/SATA interfaces. And since each segment of the copy/move has to be confirmed which involves two-way communictaion over the USB bus. Doing multiple ops at once means these to/from signals have to be scheduled. Copying is faster than moving. So, even when I really want to move the items, I simply copy them and then remove the originals when done.

npierce
Posts: 858
Joined: Tue 29 Dec 2009, 01:40

#4 Post by npierce »

firak wrote:How can i do it, in one command, if files are form different folders?
Just use semicolons.

Code: Select all

cp /tmp/alpha/file1 /tmp/xray/ ; cp /tmp/bravo/file2 /tmp/yankee/ ; cp /tmp/charlie/file3 /tmp/zulu/
(Note: if your browser window is narrow, that may look like multiple lines, but it is only one.)

firak
Posts: 175
Joined: Mon 14 Apr 2008, 14:07

#5 Post by firak »

Yes!
My problem is confirmed by you ( amigo, 8bit , nepierce).
I test by manual command and is faster. I test it!
Im not used to the command console,
but is ok.
I open console in a source folder
cp sourecefile(tabcanhelp) space /mnt/sdc1 ; sourecefile2(tabcanhelp) space /mnt/sdc1 ; sourecefile3(tabcanhelp) space /mnt/sdc1

for copying entere folders:

Code: Select all

cp -r sourecefolder(tabcanhelp) space /mnt/sdc1 
So is better to use console for big transferring.Thaks

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#6 Post by Flash »

I used

Code: Select all

cp -r -v

to copy a directory containing about 180 files totaling over 250 MB to a mp3 player (from a USB flash drive.) The transfer went very fast compared to ROX, and all the files seemed to get copied, but not in their numerical order. Using the cp command, the files were copied beginning with file number 3. The rest of the files were copied in numerical order, then files number 1 and 2 were added at the end.

The reason this is bad for me is that every mp3 player I've used plays the files in a directory in the order they were copied to the mp3 player, not in the numerical order of the filenames.

I developed a workaround to force ROX to copy the files in a directory in the numerical order of the files, but it requires several steps and ROX is slow compared to cp. I've been looking for a better solution for years. :(

User avatar
Monsie
Posts: 631
Joined: Thu 01 Dec 2011, 07:37
Location: Kamloops BC Canada

Thumb drive slow if multiple file transfer?

#7 Post by Monsie »

Flash wrote:I used

Code: Select all

cp -r -v

to copy a directory containing about 180 files totaling over 250 MB to a mp3 player (from a USB flash drive.) The transfer went very fast compared to ROX, and all the files seemed to get copied, but not in their numerical order. Using the cp command, the files were copied beginning with file number 3. The rest of the files were copied in numerical order, then files number 1 and 2 were added at the end.

The reason this is bad for me is that every mp3 player I've used plays the files in a directory in the order they were copied to the mp3 player, not in the numerical order of the filenames.

I developed a workaround to force ROX to copy the files in a directory in the numerical order of the files, but it requires several steps and ROX is slow compared to cp. I've been looking for a better solution for years. :(
Flash, have a look at this post here at LinuxQuestions because I think it relates to your problem specifically and offers more than one possible solution.

Monsie
My [u]username[/u] is pronounced: "mun-see". Derived from my surname, it was my nickname throughout high school.

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#8 Post by Flash »

Thanks, it does look like this post could have the solution I'm looking for.
Ditto for this post about rsync, which Puppy also has.
So I tried

Code: Select all

rsync -av Path/to/source/directory Path/to/destination/directory
If the -v result can be believed, it did exactly what I want, and very fast compared to ROX. The proof will be when I listen to the book.

I never knew you could string the options together (-av); I thought you had to keep them separate (-a -v). So I learned two new things tonight. :D

Post Reply