USB Flash Drive Quality

[scsijon]

I've a number of mixed brands but the best i've found are my TDK 64Gig Blue's, Toshiba Black's (19mbps read/4mbps write) for the largest files (without the need for rewriting multiple blocks as they all can, and do seem to loose their way transfering data) and the old Sandisk Cruser Edge series. My workhorses are Lexar 16gig dark grey Slides (most without case as it makes them capable of being used anywhere) which have been going since 2014.

Latest aquisition are a couple of sandisk usb adapters for microSD cards, great idea by sandisk adding them to their microDS to SD adapter range.

My oldest are some Sony 512Meg usb2.0 and even though they are 'chunky' they've not let me down and worked even in old usb 1.0 ports that have been classed as unuseable. It usually just meant the port hardware is the origonal 1.0 specification which the latest usb2/3 devices are too fast in their i/o buffering to handle.

I do have some Emtec, and if you read the specs you will see that they are for photo storage in cameras and smartphones/tablets and not 'constant data movement'. Also Emtec have a number of sub-types which are usually defined by their body color as far as write speed is concerned and gold is not the best write speed, some are ok and some not for what we use them for.

You also need to look at what Class they are.
for example with usb3 ...
Class 4 is:
Read speed Up to 30MB/s
Write speed Up to 6MB/s
Class is:
Read speed Up to 45 MB/s
Write speed Up to 14 MB/s
all the way up too..
Class 10 is:
Read speed Up to 95 MB/s
Write speed Up to 90 MB/s

I do note that most of the sites now state something like (and I quote from transMemory's) :
* The terms ‘Super Speed USB 3.0’ and 'Hi-Speed USB 2.0' used herein are the name of a specification upon which this product is based, it does not guarantee the speed of its operation.
** e.g. Read and write speeds may vary depending on the read and write conditions, such as devices you use and file sizes you read and/or write.

Oh, and for those with dead usb units you can try recharging the internal capacitor/solid-state battery by leaving them plugged in to a powered up not just turned on (I don't have to say why do I) old workstation (not a notebook or too modern a pc) that's doing nothing for a day or two and see what happens. They will usually come up enough to get your data off if they don't come up to be usefull again. The old 1.0 specification did say they should be plugged in at least once every 90 days for an hour to 're-stabalise their internals'.

Oh and just something for your minds:
There are usb3.1 256gig models with up to class 12 speeds (140/140) due out there for public release this year. Will we need SSD's soon?

anyway have fun

[BarryK]

I ran the test with a 32gb usb-3.0 Kingston Data Traveler 3.0 flash drive which
is plugged into a usb-2.0 port.

# sync
# #echo 3 > /proc/sys/vm/drop_caches
# dd if=/dev/zero of=/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 54.245 s, 19.8 MB/s
#

Don't forget to put the path to where the partition is mounted, for example of drive sdb and partition sdb2 mounted at /mnt/sdb2:

Code: Select all

#  sync
# echo 3 > /proc/sys/vm/drop_caches
#  dd if=/dev/zero of=/mnt/sdb2/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct

Oh, and don't forget to delete 'dummyfile' afterward, if you don't want a 1GB file left on the usb stick!

[Billtoo]

I ran the test with a 32gb usb-3.0 Kingston Data Traveler 3.0 flash drive which
is plugged into a usb-2.0 port.

# sync
# #echo 3 > /proc/sys/vm/drop_caches
# dd if=/dev/zero of=/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 54.245 s, 19.8 MB/s
#

Don't forget to put the path to where the partition is mounted, for example of drive sdb and partition sdb2 mounted at /mnt/sdb2:

Code: Select all

#  sync
# echo 3 > /proc/sys/vm/drop_caches
#  dd if=/dev/zero of=/mnt/sdb2/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct

Oh, and don't forget to delete 'dummyfile' afterward, if you don't want a 1GB file left on the usb stick!

There was a 1gb dummyfile on the drive after the test that I did which I have deleted.
I can't get the following to work:

Code: Select all

#  sync
# echo 3 > /proc/sys/vm/drop_caches
#  dd if=/dev/zero of=/mnt/sdb2/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct

Gets an error and quits.

# sync
# echo 3 > /proc/sys/vm/drop_caches
# dd if=/dev/zero of=/mnt/sdb2/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
dd: failed to open '/mnt/sdb2/dummyfile': No such file or directory
#

[tallboy]

Barry, will the FS on the drive have any influence on the result? You have played around with these flash drives for some years now, do you prefer to format the drives to another FS, like ext-something, and in that case, which?

tallboy

Oh, and /proc/sys/vm/drop_caches, should it be set back to 0?

[tallboy]

Hmmm...
The stick is the SanDisk Ultra 16Gb that I dislike because it is too wide to mount side-by-side with anything.

# dd if=/dev/zero of=/mnt/sdb1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 117.339 s, 9.2 MB/s

CPU is a 2.6GHz P4, USB2.0, not very impressive....

I take that back, my results are extremely good compared to these!:

Emtec: 1.6MB/s
Lexar: 5.3MB/s

tallboy

[tallboy]

Interesting, because I don't know why it is so slow! The SanDisk Ultra 16Gb in the former test had never been used. Can the FS mean anything, or is it defragmentation?

SanDisk Cruzer Edge 32 Gb with 15 Gb free space, format ext2:

Code: Select all

# cat /proc/sys/vm/drop_caches
3
# sync
# dd if=/dev/zero of=/mnt/sdb1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 327.375 s, 3.3 MB/s
# echo 0 > /proc/sys/vm/drop_caches
# rm /mnt/sdb1/dummyfile 

Ok. let's do a clean-up:

Code: Select all

# fsck.ext2 /dev/sdb1
e2fsck 1.41.14 (22-Dec-2010)
/dev/sdb1 has gone 196 days without being checked, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 3A: Optimizing directories
Pass 4: Checking reference counts
Pass 5: Checking group summary information

/dev/sdb1: ***** FILE SYSTEM WAS MODIFIED *****
/dev/sdb1: 56233/1908736 files (0.3% non-contiguous), 3432721/7631616 blocks

And try again:

Code: Select all

# echo 3 > /proc/sys/vm/drop_caches 
# sync
# dd if=/dev/zero of=/mnt/sdb1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 331.832 s, 3.2 MB/s
# echo 0 > /proc/sys/vm/drop_caches
# rm /mnt/sdb1/dummyfile 

It actually slowed down??

tallboy

[smokey01]

I thought I would try a few tests as well. Below are my results.

SanDisk Cruzer Blade 4GB USB2 in USB2 Port vfat
dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct 1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 279.549 s, 3.8 MB/s

Verbatim 4GB USB2 in a USB2 port ext4
dd if=/dev/zero of=/mnt/sdd2/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 112.667 s, 9.5 MB/s

Verbatim 4GB USB2 in a USB2 port vfat
dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 113.166 s, 9.5 MB/s


Lexar 8GB USB2 in USB2 Port vfat
# dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 171.287 s, 6.3 MB/s

Lexar 8GB USB2 in USB3 Port vfat
dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 164.466 s, 6.5 MB/s

SanDisk 16 GB Ultra USB3 in a USB2 Port vfat
dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 90.3283 s, 11.9 MB/s

SanDisk 16 GB Ultra USB3 in a USB3 Port vfat
dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 83.074 s, 12.9 MB/s

SanDisk 16 GB Ultra USB3 in a USB3 Port f2fs
dd if=/dev/zero of=/mnt/sdd2/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 68.8602 s, 15.6 MB/s

[BarryK]

I ran the test with a 32gb usb-3.0 Kingston Data Traveler 3.0 flash drive which
is plugged into a usb-2.0 port.

# sync
# #echo 3 > /proc/sys/vm/drop_caches
# dd if=/dev/zero of=/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 54.245 s, 19.8 MB/s
#

Don't forget to put the path to where the partition is mounted, for example of drive sdb and partition sdb2 mounted at /mnt/sdb2:

Code: Select all

#  sync
# echo 3 > /proc/sys/vm/drop_caches
#  dd if=/dev/zero of=/mnt/sdb2/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct

Oh, and don't forget to delete 'dummyfile' afterward, if you don't want a 1GB file left on the usb stick!

There was a 1gb dummyfile on the drive after the test that I did which I have deleted.
I can't get the following to work:

Code: Select all

#  sync
# echo 3 > /proc/sys/vm/drop_caches
#  dd if=/dev/zero of=/mnt/sdb2/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct

Gets an error and quits.

# sync
# echo 3 > /proc/sys/vm/drop_caches
# dd if=/dev/zero of=/mnt/sdb2/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
dd: failed to open '/mnt/sdb2/dummyfile': No such file or directory
#

Click on the flash drive partition and it will be mounted. "/mnt/sdb2" was just an example, you have to put the path to where your partition is mounted.

Ah, it has just occurred to me, I have misunderstood. If you booted up on the Kingston flash drive, full install of Quirky, then, yes, "of=/dummyfile" is correct.

My comment about mount path is if you plug in a flash stick while already running Linux.
Sorry, your original test is ok.

[BarryK]

Barry, will the FS on the drive have any influence on the result? You have played around with these flash drives for some years now, do you prefer to format the drives to another FS, like ext-something, and in that case, which?

tallboy

Oh, and /proc/sys/vm/drop_caches, should it be set back to 0?

My testing is with ext4 no-journal.

I don't know about other filesystems, but I would expect fuse-based driver such as ntfs-3g to be slower.

That last question, I don't know. It was something that I read about, but there was no advice to change it back afterward.

[Billtoo]

Ah, it has just occurred to me, I have misunderstood. If you booted up on the Kingston flash drive, full install of Quirky, then, yes, "of=/dummyfile" is correct.

My comment about mount path is if you plug in a flash stick while already running Linux.
Sorry, your original test is ok.

I ran the speed test in my Quirky 8.3 with a Kingston Datatraveler 3.0 plugged into a usb-3.0 port:

# sync
# #echo 3 > /proc/sys/vm/drop_caches
# dd if=/dev/zero of=/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.504 s, 79.5 MB/s
#

That's about 4 times faster than when plugged into a usb-2.0 port.

[Mike Walsh]

The fact that the newer Ultra USB 3.0 versions exist in a 128GB size is simply mind blowing! If only I had a 3.0-ported computer...

^^^

@Bob:-

I believe they're due out in a 256 GB capacity later this year. In the same form factor size, no less. And although mine run in USB2.0 ports, the read/write access times are plenty fast enough for me; half the time, I reckon the buffers are idling while the P4 catches up..!!

A pair of those would up me to 576 GB total. On an elderly, 15-yr old Inspiron, natch.

Do I really need that much capacity? I doubt it. (Nice to dream, through! )

Back on track, although I have a USB 3.0 adapter card in the PCIex16 slot of my 'big' Compaq, I can't boot the Ultra Fits on there. The controller chip on the card doesn't have the appropriate protocols 'burnt' into its ROM to permit that, apparently...

Such is life.


Mike.

[wdt]

For writing a jpg to a camera memory dd is an appropriate test,,
but for an OS, something like
iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
is much more appropriate. You will find 90+% of flash mem is shit for small file writes.
(anything over 2M/s for 4K writes is good)
For many more grisley tales see the armbian board (&tkaiser), but these are uSD
with the 25M/s port restriction
Maybe puppy doesn't have iozone?
some concrete examples
Samsung EVO+, uSD,, 64GB, all usb3

Code: Select all

                                                       random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread

          102400       4    66325    67839   167321   156568    25222     4822                                                          
          102400      16    66187    70289   495456   490783    93722    20577                                                          
          102400     512    70355    70930   912412  1113308   858670    66369                                                          
          102400    1024    70784    71019  1092766   888511   931722    68786                                                          
          102400   16384    70100    70713  1095174  1015284   959223    71150 

and Sandisk extreme usb3 (32GB stick),, usb3

Code: Select all

 kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    25307    26383    26410    25825    10499     8750                                                          
          102400      16    76227    89201    80656    81546    33139    19260                                                          
          102400     512   105397   110260   200216   196873   131325    19683                                                          
          102400    1024    76372   110908   199469   201110   162778    29011                                                          
          102400   16384    50357   109699   206370   210531   199944   100871                                                          

Sorry, too narrow, last 2 are random read and write

BTW,,These 2 are about the fastest you will find, notice the speed of 4K and 16K (speed is kB/s)

[smokey01]

Ah, it has just occurred to me, I have misunderstood. If you booted up on the Kingston flash drive, full install of Quirky, then, yes, "of=/dummyfile" is correct.

My comment about mount path is if you plug in a flash stick while already running Linux.
Sorry, your original test is ok.

I ran the speed test in my Quirky 8.3 with a Kingston Datatraveler 3.0 plugged into a usb-3.0 port:

# sync
# #echo 3 > /proc/sys/vm/drop_caches
# dd if=/dev/zero of=/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.504 s, 79.5 MB/s
#

That's about 4 times faster than when plugged into a usb-2.0 port.

Wow, 79.5 MB/s. I thought that was pretty fast so I did a test on another USB flash drive I have. SanDisk Extreme 3.0 64 GB vfat.

dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 11.2764 s, 95.2 MB/s
#

It is suppose to be fast and I guess it is at 95.2 MB/s.

I thought the SanDisk Ultra was fast, not so much.

[tallboy]

Oh, and /proc/sys/vm/drop_caches, should it be set back to 0?

That last question, I don't know. It was something that I read about, but there was no advice to change it back afterward.

I looked in the drop_caches file (in Lucid 5.2.8.7) before setting it to 3, and it was default set to 0.

From http://man7.org/linux/man-pages/man5/proc.5.html.

/proc/sys/vm/drop_caches (since Linux 2.6.16)
Writing to this file causes the kernel to drop clean
caches, dentries, and inodes from memory, causing that
memory to become free. This can be useful for memory
management testing and performing reproducible
filesystem benchmarks. Because writing to this file
causes the benefits of caching to be lost, it can
degrade overall system performance.

To free pagecache, use:

echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes, use:

echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes, use:

echo 3 > /proc/sys/vm/drop_caches

Because writing to this file is a nondestructive
operation and dirty objects are not freeable, the user
should run sync(1) first.

I see from my code that I incorrectly modified the drop_cashes setting before I ran sync, but a re-test showed no change in speed. I believe drop_caches should be set to 0 (zero) again after testing flash memory.

tallboy

[smokey01]

This got me thinking how flash drives compare to SSD and HDD.

ATA WDC WDS240G1G0A-
dd if=/dev/zero of=/mnt/sdc1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 3.81126 s, 282 MB/s

ATA WDC WD3001FAEX-0
dd if=/dev/zero of=/mnt/sdb3/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 9.65869 s, 111 MB/s

ATA TOSHIBA DT01ACA3
dd if=/dev/zero of=/aufs/devsave/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 5.81195 s, 185 MB/s

I guess there was no surprise to see the SSD out perform the two HDD. What was surprising how much slower the WD was to the Toshiba. Both are 3 TB.

[Moat]

And although mine run in USB2.0 ports, the read/write access times are plenty fast enough for me...

Ya' know, I'm just gonna have to cave in and go ahead and hunt one of these down, and see (enjoy?) any of the potential speed advantages some of you guys are mentioning (i.e. - running a 3.0 stick from a 2.0 port/board)... and currently Best Buy is advertising 128 GB Ultra Fits for ~$33!! Almost silly not to grab one (or more... ?! ) at that price. Really seems a fantastic deal for that amount of storage capacity - let alone it's tiny size advantage. Cheaper per byte than CD-R's!

Bob

[BarryK]

I believe drop_caches should be set to 0 (zero) again after testing flash memory.

Thanks for the info!

Here are more tests:

http://barryk.org/news/?viewDetailed=00631

The question that was asked whether type of filesystem makes a difference, it does!

[BarryK]

Ya' know, I'm just gonna have to cave in and go ahead and hunt one of these down, and see (enjoy?) any of the potential speed advantages some of you guys are mentioning (i.e. - running a 3.0 stick from a 2.0 port/board)... and currently Best Buy is advertising 128 GB Ultra Fits for ~$33!! Almost silly not to grab one (or more... ?! ) at that price. Really seems a fantastic deal for that amount of storage capacity - let alone it's tiny size advantage.

Bob

Ultra-fits, those are the Sandisk tiny things, aren't they? I know some people like them, but be careful. They run very hot, and reports are they damage the usb socket.

I have one, 32GB, purchased several months ago. But, I was alarmed how hot it got, also had trouble with plugging it in, the pins didn't align right or something -- had to plug it in more than once to get Linux to recognise it.

I put it into my "old flash drives" jar and haven't used it since. And won't, will probably bin it.

[Moat]

Ultra-fits, those are the Sandisk tiny things, aren't they? I know some people like them, but be careful. They run very hot, and reports are they damage the usb socket.

Aah, interesting - and yes, the tiny ones. I think earlier Mike Walsh also mentioned their running hot... my guess is they'd likely be OK running at the limited read/write USB 2.0 buss speeds of my older laptops. Their tiny predecessor USB 2.0 Cruzer Fit versions I've been running Puppies on for years, and have experienced no particular heat or fit (ha!) issues with those, as of yet (knock on wood...).

Thanks for that heads-up, Barry!

Bob

[smokey01]

Those little fella's do get a bit warm. Not as quick as some but not bad.

SanDisk SDCZ43 USB3 64GB vfat. I think they call them Nano.
dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct 1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 21.116 s, 50.8 MB/s