Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sun 17 Dec 2017, 23:21
All times are UTC - 4
 Forum index » Advanced Topics » Hardware
USB Flash Drive Quality
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 4 of 6 [82 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6 Next
Author Message
tallboy


Joined: 21 Sep 2010
Posts: 821
Location: Oslo, Norway

PostPosted: Sat 29 Jul 2017, 00:18    Post subject:  

Interesting, because I don't know why it is so slow! The SanDisk Ultra 16Gb in the former test had never been used. Can the FS mean anything, or is it defragmentation?

SanDisk Cruzer Edge 32 Gb with 15 Gb free space, format ext2:
Code:
# cat /proc/sys/vm/drop_caches
3
# sync
# dd if=/dev/zero of=/mnt/sdb1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 327.375 s, 3.3 MB/s
# echo 0 > /proc/sys/vm/drop_caches
# rm /mnt/sdb1/dummyfile

Ok. let's do a clean-up:
Code:
# fsck.ext2 /dev/sdb1
e2fsck 1.41.14 (22-Dec-2010)
/dev/sdb1 has gone 196 days without being checked, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 3A: Optimizing directories
Pass 4: Checking reference counts
Pass 5: Checking group summary information

/dev/sdb1: ***** FILE SYSTEM WAS MODIFIED *****
/dev/sdb1: 56233/1908736 files (0.3% non-contiguous), 3432721/7631616 blocks

And try again:
Code:
# echo 3 > /proc/sys/vm/drop_caches
# sync
# dd if=/dev/zero of=/mnt/sdb1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 331.832 s, 3.2 MB/s
# echo 0 > /proc/sys/vm/drop_caches
# rm /mnt/sdb1/dummyfile


It actually slowed down??

tallboy

_________________
True freedom is a live Puppy on a multisession CD/DVD.
Back to top
View user's profile Send private message 
smokey01


Joined: 30 Dec 2006
Posts: 2659
Location: South Australia

PostPosted: Sat 29 Jul 2017, 00:48    Post subject:  

I thought I would try a few tests as well. Below are my results.

SanDisk Cruzer Blade 4GB USB2 in USB2 Port vfat
dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct 1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 279.549 s, 3.8 MB/s

Verbatim 4GB USB2 in a USB2 port ext4
dd if=/dev/zero of=/mnt/sdd2/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 112.667 s, 9.5 MB/s

Verbatim 4GB USB2 in a USB2 port vfat
dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 113.166 s, 9.5 MB/s


Lexar 8GB USB2 in USB2 Port vfat
# dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 171.287 s, 6.3 MB/s

Lexar 8GB USB2 in USB3 Port vfat
dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 164.466 s, 6.5 MB/s

SanDisk 16 GB Ultra USB3 in a USB2 Port vfat
dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 90.3283 s, 11.9 MB/s

SanDisk 16 GB Ultra USB3 in a USB3 Port vfat
dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 83.074 s, 12.9 MB/s

SanDisk 16 GB Ultra USB3 in a USB3 Port f2fs
dd if=/dev/zero of=/mnt/sdd2/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 68.8602 s, 15.6 MB/s

_________________
Software <-> Distros <-> Tips <-> Newsletters
Back to top
View user's profile Send private message Visit poster's website 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 8348
Location: Perth, Western Australia

PostPosted: Sat 29 Jul 2017, 01:52    Post subject: Re: USB Flash Drive Quality  

Billtoo wrote:
BarryK wrote:
Billtoo wrote:
I ran the test with a 32gb usb-3.0 Kingston Data Traveler 3.0 flash drive which
is plugged into a usb-2.0 port.

# sync
# #echo 3 > /proc/sys/vm/drop_caches
# dd if=/dev/zero of=/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 54.245 s, 19.8 MB/s
#


Don't forget to put the path to where the partition is mounted, for example of drive sdb and partition sdb2 mounted at /mnt/sdb2:



Code:
#  sync
# echo 3 > /proc/sys/vm/drop_caches
#  dd if=/dev/zero of=/mnt/sdb2/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct


Oh, and don't forget to delete 'dummyfile' afterward, if you don't want a 1GB file left on the usb stick!


There was a 1gb dummyfile on the drive after the test that I did which I have deleted.
I can't get the following to work:

Code:
#  sync
# echo 3 > /proc/sys/vm/drop_caches
#  dd if=/dev/zero of=/mnt/sdb2/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct


Gets an error and quits.

# sync
# echo 3 > /proc/sys/vm/drop_caches
# dd if=/dev/zero of=/mnt/sdb2/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
dd: failed to open '/mnt/sdb2/dummyfile': No such file or directory
#


Click on the flash drive partition and it will be mounted. "/mnt/sdb2" was just an example, you have to put the path to where your partition is mounted.

Ah, it has just occurred to me, I have misunderstood. If you booted up on the Kingston flash drive, full install of Quirky, then, yes, "of=/dummyfile" is correct.

My comment about mount path is if you plug in a flash stick while already running Linux.
Sorry, your original test is ok.

_________________
http://barryk.org/news/
Back to top
View user's profile Send private message Visit poster's website 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 8348
Location: Perth, Western Australia

PostPosted: Sat 29 Jul 2017, 02:02    Post subject:  

tallboy wrote:
Barry, will the FS on the drive have any influence on the result? You have played around with these flash drives for some years now, do you prefer to format the drives to another FS, like ext-something, and in that case, which?

tallboy

Oh, and /proc/sys/vm/drop_caches, should it be set back to 0?


My testing is with ext4 no-journal.

I don't know about other filesystems, but I would expect fuse-based driver such as ntfs-3g to be slower.

That last question, I don't know. It was something that I read about, but there was no advice to change it back afterward.

_________________
http://barryk.org/news/
Back to top
View user's profile Send private message Visit poster's website 
Billtoo


Joined: 07 Apr 2009
Posts: 3299
Location: Ontario Canada

PostPosted: Sat 29 Jul 2017, 18:22    Post subject: Re: USB Flash Drive Quality  

BarryK wrote:


Ah, it has just occurred to me, I have misunderstood. If you booted up on the Kingston flash drive, full install of Quirky, then, yes, "of=/dummyfile" is correct.

My comment about mount path is if you plug in a flash stick while already running Linux.
Sorry, your original test is ok.


I ran the speed test in my Quirky 8.3 with a Kingston Datatraveler 3.0 plugged into a usb-3.0 port:

# sync
# #echo 3 > /proc/sys/vm/drop_caches
# dd if=/dev/zero of=/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.504 s, 79.5 MB/s
#

That's about 4 times faster than when plugged into a usb-2.0 port.
Back to top
View user's profile Send private message 
Mike Walsh


Joined: 28 Jun 2014
Posts: 3245
Location: King's Lynn, UK.

PostPosted: Sat 29 Jul 2017, 19:48    Post subject:  

Moat wrote:
The fact that the newer Ultra USB 3.0 versions exist in a 128GB size is simply mind blowing! Shocked If only I had a 3.0-ported computer...


^^^ Laughing

@Bob:-

I believe they're due out in a 256 GB capacity later this year. In the same form factor size, no less. And although mine run in USB2.0 ports, the read/write access times are plenty fast enough for me; half the time, I reckon the buffers are idling while the P4 catches up..!!

A pair of those would up me to 576 GB total. On an elderly, 15-yr old Inspiron, natch.

Do I really need that much capacity? I doubt it. (Nice to dream, through! Laughing )

Back on track, although I have a USB 3.0 adapter card in the PCIex16 slot of my 'big' Compaq, I can't boot the Ultra Fits on there. The controller chip on the card doesn't have the appropriate protocols 'burnt' into its ROM to permit that, apparently...

Such is life.


Mike. Wink

_________________
If I've helped you.....please say 'Thanks'!
MY PUPPY PACKAGES
--------------------------------------

Back to top
View user's profile Send private message Visit poster's website 
wdt

Joined: 27 Dec 2011
Posts: 37

PostPosted: Sun 30 Jul 2017, 00:20    Post subject: dd is not best measure  

For writing a jpg to a camera memory dd is an appropriate test,,
but for an OS, something like
iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
is much more appropriate. You will find 90+% of flash mem is shit for small file writes.
(anything over 2M/s for 4K writes is good)
For many more grisley tales see the armbian board (&tkaiser), but these are uSD
with the 25M/s port restriction
Maybe puppy doesn't have iozone?
some concrete examples
Samsung EVO+, uSD,, 64GB, all usb3

Code:

                                                       random    random     bkwd    record    stride                                   
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread

          102400       4    66325    67839   167321   156568    25222     4822                                                         
          102400      16    66187    70289   495456   490783    93722    20577                                                         
          102400     512    70355    70930   912412  1113308   858670    66369                                                         
          102400    1024    70784    71019  1092766   888511   931722    68786                                                         
          102400   16384    70100    70713  1095174  1015284   959223    71150


and Sandisk extreme usb3 (32GB stick),, usb3
Code:

 kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    25307    26383    26410    25825    10499     8750                                                         
          102400      16    76227    89201    80656    81546    33139    19260                                                         
          102400     512   105397   110260   200216   196873   131325    19683                                                         
          102400    1024    76372   110908   199469   201110   162778    29011                                                         
          102400   16384    50357   109699   206370   210531   199944   100871                                                         


Sorry, too narrow, last 2 are random read and write

BTW,,These 2 are about the fastest you will find, notice the speed of 4K and 16K (speed is kB/s)
Back to top
View user's profile Send private message 
smokey01


Joined: 30 Dec 2006
Posts: 2659
Location: South Australia

PostPosted: Sun 30 Jul 2017, 02:13    Post subject: Re: USB Flash Drive Quality  

Billtoo wrote:
BarryK wrote:


Ah, it has just occurred to me, I have misunderstood. If you booted up on the Kingston flash drive, full install of Quirky, then, yes, "of=/dummyfile" is correct.

My comment about mount path is if you plug in a flash stick while already running Linux.
Sorry, your original test is ok.


I ran the speed test in my Quirky 8.3 with a Kingston Datatraveler 3.0 plugged into a usb-3.0 port:

# sync
# #echo 3 > /proc/sys/vm/drop_caches
# dd if=/dev/zero of=/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.504 s, 79.5 MB/s
#

That's about 4 times faster than when plugged into a usb-2.0 port.


Wow, 79.5 MB/s. I thought that was pretty fast so I did a test on another USB flash drive I have. SanDisk Extreme 3.0 64 GB vfat.

dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 11.2764 s, 95.2 MB/s
#

It is suppose to be fast and I guess it is at 95.2 MB/s.

I thought the SanDisk Ultra was fast, not so much.

_________________
Software <-> Distros <-> Tips <-> Newsletters
Back to top
View user's profile Send private message Visit poster's website 
tallboy


Joined: 21 Sep 2010
Posts: 821
Location: Oslo, Norway

PostPosted: Sun 30 Jul 2017, 02:21    Post subject:  

I wrote:
Oh, and /proc/sys/vm/drop_caches, should it be set back to 0?

BarryK wrote:
That last question, I don't know. It was something that I read about, but there was no advice to change it back afterward.

I looked in the drop_caches file (in Lucid 5.2.8.7) before setting it to 3, and it was default set to 0.

From http://man7.org/linux/man-pages/man5/proc.5.html.
Quote:
/proc/sys/vm/drop_caches (since Linux 2.6.16)
Writing to this file causes the kernel to drop clean
caches, dentries, and inodes from memory, causing that
memory to become free. This can be useful for memory
management testing and performing reproducible
filesystem benchmarks. Because writing to this file
causes the benefits of caching to be lost, it can
degrade overall system performance.

To free pagecache, use:

echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes, use:

echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes, use:

echo 3 > /proc/sys/vm/drop_caches

Because writing to this file is a nondestructive
operation and dirty objects are not freeable, the user
should run sync(1) first.

I see from my code that I incorrectly modified the drop_cashes setting before I ran sync, but a re-test showed no change in speed. I believe drop_caches should be set to 0 (zero) again after testing flash memory.

tallboy

_________________
True freedom is a live Puppy on a multisession CD/DVD.
Back to top
View user's profile Send private message 
smokey01


Joined: 30 Dec 2006
Posts: 2659
Location: South Australia

PostPosted: Sun 30 Jul 2017, 02:35    Post subject:  

This got me thinking how flash drives compare to SSD and HDD.

ATA WDC WDS240G1G0A-
dd if=/dev/zero of=/mnt/sdc1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 3.81126 s, 282 MB/s

ATA WDC WD3001FAEX-0
dd if=/dev/zero of=/mnt/sdb3/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 9.65869 s, 111 MB/s

ATA TOSHIBA DT01ACA3
dd if=/dev/zero of=/aufs/devsave/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 5.81195 s, 185 MB/s

I guess there was no surprise to see the SSD out perform the two HDD. What was surprising how much slower the WD was to the Toshiba. Both are 3 TB.

_________________
Software <-> Distros <-> Tips <-> Newsletters
Back to top
View user's profile Send private message Visit poster's website 
Moat


Joined: 16 Jul 2013
Posts: 775
Location: Mid-mitten, USA

PostPosted: Sun 30 Jul 2017, 06:03    Post subject:  

Mike Walsh wrote:
And although mine run in USB2.0 ports, the read/write access times are plenty fast enough for me...


Ya' know, I'm just gonna have to cave in and go ahead and hunt one of these down, and see (enjoy?) any of the potential speed advantages some of you guys are mentioning (i.e. - running a 3.0 stick from a 2.0 port/board)... and currently Best Buy is advertising 128 GB Ultra Fits for ~$33!! Almost silly not to grab one (or more... ?! Laughing) at that price. Really seems a fantastic deal for that amount of storage capacity - let alone it's tiny size advantage. Cheaper per byte than CD-R's!

Bob

Last edited by Moat on Sun 30 Jul 2017, 06:18; edited 1 time in total
Back to top
View user's profile Send private message 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 8348
Location: Perth, Western Australia

PostPosted: Sun 30 Jul 2017, 06:12    Post subject:  

tallboy wrote:
I believe drop_caches should be set to 0 (zero) again after testing flash memory.


Thanks for the info!

Here are more tests:

http://barryk.org/news/?viewDetailed=00631

The question that was asked whether type of filesystem makes a difference, it does!

_________________
http://barryk.org/news/
Back to top
View user's profile Send private message Visit poster's website 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 8348
Location: Perth, Western Australia

PostPosted: Sun 30 Jul 2017, 06:21    Post subject:  

Moat wrote:
Ya' know, I'm just gonna have to cave in and go ahead and hunt one of these down, and see (enjoy?) any of the potential speed advantages some of you guys are mentioning (i.e. - running a 3.0 stick from a 2.0 port/board)... and currently Best Buy is advertising 128 GB Ultra Fits for ~$33!! Almost silly not to grab one (or more... ?! Laughing) at that price. Really seems a fantastic deal for that amount of storage capacity - let alone it's tiny size advantage.

Bob


Ultra-fits, those are the Sandisk tiny things, aren't they? I know some people like them, but be careful. They run very hot, and reports are they damage the usb socket.

I have one, 32GB, purchased several months ago. But, I was alarmed how hot it got, also had trouble with plugging it in, the pins didn't align right or something -- had to plug it in more than once to get Linux to recognise it.

I put it into my "old flash drives" jar and haven't used it since. And won't, will probably bin it.

_________________
http://barryk.org/news/
Back to top
View user's profile Send private message Visit poster's website 
Moat


Joined: 16 Jul 2013
Posts: 775
Location: Mid-mitten, USA

PostPosted: Sun 30 Jul 2017, 06:50    Post subject:  

BarryK wrote:
Ultra-fits, those are the Sandisk tiny things, aren't they? I know some people like them, but be careful. They run very hot, and reports are they damage the usb socket.


Aah, interesting - and yes, the tiny ones. I think earlier Mike Walsh also mentioned their running hot... my guess is they'd likely be OK running at the limited read/write USB 2.0 buss speeds of my older laptops. Their tiny predecessor USB 2.0 Cruzer Fit versions I've been running Puppies on for years, and have experienced no particular heat or fit (ha!) issues with those, as of yet (knock on wood...).

Thanks for that heads-up, Barry!

Bob
Back to top
View user's profile Send private message 
smokey01


Joined: 30 Dec 2006
Posts: 2659
Location: South Australia

PostPosted: Sun 30 Jul 2017, 07:05    Post subject:  

Those little fella's do get a bit warm. Not as quick as some but not bad.

SanDisk SDCZ43 USB3 64GB vfat. I think they call them Nano.
dd if=/dev/zero of=/mnt/sdd1/dummyfile bs=1M count=1024 conv=fdatasync oflag=direct 1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 21.116 s, 50.8 MB/s

_________________
Software <-> Distros <-> Tips <-> Newsletters
Back to top
View user's profile Send private message Visit poster's website 
Display posts from previous:   Sort by:   
Page 4 of 6 [82 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Hardware
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0789s ][ Queries: 14 (0.0069s) ][ GZIP on ]