What flash drive speeds do you get?

[mikeb]

urandom device/file? you copied the output from the device to a file in root.
Well I assume the urandom is a random number generator...it get reseeded at boot here...this is slax 6 by the way. Since it generates I assume cpu speed would be a factor...hence turning to reading from elsewhere.

Well creaky it may be in relative terms but notice I got a decent ~20MB/s write with a large file copied from hard drive. I have a usb2 card by the way and testing through that....for straight data transfers the cpu will be a much less significant factor.

Windows... similar speeds really... windows 2000 and XP. Are you testing on 7? There are kernel improvements in windows that happen and data throughput has been much improved since their cock ups with vista.

As mentioned you are finding it hard to determine...well..anything and its pretty clear that the whole subject seems to lack definitive (and non destructive) tests...which is a good reason to have a thread like this.

There is also a value in testing on older hardware...it shows up weaknesses and bad code much better....

mike

ps how many thanksgivings do you people have a year??

[stemsee]

SanDisk Cruzer Extreme 16gb usb 3.0

as reported by @Keisha earlier. The fastest flash drive on the planet (overall).

[LazY Puppy]

700MB of tiny files... ok that would also test the file table speed as well as raw data writing.

Since we are in Linux and Linux stands for Freedom/Liberty I assume Linux also stands for Truth!

Writing any kind of data to a storage device usually includes:

- open a file
- mark it opened inside of the file table
- write content of the file (raw data)
- close the file
- mark it closed inside of the file table

A benchmark reporting only speed of writing raw data of an already opened file would be (imo) a so-called Business-Man-Money-Maker-Story-Teller-Benchmark or as we say in German: Lug, Trug und Beschiß!

So I would suggest a three step benchmark:

- writing 1000 files at 1MB per file
- writing 100 files at 10MB per file
- writing 10 files at 100MB per file

Note: MB doesn't mean the Business-Man-Money-Maker-Story-Teller-MB at 1000KB - NO 1024KB!

The results out of this are able to tell you something like close to the truth - not just a Business-Man-Money-Maker-Story-Teller-Fable!

[Jasper]

Even Sandisk do not claim their 16GB cruzer extreme model has their own fastest write speed (see screen shot table) and another maker claims their write speed as (up to) more than twice as fast as the 190MB/s shown below - and with cheaper cost per GB of storage.

So, it is easily seen that the proposed benchmarks (see second screen shot) are meaningless. However, even if they were of real practical use - where is the published product table compiled from precisely those benchmarks? Also, if I want to compare my own tests (or my flash drive is not listed) where is the app/script to easily conduct my tests using those exact specs?

[Keisha]

The SanDisk Cruzer Extreme line was a PC Magazine Editor's Choice when I bought it two years ago.

I just tried copying the entire contents of /usr (48,860 files, according to "find -type f | wc -l") (2.05 GB, according to GDMap) in a frugal install of upup-3.9.9.1 to the SanDisk. It took 190 seconds, = 10.51 MB/second measured write speed.

Copy from the SanDisk to hard disk, took 130 seconds, so 15.77 MB/second measured read speed.

Same tests, to and from a KingstonDT 101 II (usb2) 8 GB stick also formatted ext4, plugged into a usb2 port of the same computer (Asus P8H77-I motherboard, i3770K cpu, 8 GB DDR3-1333 memory):

Copy /usr to the stick: 510 seconds, so 4.02 MB/second measured write speed.

Copy /usr from the stick to the hard disk: 260 seconds, so 7.88 MB/second measured read speed.

[gcmartin]

Hi ALL.

I would like to offer "a measurement approach" that will provide some good understanding of the behavior of storage while keeping the distro environment "constant" (as something MUST be constant for adequate understanding of device behavior within the constant arrangement).

• FIRSTLY, Let's try to agree on a testbed ISO for our USB, SSD, & HDD measurements. That testbed should be an ISO running in RAM with NOTHING other than to have used FirstRUN for local stamping. In fact, the LAN services should be turned OFF after desktop start ("ifconfig eth0 down" in a termina). The distro should not be used for anything else other than opening it for these storage measurements. EVERYONE MUST USE THE SAME DISTRO!!! (Without covering the particulars, I think most everyone understands the importance of this!) Now, assuming everyone,contributing here, understands and agrees, then lets concentrate on device data collections.
• Let's agree that all reads and write operation will be RAM to/from real storage device(s) that will be measured.
• Let's agree that an initial RAM disk will be created for testing as to get a PC's baseline.
• Let's agree that a filesystem directory will be written to from RAM to RAM disk and measured to be the baseline of the RAM distro transfer rate RAM to RAM
• Let's agree that the same filesystem directory will be written from RAM to target(s) and measurements saved to a table.
• Let's agree that the written directory wil be read from target to a new filesystem path in RAM and these measurements, too, are saved to a table.
• Let's agree that the above 2 steps could be repeated as many times as the user feels acceptable for an observation for consistency.

This will yield a single baseline, with write-read measurements for observations. These observations could be categorized by the user's device(s) on the particular user's machine.

Now, community users here should decide which PUPPY distro's ISO to use and whether it should be 32bit or 64bit as the hardware's data path on 64bit PCs are different.

If the community can agree to use the "same ISO" or find one ISO that members are willing to test with, the measurements and its steps can be accomplished or scripted and presentations will allow members here to begin to see some of the "problems" where changing PC platforms can dramatically affect storage behavior. The measurements will most certainly show, all, some interesting behavior patterns on the storage device on their systems.

Out of this will be something "consistent".

OK, if this approach is found to be generally acceptable to provide sensible measurements to this thread's members, which distro do you choose to be the distro that everyone uses?

[stemsee]

What about difference in ram speed/capacity/type or is it negligible?
What about disk speed/interface/cache/capacity/brand or is it negligible?

In reality all usb sticks need to be tested on the same setup/machine.

[gcmartin]

What about ...

I "FULLY" agree! And, this is what manufacturers do.

But, this community does NOT have a way to make that a constant. The ONLY constant we can come up with is the distro and measurement approach in such a way as to eliminate most of the extraneous factors to isolate traffic to the tested device(s).

So, to get some semblance of what the behavior would be on a user's given PC(s), this is about as good as it could get.

If this thread would agree on a given distro's ISO to be the only valid measuring tool and some predefined steps, then it is one method that we can have that would be used to measure our findings in a reasonable comparative manner for, both, observation and presentation.

This, further,would yield some potential uses beyond our measurement.

Again, it could be one PUPPY method to isolate I/O traffic in a consistent manner for observation and reporting.(because everyone uses same approach and tool) .

[Jasper]

Hi gcmartin,

You make all the choices. I will not argue with them so give it, say 24 hours and if not a single forum member disagrees - you publish full details of your specific tests.

However, you may as well let us know without delay precisely which flash drive makes and models your report will cover and also the complete specs of the test machine you will use.

Kindly also publish your script/data so that forum member benchmark comparison tests may be as compatible as possible with your ideal yardstick.

[ThoriumBlvd]

One other metric should be included... small size files

16000 x 16384 bytes
or better yet 16000 * (RND(1024 to 65535))

small sized files are the bane of $$$ tests.

[mikeb]

small sized files are the bane of $$$ tests.

well since the banter goes on I notice this gets cited often.

Thing is do I or any one else care if it takes ages to write tens of thousands of tiny files...who actually does this?

There may be a couple of dozen now and then but strangely enough its the big stuff that bothers me.... I want to pop a few vids onto a stick to take with me or a pile of music or a copy of useful software... we are 99% of the time dealing wit several megs or more and its the time to copy these that makes the difference in my real world since its the difference between waiting seconds of many minutes (remember usb 1.1) if you happen to have purchased a crap flash stick.

In puppy its similar...time to mak a save file/sfs and how long to shutdown from a usb boot after adding a bundle of pets.

Raw data writing seems the most useful statistic we could test for and generate I feel and the least affected by system variations.

Just my 5 cents there

mike

[gcmartin]

@Jasper, I'm not quite sure you understand what we are trying to achieve here. Most of the rest of us are interested in device speeds and some reasonable way of measuring and presenting this. The appeal here is about a consensus to moving with some front-ended positioning that would be agreed as a valid approach. @MikeB is echoing what we are all talking about. Hope that helps

If you would like, please open a thread and see if others will follow your lead.

Or, offer an idea of what you think WE should be talking about that would meet YOUR objective to help.

Here to help

[Jasper]

gcmartin

You repeatedly write “Here to help

[gcmartin]

@Japer.! Have you taken a look at the thread's title. Again. Let me see if I can help you a litltle, This thread is NOT about me. So, look at the title, harder this time. Then read my post over again. The light in the tower will come on ... I think.

Hope this helps.

[Jasper]

gcmartin,

This post was yours (and yours alone):

http://murga-linux.com/puppy/viewtopic. ... 030#811030

As is easily seen it was addressed to ALL and it began "I would like to offer ..."

To produce a useful table of results - exactly which Forum member did you expect to offer to find the resources (thousands of dollars and hundreds of hours)?

Exactly how have you helped one iota to achieve what YOU superficially offered?

[gcmartin]

Please @Jasper. This thread is about the author's request. Not yours nor mine.

Maybe you fail in understanding. Again, in case you didn't notice, it is not about me. Several occasions this has been repeated. Please assist, if you choose.

Here to help

[Jasper]

To whom it may concern,

In March this year I received an astonishingly illuminating PM.

I remember gcmartin has strong views on the publication of PMs.

So, I shall omit the sender's name from my upcoming screenshot.

Who will object to that?

[mikeb]

And jasper you sent one telling me to remove a posts of 2 of mine.... can we stop this muck slinging as this sort of thing totally trashes topics.

A script using standard unix tools perhaps with a series of tests.... small file...large file...many small files..... incorporating tests already mentioned here.

I was comparing dd and cp just now but was getting 1.7MB/s to the 8GB stick with either...reasons unknown... file size 300MB. Copied same to ramdisk and it took 8 seconds so no problem with the source and flash sticks don't have fragmentation problems (or do they?)

mike

[Jasper]

Hi mikeb,

I did once send you a PM with a very strong request that you withdraw what you had erroneously and maliciously written because it disrespected a valued member of this Forum.

I well remember that you responded with a PM that threatened to report me to a moderator solely on the basis you had interpreted my strong request as an order.

I no longer have your PM, but if you still have mine you are most welcome to publish it.

I let the matter drop for the reason that you frequently help on this Forum.

----------

Now to the subject of this thread and its three opening questions (as opposed to the single request gcmartin refers to). The three Rs (Readin’ ‘Riting and ‘Rithmatic) are under scrutiny - so let’s agree (or not) that gcmartin’s post that I quoted was on topic and in order so that readers may (or may not) agree that my questions to gcmartin about HIS offer are relevant and in order.

[mikeb]

I believe unwelcome and threatening PM's are against the policy of this forum and that a moderator would agree with this.

My public comments were just that... freely voicing my opinion and if I think someone is out of order then I say so...just like I am saying now. You appear to be badgering gcmartin for you own personal reasons and that's unacceptable if so. Once again YOU seem to be deciding what should and should not be posted here.

Being in the public part of the forum anyone is free to comment on any post though descending into personal attacks I believe is not forum policy either.

mike