Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Fri 19 Dec 2014, 16:19
All times are UTC - 4
 Forum index » House Training » Users ( For the regulars )
TestDisk: Flash Drive="CHS and LBA don't match" [SOLVED]
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 11 of 11 [156 Posts]   Goto page: Previous 1, 2, 3, ..., 9, 10, 11
Author Message
amigo

Joined: 02 Apr 2007
Posts: 2291

PostPosted: Fri 22 Mar 2013, 13:40    Post subject:  

hdparm doesn't work on flash drives.
Back to top
View user's profile Send private message 
Karl Godt


Joined: 20 Jun 2010
Posts: 3982
Location: Kiel,Germany

PostPosted: Fri 22 Mar 2013, 14:14    Post subject:  

hdparm does not run good to aquire information from usb flash drives.

On newer sda* kernels it should be invoked like
Code:
hdparm -iI /dev/sd*


Note the BIG EGG "I" insted the little one. The little one is "just in case" .
Back to top
View user's profile Send private message Visit poster's website 
Sylvander

Joined: 15 Dec 2008
Posts: 3551
Location: West Lothian, Scotland, UK

PostPosted: Fri 22 Mar 2013, 15:25    Post subject:  

Code:
# hdparm -iI /dev/sdb

/dev/sdb:
hdparm: ioctl 0x304 failed: Invalid argument
hdparm: HDIO_GET_IDENTITY: Invalid argument
hdparm: HDIO_DRIVE_CMD: Invalid argument
#
Back to top
View user's profile Send private message 
npierce

Joined: 28 Dec 2009
Posts: 858

PostPosted: Fri 22 Mar 2013, 16:32    Post subject:  

Sylvander wrote:
Any idea what is different about this "Recovery Tool" that makes it able to do what all those other things failed to do?

Short answer - - -

This utility knew how to talk to the specific controller chip that is in your flash drive. And it knew what registers on the chip it needed to write to and what it had to write to them in order to properly do a low-level format. In short, it spoke the controller's language.

There are a lot of different models of flash drives in this world, and a lot of different controller chips for flash drives. There are a bunch of utilities that can do a low-level format on a flash drive, some designed for specific controller chips, some designed to support many controller chips, but apparently none that support every controller chip ever invented.

So the key was to find a utility that actually knew about the controller chip in your flash drive. (That's where we needed the 0a16:2004 (the vendor code and product code that you found with the lsusb command), and the database at flashboot.ru that listed the controller chips used in drives with the 0a16 vendor code and the utilities successfully used by others to talk to those chips.)

If your flash drive had been healthy, there would not have been the need to use a utility that knew how to do a low-level format with your specific controller chip. For a healthy drive, the Linux utilities you tried, such as mkdosfs and mke2fs would normally have worked, but they just do a filesystem format and you needed a low-level format because your drive had problems.


Long answer - - -

Along with the area used for filesystems by the operating system (be it Linux, Windows, or whatever), there is other data contained in your drive such as information about the drive itself and tables used to map logical disk sectors to physical blocks of flash memory.

The Linux kernel, like UNIX, allows programs to look at devices as if they are files. Of course, a piece of hardware is not just a file. For instance, accessing a hard drive requires heads to be positioned over the desired data, the correct head to be used to read the desired track, and formatting information to be interpreted to determine the desired sector. Or, for a flash drive, the flash data block that corresponds to the desired sector needs to be looked up. But the kernel driver and the firmware in the drive itself take care of those details, so that a program can just directly access any byte in the area of the drive used for filesystems. This doesn't include the area with the other data mentioned above.

Of course, most programs don't want to look at disk drives as files, they want to look at actual files on a mounted filesystem, and so most programs would rather talk to a partition's mount point (e.g., /mnt/sdb1) rather than its device node (e.g., /dev/sdb1) or the entire drive's device node (e.g., /dev/sdb). But some utility programs, such as mkdosfs do want to talk to the partition's device node. And other utilities, such as fdisk want talk to the device node for the entire drive.

When you format a partition in a flash drive with mkdosfs, you are initializing certain sectors of the partition with information used by the file system, such as the first sector (a.k.a. boot sector) (that contains a description of the filesystem (bytes per sector, sectors per cluster, location and number of allocation tables, etc.), the allocation tables themselves, and the partition's root directory (i.e, the top directory, not to be confused with the directory named "root").

Similar information is initialized when you use mke2fs, although of course the exact format is different for Linux filesystems.

Anyway, this filesystem format only writes to the partition, which is, obviously, part of the area of your drive that is used for filesystems. It doesn't write to the other areas containing information about the drive itself, or the tables used to map logical sectors to physical blocks. And so it is not a low-level format.

Usually, if the drive is healthy, this is enough, and a low-level format is not necessary. Your drive was not healthy.


If you are unfamiliar with the term "low-level format", I'll try to explain.

Exactly what happens during a low-level format is different for a flash drive than it was for magnetic media, like hard drives and floppy drives. But in both cases it involves writing data to the drive that is necessary for the drive to properly store and retrieve your data. This low-level data is not something that the operating system normally needs to concern itself with. The operating system is normally only interested in the data used for its filesystem format, the data in the directories, and the data in the files. On modern drives, the low-level data is normally only seen by the drive itself, not by the operating system. But without the low-level data being correct, the filesystem format data, the directory data, and the file data will not be stored correctly.

When talking of traditional hard drives or floppy drives, based on magnetic media, low-level formatting is what writes the addressing information to the unformatted magnetic particles on the surface of the disk, because, as you probably know, the disk contains not only the data that you write to it, but also many bits of data needed to identify the sectors so that your data can be found again after you write it.

Modern operating systems usually don't need to know the details of low-level formatting. They just need to know how to ask a disk drive to give it the data in sector 12345 (or whatever), or to say here is a sector's worth of data, please write it to sector 54321 (or whatever).

Hard drives have been given the low-level format at the factory. Floppy disks also come preformatted (assuming that such things are still manufactured Smile ), but in the early days they did not, so operating systems have the ability to do a low-level format on them. But my guess is that the operating system doesn't need to know the details of low-level formatting even to do that. I suspect they just talk to the PC's BIOS or the firmware in the floppy drive and say, "please do a low-level format on this floppy."

(One exception was the first disk operating system for a personal computer (although, as you may remember, they were usually known as "home computers" or "micro computers" at the time) which did need to know about low-level formatting. If Steve Wozniak could figure a way to reduce the number of chips in Apple II hardware by even a single chip, he would do so. So rather than designing a complex controller for the floppy disk drive, he decided to just create a very simple controller, and do the complex stuff in software, in the operating system itself. So instead of just telling the drive to format the disk, Apple DOS had to tell the disk to start spinning and to move the read/write head to the first track, and then it sent a stream of data bits (containing the formatting data) directly to the head. Then it would command the stepping motor to move the head to the next track and write another stream of bits, and so on, and so forth.)

(I don't know if the original IBM PC DOS had to know any of the details of low-level formatting, but I suspect not. That knowledge was probably contained in the BIOS or the firmware for the floppy drive.)

But you don't want to know about magnetic media, you want to know about flash drives.

Since the main component of a flash drive is a big memory chip, one might think there would be no need for a low-level format. Shouldn't it be possible to directly address the chip? There shouldn't be any need to write addressing bits to the sectors in order to find them again. True. But things aren't that simple. Low-level formatting is very different for a flash drive than for a spinning magnetic disk. Although writing address bits is not necessary, other stuff is.

For instance, the type of flash memory used in most flash drives ("NAND flash") is written in blocks larger than the typical 512-byte sector size of a PC disk drive. And flash drives do "wear leveling" to prevent blocks from wearing out quickly because of being written much more often than others -- which means that when a sector is re-written, it is probably not going to be physically located in the same place in the flash chip as it was before.

For those reasons (and probably others), the logical sectors must be mapped to the physical blocks on the flash chip (i.e, sector 12345 (or whatever) is currently in block 9876 (or whatever)). It would be my guess that low-level formatting does stuff like set up the tables used for this mapping.

Also, there is configuration data contained in the flash drive, such as its size. Some of that probably gets rewritten as well. (This utility should automatically write the correct values there, but there are people in this world who use other utilities to write false values for the drive size. If you see an unbelievably good price on ebay for flash drives, chances are good that they have been counterfeited by doing just that. The drive may be advertised as, say, 32 GB, but contain maybe 8GB. You can plug it into your PC and it will claim to have 32 GB, and will work fine at first. But, needless to say, when a block of data beyond its real memory is written the result won't be pretty.)

Another important part of low-level formatting is probably the detection of bad blocks, and ensuring that the tables don't allow the bad blocks to be used. (It is not unusual for even a new flash drive to have some bad blocks -- but if the low-level format is good, you will never see them. It is more economical for a chip manufacturer to cram as many memory cells as possible onto a chip, knowing that some will likely be bad when manufactured, then it is to reduce the density enough to get perfect chips with all good cells, but fewer of them.)


Anyway, Sylvander, I've probably told you more here than you wanted. So I'll not ramble on further for now. Take all the above with a grain of salt, because I am certainly no expert on flash drives. I see the words "probably" and "guess" sprinkled liberally through my above text. Let those words remind you not to treat all the above as fact, since it's just the view from someone who doesn't yet have a clear understanding of the subject. I wish I could be more definite about all this, but I've only just begun learning about flash drives -- mostly what I've learned has been a result of this thread.
Back to top
View user's profile Send private message 
Sylvander

Joined: 15 Dec 2008
Posts: 3551
Location: West Lothian, Scotland, UK

PostPosted: Sat 23 Mar 2013, 08:00    Post subject:  

WOW! Thanks for the explanation npierce. Very Happy

I've read it, and some of it is beyond my level of understanding, but I get the general idea, and will re-read again and again, to try to understand.

1. Understood all of the short answer.
Understand:
Low-level-format.

2. Don't understand:
Mount point; device node.
Back to top
View user's profile Send private message 
npierce

Joined: 28 Dec 2009
Posts: 858

PostPosted: Sat 23 Mar 2013, 11:05    Post subject:  

Hi Sylvander,

You're welcome.

Sylvander wrote:
2. Don't understand:
Mount point; device node.

As you've probably noticed, Puppy (or any Linux) likes to organize everything into a tree of files -- even things that are not really files. Hardware devices like your flash drive and the individual partitions on your flash drive can be accessed via the /dev/ directory. By now you are well aware that, in your case, Puppy detects your flash drive at /dev/sdb and its first partition at /dev/sdb1. Each of those is known as a "device node" since they are not really files.

But even though they are not actual files, they can be read and written to as if they were actual files. For instance, when reading the partition table fdisk opens /dev/sdb just as if it were a file, then reads 64 bytes beginning with the byte which is 446 bytes from the beginning of the "file". Similarly, mkdosfs opens /dev/sdb1 just as if it were a file, and writes formatting information to the appropriate locations in partition 1.

The fdisk and mkdosfs utilities need to access places on the drive that most applications never need to go, and which do not appear as entries in the tree of directories and files on the flash drive partition. So it is appropriate for them to use /dev/sdb and /dev/sdb1.

But most applications are interested in accessing files. Applications certainly could access a file through /dev/sdb1 or even /dev/sdb, but it would involve a lot of work for them to do so. The application would have to know a lot about the format of the filesystem used (FAT32, ext2, or whatever) including its directory structure, and do a lot of math to track down all of the bytes in the desired file.

To avoid all of that math, we mount drives and let the kernel do the math. Then applications can just open a file and read or write any byte in the file without knowing anything about where it physically lives on a disk.

And to mount a file, you need a mount point.

Say I have a flash drive with two directories in the root directory of partition 1. (Again I am talking about the top directory of partition one, not a directory named "root". That can be confusing.) The two directories are named "Music" and "Photos". So the directory tree looks like this:
Code:
/
|-- Music
`-- Photos

Then say that the tree of my Puppy directories looks something like this (with all the sub-directories omitted for clarity):
Code:
/
|-- archive
|-- bin
|-- dev
|-- etc
|-- lib
|-- mnt
|-- opt
|-- proc
|-- root
|-- sbin
|-- sys
|-- usr
`-- var

Then I plug the drive in. Puppy detects it at /dev/sdb1, creates a directory at /mnt/sdb1, and mounts the drive on that directory. Now the tree of my Puppy directories looks like this:
Code:
/
|-- archive
|-- bin
|-- dev
|-- etc
|-- initrd
|-- lib
|-- mnt
|   `-- sdb1
|       |-- Music
|       `-- Photos
|-- opt
|-- proc
|-- root
|-- sbin
|-- sys
|-- usr
`-- var

The /mnt/sdb1 directory is called a "mount point" because it is the place in the tree that partition 1 of the flash drive was mounted at.


In summary: The device node allows the raw data on the disk drive or partition to be read as one gigantic file. The mount point allows the files on the drive to be accessed through the filesystem's tree of directories and files.


I hope this makes things more clear, not less. Smile
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 11 of 11 [156 Posts]   Goto page: Previous 1, 2, 3, ..., 9, 10, 11
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » Users ( For the regulars )
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0965s ][ Queries: 12 (0.0057s) ][ GZIP on ]