How to do RAID0 with USB thumb drives using lvm

How to do things, solutions, recipes, tutorials
Post Reply
Message
Author
arrgk
Posts: 1
Joined: Thu 11 Sep 2008, 05:02

How to do RAID0 with USB thumb drives using lvm

#1 Post by arrgk »

I have a eee running Pupeee off an 8g SDHC. I also have a pair of 16G USB thumb drives. So we want to tie the USB drives together and make them go fast. I chose raid0. Reliability will not be an issue. I do not need raid1 or any other mirror. I chose to use the raid built in to Linux lvm, because you can mix raid0, raid1, and ordinary lvm partitions all on the same virtual drive. You can also resize them very easily. In other words, administration is easy with lvm.

first what did not work ::
dmsetup :: almost useless, --notable does not work
very difficult to master, forget it

ndadm :: not worth the effort

Here is a breakdown of what I did.

/ install gslapt ( great stuff )
/ install Slackware 12.0 lvm and devmapper
/ modprobe dm-mod
/ pvcreate /dev/sdb1
/ pvcreate /dev/sdc1
/ vgcreate Egrp00 /dev/sdb1 /dev/sdc1
/ lvcreate -L 4567m -i 2 -I 64 Egrp00 -n Eraid00
/ mkreiserfs /dev/Egrp00/Eraid00

And you are ready to rock.

Speed read test ::
dd if=/dev/sdb of=/dev/null bs=512 count=40960
gave 10.5 MBs
dd if=/dev/Egrp00/Eraid00 bs=512 count=40960
gave 19.6 MBs

93.333% of double the speed of one drive is what I count as success.

I use a journaled fs on flash !! ? yep. With normal heavy usage, this will last for years, and replacing any failing stick is easy, as is adding to the array. The BIG advantage of lvm raid is that you can add drives which have different sizes. So, for me lvm is a clear winner.

The various commands like vgs, vgscan, ... lvs, ... are at your fingertips.
Read the lvm howto, and make this turn on at boot. You get one very fat pupeee.

Thanks for the gslapt package and to the pupeee team

tester321
Posts: 10
Joined: Sat 26 Apr 2008, 11:07

Re: lvm and RAID with USB thumb drives

#2 Post by tester321 »

arrgk wrote:...
Here is a breakdown of what I did.

/ install gslapt ( great stuff )
/ install Slackware 12.0 lvm and devmapper
arrgk: Can you explain how to install the Slackware lvm onto Puppy? I have searched but haven't found it and I am not sure how to do it.

Thanks in advance for any info.

rrolsbe
Posts: 185
Joined: Wed 15 Nov 2006, 21:53

Re: lvm and RAID with USB thumb drives [SOLVED]

#3 Post by rrolsbe »

tester321 wrote:
arrgk wrote:...
Here is a breakdown of what I did.

/ install gslapt ( great stuff )
/ install Slackware 12.0 lvm and devmapper
arrgk: Can you explain how to install the Slackware lvm onto Puppy? I have searched but haven't found it and I am not sure how to do it.

Thanks in advance for any info.
Since no one replied two your request, hopefully the following will help.

My plan is to purchase four fairly fast USB flash sticks and install them into a four port Targus USB hub. Then create a poor mans SSD device for improved speed. The purpose of the remainder of this post is to verify that it SHOULD work before I purchase the USB flash devices.


I used Puppy 4.11 for the following on my ASUS EEE 904HA

Code: Select all

# uname -a
Linux puppypc 2.6.25.16 #1 Tue Aug 26 10:45:53 GMT-8 2008 i686 GNU/Linux
Get these two slackware packages from the following web site http://packages.slackware.it/

device-mapper-1.02.19-i486-2.tgz
lvm2-2.02.09-i486-1.tgz


I choose the slackware version 12.0 maybe you could use 12.1 or 12.2.
I think you use 12.0 for Puppy Linux 4.11 and
use 12.2 if you are using Puppy Linux 4.2. Comments Welcome

Convert them to pet packages with the commands below:

Code: Select all

tgz2pet device-mapper-1.02.19-i486-2.tgz
tgz2pet lvm2-2.02.09-i486-1.tgz
Install the two newly created pet packages via the file browser. Install the device-mapper... 1st then the lvm2... pet.

Load the device mapper module

Code: Select all

modprobe dm-mod
Verify it loaded

Code: Select all

# lsmod | grep mod
dm_mod                 56160  3 
Add modprobe to your /etc/rc.d/rc.local file so the device mapper is loaded at boot time.

Code: Select all

# cat /etc/rc.d/rc.local
#this file called from rc.sysinit
#you can edit this file
#When firewall is installed, will append lines to this file...
modprobe pciehp pciehp_force=1
modprobe asus_acpi
modprobe dm-mod
acpid

if [ -x /etc/rc.d/rc.firewall ]; then
  /etc/rc.d/rc.firewall start
  
# diff rc.local rc.local_orig
6d5
< modprobe dm-mod
Do the LVM stuff as below:

Code: Select all

# pwd
/etc/lvm
# diff lvm.conf lvm.conf_orig
76,78c76
<     # sysfs_scan = 1  orig
<     # Line below changed by rro so the pvcreate commands below would not fail 
<     sysfs_scan = 0
---
>     sysfs_scan = 1    


pvcreate /dev/sdc1		#8Gig Kingston Flash
pvcreate /dev/sdd1		#2Gig Kingston Flash
vgcreate flash_lvm /dev/sdc1 /dev/sdd1
lvcreate -L 2000m -i 2 -I 64 flash_lvm -n flash_raid00
I have not totally figured out exactly what the options do in the lvcreate command above, comments welcome.

Format the newly created stripped device, I choose ext2:

Code: Select all

 mke2fs /dev/flash_lvm/flash_raid00
Mount the LVM RAID0 Volume with the two USB flash devices stripped for improved speed (Hopefully anyway).

Code: Select all

mount /dev/flash_lvm/flash_raid00 /mnt/test

# mount
/dev/flash_lvm/flash_raid00 on /mnt/test type ext2 (rw,errors=continue)

# df  -kh
/dev/flash_lvm/flash_raid00
                      2.0G   29M  1.8G   2% /mnt/test


I have not rebooted or added the above mount command into my fstab yet.

I probably missed a step or two, but hopefully this will will save fellow forum members some time. PLEASE feel free to make comments in this tread.

Regards
Ron

UPDATE:

When I rebooted the BIOS reassigned my LVM devices from /dev/sdc1& /dev/sdd1 to /dev/sdb1 & /dev/sdd1. After going through the entire procedure, it worked again; however, after rebooting I got an error when trying to mount the LVM volumes. See below

I found an article on the web that stated-- if you made any changes to the /etc/lvm/lvm.conf file you needed to make a change to the initrd.gz file. Maybe this has something to do with not being able to mount the LVM devices after a reboot. Anyone have any idea what might need to be changed in the initrd.gz file? I inflated the initrd.gz file and it is a cpio archive.

This web article http://www.bigbruin.com/reviews05/thumbraid_6 indicated a speed up using four devices of 2X on reads and 6X on writes; however, He was using a different USB hub and different USB flash devices. I have read other articles where the read speeds using four devices was better that 2x. See dd commands below

The USB hub I used for these tests is a four port Targus model PA055 and the USB flash sticks were both Kingston DataTraveler one an 8G and the other a 2G.

Code: Select all

# mount /dev/flash_b_d_lvm/flash_b_d__raid00 /mnt/test
mount: /dev/flash_b_d_lvm/flash_b_d__raid00 is [b]not a valid block device
[/b]
# ls -al /dev/flash_b_d_lvm/flash_b_d__raid00          
lrwxrwxrwx 1 root root 43 2009-04-04 12:15 /dev/flash_b_d_lvm/flash_b_d__raid00 -> /dev/mapper/flash_b_d_lvm-flash_b_d__raid00

# ls -al /dev/mapper/flash_b_d_lvm-flash_b_d__raid00
brw-rw---- 1 root root 253, 0 2009-04-04 11:57 /dev/mapper/flash_b_d_lvm-flash_b_d__raid00

# lsmod | grep mod
dm_mod                 56160  0 
When I did the following dd speed tests the LVM raid was slower on reads??? Did not test write speeds.

Code: Select all

dd if=/dev/sdb1 of=/dev/null bs=512 count=4000
 dd if=/dev/sdd1 of=/dev/null bs=512 count=4000

 dd if=/dev/flash_b_d_lvm/flash_b_d__raid00 of=/dev/null bs=512 count=4000

# vgdisplay
  --- Volume group ---
  VG Name               flash_b_d_lvm
  System ID             
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               0
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               9.37 GB
  PE Size               4.00 MB
  Total PE              2398
  Alloc PE / Size       500 / 1.95 GB
  Free  PE / Size       1898 / 7.41 GB
  VG UUID               wgYgfX-q1CT-zReN-fExh-xyS3-Zfw1-Pn2KhZ
   
# 
# lvdisplay
  --- Logical volume ---
  LV Name                /dev/flash_b_d_lvm/flash_b_d__raid00
  VG Name                flash_b_d_lvm
  LV UUID                bgc4pv-IieA-FHs0-wsRk-1K8D-YDMV-UGUp2e
  LV Write Access        read/write
  LV Status              NOT available
  LV Size                1.95 GB
  Current LE             500
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
I was feeling good until I rebooted and tested the read speed..
Last edited by rrolsbe on Sun 05 Apr 2009, 15:17, edited 2 times in total.

User avatar
Dougal
Posts: 2502
Joined: Wed 19 Oct 2005, 13:06
Location: Hell more grotesque than any medieval woodcut

Re: lvm and RAID with USB thumb drives [SOLVED]

#4 Post by Dougal »

rrolsbe wrote:UPDATE:

When I rebooted the BIOS reassigned my LVM devices from /dev/sdc1& /dev/sdd1 to /dev/sdb1 & /dev/sdd1.
To make sure you have the same device names every boot, you might want to use the uuid of the drives, rather than /dev/sd[a-z].

Disktype should give you the info. For example:

Code: Select all

bash-3.00# disktype /dev/sdb

--- /dev/sdb
Block device, size 1.906 GiB (2046557696 bytes)
DOS/MBR partition map
Partition 1: 1.003 GiB (1077479424 bytes, 2104452 sectors from 63, bootable)
  Type 0x83 (Linux)
  Ext3 file system
    UUID BDBE2A9B-F9A6-4EB1-A3D4-9FF0C99FB709 (DCE, v4)
    Volume size 1.003 GiB (1077477376 bytes, 263056 blocks of 4 KiB)
Partition 2: 917.8 MiB (962357760 bytes, 1879605 sectors from 2104515)
  Type 0x83 (Linux)
  Ext3 file system
    UUID 25037EF3-A8C8-4689-AF9B-6C20508FA860 (DCE, v4)
    Volume size 917.8 MiB (962355200 bytes, 234950 blocks of 4 KiB)
Now, you might be able to use the lvm commands directly with the uuid (you can put the uuid -- or even the disk label -- instead of the device in fstab), but if not, you can always use udev to give you the uuid device nodes.
I'm including the udev rules file that handles all persistent storage node creation (as Puppy doesn't use this...). You just need to put it in /etc/udev/rules.d and hopefully it won't cause any trouble.

Anyway, here's the kind of thing you will get:

Code: Select all

bash-3.00# ls -1 /dev/disk/by-uuid/                                             07e2703e-0118-47fb-84f9-98570b9f41bf
25037ef3-a8c8-4689-af9b-6c20508fa860
bdbe2a9b-f9a6-4eb1-a3d4-9ff0c99fb709
Those are all links to the device nodes, but you can nonetheless mount them (and so I assume use them with lvm):

Code: Select all

bash-3.00# mount "/dev/disk/by-uuid/25037ef3-a8c8-4689-af9b-6c20508fa860" /mnt/data/
bash-3.00# ls /mnt/data/
archive  dev           etc  lost+found  proc          root  tmp  var
bin      devx_214.sfs  lib  mnt         pup_214R.sfs  sbin  usr  zdrv_214R.sfs
Oh, and you should use the "

Code: Select all

" formatting to make your post more readable.
Attachments
60-persistent-storage.rules.gz
(1.36 KiB) Downloaded 846 times
What's the ugliest part of your body?
Some say your nose
Some say your toes
But I think it's your mind

rrolsbe
Posts: 185
Joined: Wed 15 Nov 2006, 21:53

Re: lvm and RAID with USB thumb drives [SOLVED] Finally

#5 Post by rrolsbe »

Dougal wrote:
rrolsbe wrote:UPDATE:

When I rebooted the BIOS reassigned my LVM devices from /dev/sdc1& /dev/sdd1 to /dev/sdb1 & /dev/sdd1.
To make sure you have the same device names every boot, you might want to use the uuid of the drives, rather than /dev/sd[a-z].

Disktype should give you the info. For example:

Code: Select all

bash-3.00# disktype /dev/sdb

--- /dev/sdb
Block device, size 1.906 GiB (2046557696 bytes)
DOS/MBR partition map
Partition 1: 1.003 GiB (1077479424 bytes, 2104452 sectors from 63, bootable)
  Type 0x83 (Linux)
  Ext3 file system
    UUID BDBE2A9B-F9A6-4EB1-A3D4-9FF0C99FB709 (DCE, v4)
    Volume size 1.003 GiB (1077477376 bytes, 263056 blocks of 4 KiB)
Partition 2: 917.8 MiB (962357760 bytes, 1879605 sectors from 2104515)
  Type 0x83 (Linux)
  Ext3 file system
    UUID 25037EF3-A8C8-4689-AF9B-6C20508FA860 (DCE, v4)
    Volume size 917.8 MiB (962355200 bytes, 234950 blocks of 4 KiB)
Now, you might be able to use the lvm commands directly with the uuid (you can put the uuid -- or even the disk label -- instead of the device in fstab), but if not, you can always use udev to give you the uuid device nodes.
I'm including the udev rules file that handles all persistent storage node creation (as Puppy doesn't use this...). You just need to put it in /etc/udev/rules.d and hopefully it won't cause any trouble.

Anyway, here's the kind of thing you will get:

Code: Select all

bash-3.00# ls -1 /dev/disk/by-uuid/                                             07e2703e-0118-47fb-84f9-98570b9f41bf
25037ef3-a8c8-4689-af9b-6c20508fa860
bdbe2a9b-f9a6-4eb1-a3d4-9ff0c99fb709
Those are all links to the device nodes, but you can nonetheless mount them (and so I assume use them with lvm):

Code: Select all

bash-3.00# mount "/dev/disk/by-uuid/25037ef3-a8c8-4689-af9b-6c20508fa860" /mnt/data/
bash-3.00# ls /mnt/data/
archive  dev           etc  lost+found  proc          root  tmp  var
bin      devx_214.sfs  lib  mnt         pup_214R.sfs  sbin  usr  zdrv_214R.sfs
Oh, and you should use the "

Code: Select all

" formatting to make your post more readable.[/quote]

Dougal

Thanks for the post

The pvcreate commands below appear to write a new random UUID ID's to both flash devices.  I think writing this UUID should (is intended to) solve the problem of being scanned in different later. I believe the mount command is NOT referencing by the physical device names?  If true, I still do not know why the LVM devices can't be mounted using the same mount command after a reboot.

This go round, I used the same two flash devices plugged directly into two of my EEE USB ports.  The speed WAS faster using the RAIDO, see dd results later in this post.  So I guess my TARGUS USB 2.0 hub is the bottleneck?

[code]pvcreate -Zn /dev/sdb1  //The -Zn does not rewrite the 1st 2048 bytes
pvcreate -Zn /dev/sdd1 //The 1st 2048 bytes contain the partition name
                                     //Linux LVM below in the fdisk -l output

Code: Select all

# fdisk -l
Disk /dev/sdb: 2046 MB, 2046033920 bytes
63 heads, 62 sectors/track, 1023 cylinders
Units = cylinders of 3906 * 512 = 1999872 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1023     1997888   8e  Linux LVM

Disk /dev/sdd: 8023 MB, 8023703552 bytes
247 heads, 62 sectors/track, 1023 cylinders
Units = cylinders of 15314 * 512 = 7840768 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1        1023     7833080   8e  Linux LVM

Code: Select all

vgcreate flash_b_d_lvm /dev/sdb1 /dev/sdd1
lvcreate -L 2000m -i 2 -I 64 flash_b_d_lvm -n flash_b_d_raid00

 mke2fs /dev/flash_b_d_lvm/flash_b_d_raid00 
 mount /dev/flash_b_d_lvm/flash_b_d_raid00 /mnt/data

Code: Select all

# dd if=/dev/flash_b_d_lvm/flash_b_d_raid00 of=/dev/null bs=512 count=
4000 
4000+0 records in
4000+0 records out
2048000 bytes (2.0 MB) copied, 0.0917553 s, 22.3 MB/s
# dd if=/dev/sdb1 of=/dev/null bs=512 count=4000 
4000+0 records in
4000+0 records out
2048000 bytes (2.0 MB) copied, 0.161036 s, 12.7 MB/s
# dd if=/dev/sdd1 of=/dev/null bs=512 count=4000 
4000+0 records in
4000+0 records out
2048000 bytes (2.0 MB) copied, 0.151505 s, 13.5 MB/s
Now for a reboot.

Again, Thank You Very Much for your reply.
Regards, Ron

PS. I placed the file you attached in your post in /etc/udev/rules.d then rebooted. Did not have the directory /dev/disk after the reboot? I then hide the original file in the rule.d directory and Puppy would not boot into X. I tried to use the CODE quoting did not work, not sure why? I code marked a LARGE section then several small code sections, neither worked?

FINALLY Working!!!!

Code: Select all

vgchange -a y flash_b_d_lvm
mount -t ext2 /dev/flash_b_d_lvm/flash_b_d_raid00 /mnt/data

#mount
/dev/flash_b_d_lvm/flash_b_d_raid00 on /mnt/data type ext2 (rw,errors=continue)
Attachments
disktype.txt.gz
LVM debug before and after reboot.
(1.23 KiB) Downloaded 796 times

User avatar
SilverPuppy
Posts: 143
Joined: Fri 29 May 2009, 02:21

Thanks a bunch.....

#6 Post by SilverPuppy »

This is really great, I was wondering how to do this.

I'm sorta obsessive-compulsive, you see, and hate to toss anything that still works. I'm going to use this to create RAID 0 swapping with old HDDs less than 1GB. Those old drives were terribly slow, so RAID 0 will double it up for me and help that little problem.

I hope it works. The Slackware file repo browser is down right now, so I had to get creative with the text directory files....but I got the files, finally. We'll get it done. :D

User avatar
SilverPuppy
Posts: 143
Joined: Fri 29 May 2009, 02:21

OOPS......

#7 Post by SilverPuppy »

Well, after arguing with it for awhile, and somewhat getting it working, I discovered that what I was attempting wasn't necessary at all.........all I had to do was create 2 swap partitions on 2 separate devices, replace the swapon command with the "real" one (it's in another forum) so that I could make them the same priority, and viola! Striped swap. So that's that.

I did get it so that the things would sorta work if I formatted it EXT2, but it was very slow, much slower than the physical devices' native speed, and I don't gather quite why. It seemed to take breaks, and have lots of I/O errors. I'm not sure that wasn't due to a mismatch between the partition type flag (which I never changed from swap) and the FS.....however it may be, I now have a very good idea of how to make this all work for future reference, and it's all wunnerful! I'm sure I can make it work in the future. I'm now that much smarter thanks to you all!

Post Reply