[Solved] Intel ICH10 RAID access from Fatdog64?

A home for all kinds of Puppy related projects
Post Reply
Message
Author
User avatar
Яadiøaktive TΣknik
Posts: 22
Joined: Sun 01 Jul 2012, 21:49
Location: Orlando, FL USA

[Solved] Intel ICH10 RAID access from Fatdog64?

#1 Post by Яadiøaktive TΣknik »

Forgive my ignorance and learning curve here. And it's not really crucial but I think it would be cool to access my three drive SATA RAID from Fatdog64. I did web reading on the subject and it seems dmraid could do it but I haven't seen the right .pet for that yet or it's not ready yet?

The drives show individually from fdisk -l, which always means the RAID wasn't recognized and mounted. I see this on a LIVE-CD boot of Fedora Core and Ubuntu. But not from an installed-to-hard-drive version of those. After install to hard drive, those versions of Linux find and mount the RAID.

Could it be as simple as installing Fatdog64 to a hard drive first? Currently I boot from a fast USB stick and save user profile, etc to an SSD and that has worked fantastic. I love the portability. Also, if you installed a package that didn't help out, you can just push the reset button!

Thanks!
Last edited by Яadiøaktive TΣknik on Thu 12 Jul 2012, 15:13, edited 2 times in total.

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#2 Post by jamesbond »

Nah, on the insistence of gcmartin I have compiled LVM/LVM2 tools as a pet, but there is no dmraid pet compiled for Fatdog64 yet. Even if such tool exist Fatdog won't be able to boot off the RAID partition; you need non-RAID boot partition and start the raid on rc.local. Note that /etc/fstab is mounted at the beginning of rc.sysinit before rc.local is called, so you can't mount RAID via /etc/fstab too - well you can, but you need to do "mount -a" again in rc.local.

You're welcome to contribute a pet; hopefully with a working configuration / customised tools for Fatdog users :D
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

User avatar
Яadiøaktive TΣknik
Posts: 22
Joined: Sun 01 Jul 2012, 21:49
Location: Orlando, FL USA

#3 Post by Яadiøaktive TΣknik »

jamesbond wrote:Nah, on the insistence of gcmartin I have compiled LVM/LVM2 tools as a pet, but there is no dmraid pet compiled for Fatdog64 yet. Even if such tool exist Fatdog won't be able to boot off the RAID partition; you need non-RAID boot partition and start the raid on rc.local. Note that /etc/fstab is mounted at the beginning of rc.sysinit before rc.local is called, so you can't mount RAID via /etc/fstab too - well you can, but you need to do "mount -a" again in rc.local.

You're welcome to contribute a pet; hopefully with a working configuration / customised tools for Fatdog users :D
^^^^^^^^^^^^^^^^^^^^^^^^
Neat. Can you point me to some tools to create the .pet for Fatdog64 600 Final? Would it be easier to just see and access the LVM? I don't want to boot from it.

That wouldn't speed up your top-notch distro, maybe shave a second off boot time. Mainly, I want to stay in Fatdog64 and only boot Win7 for DirectX gaming, etc.

- Writing from Fatdog64 600 Final
Best Regards,

Яadiøaktive TΣknik Яøøssell
***** Электроника ******

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#4 Post by jamesbond »

Sure!

1. First off, get the "devx" (Puppy's term for "development packages" - that is, compilers, development headers and libraries etc). You can get it using SFS Manager from Control Panel, or you can download it yourself from the mirrors. If you download it yourself you need to load it using the SFS Loader.

2. Then you can build the app as usual (./configure etc).

3. To make a pet, there are two ways of doing it (both are not always successful but usually works fine with app that uses autoconf):
Instead of "make install", do either one of the following:
a) "new2dir make install", or
b) "make DESTDIR=/tmp/path install" followed by "dir2pet /tmp/path". Note that "path" must consist of a name and a version, separated by a dash ("-") character.

For further info and examples, see here http://puppylinux.org/wikka/Compiling but beware that it is geared towards 32-bit puppy, so some of the configure CFLAGS are not correct and won't work in 64-bit environment.

cheers!
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#5 Post by jamesbond »

BTW - which raid do you use - dmraid or mdraid?
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

User avatar
Яadiøaktive TΣknik
Posts: 22
Joined: Sun 01 Jul 2012, 21:49
Location: Orlando, FL USA

#6 Post by Яadiøaktive TΣknik »

jamesbond wrote:BTW - which raid do you use - dmraid or mdraid?

mdraid I think. It's Intel Matrix RAID 0, so I think that would work for me. In the past, the mounting of my RAID from Linux has been pretty much transparent for me and I haven't had to get into the details much.

Tested a few other GNU distros and none of them seem to auto-mount this RAID. It would be cool to have access to my Win7 media collection from Fatdog64. Did you find an mdraid that could be compiled 64bit?

Pretty sure mdraid would work with this very mainstream controller. The Intel Southbridge Chipset 10 containing the RAID controller ships on most of the modern mainboards I look at.

- Fatdog64 600 Final [still the best thing running]
Best Regards,

Яadiøaktive TΣknik Яøøssell
***** Электроника ******

User avatar
Яadiøaktive TΣknik
Posts: 22
Joined: Sun 01 Jul 2012, 21:49
Location: Orlando, FL USA

#7 Post by Яadiøaktive TΣknik »

jamesbond wrote:BTW - which raid do you use - dmraid or mdraid?

Just did a quick check with some other mainstream distros [non-puppy] and they are using dmraid to accomplish mounting the raid. My forum reading in the puppy forums has taught me that these used to work with our kernel but don't anymore since a new release, not even for 32 bit.

There's been several attempts but dmraid either won't compile properly or won't run properly. I tried several different puppy distros and all are missing dmraid. I think this may have to wait until the packages are more compatible with the kernel, especially 64 bit. Possibly outdated info.

Fatdog64 600 Final [good doggie. here's another snack]
Best Regards,

Яadiøaktive TΣknik Яøøssell
***** Электроника ******

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#8 Post by jamesbond »

I can try to compile both dmraid and mdraid, but I don't have such hardware handy, so I can't test that it works. I'm more concerned about data loss if you use the wrong driver though.
This is mdraid: https://raid.wiki.kernel.org/index.php/Overview
This is dmraid: http://en.gentoo-wiki.com/wiki/RAID/NVRAID_with_dmraid
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

User avatar
Яadiøaktive TΣknik
Posts: 22
Joined: Sun 01 Jul 2012, 21:49
Location: Orlando, FL USA

#9 Post by Яadiøaktive TΣknik »

jamesbond wrote:I can try to compile both dmraid and mdraid, but I don't have such hardware handy, so I can't test that it works. I'm more concerned about data loss if you use the wrong driver though.
This is mdraid: https://raid.wiki.kernel.org/index.php/Overview
This is dmraid: http://en.gentoo-wiki.com/wiki/RAID/NVRAID_with_dmraid

First of all, let me say thank you for even looking into this. I think it's hard to get excited about a feature that you yourself can't test - therefore don't need right now. And there will always be someone wanting you to make Fatdog64 do new tricks, even if it's only once and only for them.

Well if this works, I'll gladly use it heavily. My alternative is to copy my library of music, movies and tv series to a new 2TB single hard drive then access that from Fatdog64. The positive aspect of that is having a backup of my stuff. I'm willing to test the .pet(s) for assurrance they work well.

Finally, great links. Better than what I found and it helps explain why my array was not portable when moving from an NVIDIA board to one with a newer Intel RAID Controller. Could have done without the "cheap" bash. This ICH10 beats my HP Proliant SCSI RAID, which was like $10K.

After reading the links, I'm still not positive. What I can say for sure is that Linux distros which do automount this array have included dmraid cli. I was unable to find anything mdraid on those OS installs, though I could still be wrong. I'm doing more web reading to be sure which it is!
Best Regards,

Яadiøaktive TΣknik Яøøssell
***** Электроника ******

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#10 Post by jamesbond »

Thanks for the understanding. I have uploaded to the package repository:
- mdraid
- dmraid (requires lvm2)
- updated lvm2 that works with dmraid

Notes:
1. mdraid consist only of two binaries
2. lvm2 is a bit more complex and it comes with a lot of tools, it also comes with some udev rules.
3. dmraid is compiled with udev support and comes with some udev rules.

All pets will install binaries in /sbin and /lib64. Existing /lib64/libdevmapper.so will be overwritten with the new one when you installed lvm2.

I have no means to test these binaries, so please use with care and especially backup your data first.
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

User avatar
Яadiøaktive TΣknik
Posts: 22
Joined: Sun 01 Jul 2012, 21:49
Location: Orlando, FL USA

#11 Post by Яadiøaktive TΣknik »

jamesbond wrote:Thanks for the understanding. I have uploaded to the package repository:
- mdraid
- dmraid (requires lvm2)
- updated lvm2 that works with dmraid

Notes:
1. mdraid consist only of two binaries
2. lvm2 is a bit more complex and it comes with a lot of tools, it also comes with some udev rules.
3. dmraid is compiled with udev support and comes with some udev rules.

All pets will install binaries in /sbin and /lib64. Existing /lib64/libdevmapper.so will be overwritten with the new one when you installed lvm2.

I have no means to test these binaries, so please use with care and especially backup your data first.
Success!

RAID is mounted and I'm playing it's media files from Fatdog64. It was a snap to install dmraid. After that, web searched "dmraid mount" and found many examples. Also learned I can edit etc/fstab to automount this RAID on each boot. Thanks.

This could be marked [solved] if the forum works that way.
Best Regards,

Яadiøaktive TΣknik Яøøssell
***** Электроника ******

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#12 Post by jamesbond »

Wow, glad to hear that ! :)

After you put the stuff in /etc/fstab - do you need to do another "mount -a" in rc.local or does it work automatically? I'm a bit surprised because "mount -a" is done very early in the boot process, before udev is started, so I thought it wouldn't work unless you do "mount -a" again later.

Yes, if you can edit the first post and add the word [Solved] to the title that would be good. Also - if you have the time, would you mind posting a simple step-by-step HOWTO in the HOWTO section of the forum, on how to do it? Write as much as or as little your feel comfortable :)

cheers!
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

User avatar
Яadiøaktive TΣknik
Posts: 22
Joined: Sun 01 Jul 2012, 21:49
Location: Orlando, FL USA

#13 Post by Яadiøaktive TΣknik »

People with the following RAID controllers can mount their arrays using dmraid.

Code: Select all

 dmraid -l 
  • asr : Adaptec HostRAID ASR (0,1,10)
    ddf1 : SNIA DDF1 (0,1,4,5,linear)
    hpt37x : Highpoint HPT37X (S,0,1,10,01)
    hpt45x : Highpoint HPT45X (S,0,1,10)
    isw : Intel Software RAID (0,1,5,01)
    jmicron : JMicron ATARAID (S,0,1)
    lsi : LSI Logic MegaRAID (0,1,10)
    nvidia : NVidia RAID (S,0,1,10,5)
    pdc : Promise FastTrack (S,0,1,10)
    sil : Silicon Image(tm) Medley(tm) (0,1,10)
    via : VIA Software RAID (S,0,1,10)
    dos : DOS partitions on SW RAIDs

    All you need is dmraid and lvm2. I installed them by selecting dmraid from Setup>FD64 Control Panel>System>FD64 Package Manager, which did the job. Handles any needed libraries, etc.

    There's a long and complex way to mount your array. But there's an exceedingly easy way too. For me, I just open a command prompt and type

    Code: Select all

     dmraid -ay 
    The -ay means 'activate' and 'yes'. Easy as pie. Works.

    dmraid will find and activate, then mount and make available any arrays on the controllers listed. I can confirm that it works great with my Intel controller. Also testing JMicron RAID Controller. RAID shows on desktop like other mounts.

    Thanks to jamesbond for making this available in Fatdog64 600.
Best Regards,

Яadiøaktive TΣknik Яøøssell
***** Электроника ******

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#14 Post by jamesbond »

You're much welcome.

There are a few ways to run commands at boot / at startup.

1. You can insert the commands in /etc/rc.d/rc.local. This is run in the final stages of sysinit, and run as root.

2. You can create your own services in /etc/init.d. "services" are just simple bash scripts, you can copy from existing ones and create it. Service are also run in the final stages of sysinit.

3. You can insert the commands in /etc/profile.local. This is run by the login shell (and thus only run once per login) and run as user.

--- all the above are run before X desktop are started ---

4. You can create (or symlink) a script in $HOME/Startup or /etc/xdg/Startup. These scripts will be started after X desktop are ready (thus can be used to start graphical applications), and are run as user.

Of course, if you autologin as root then "user" == "root".

cheers!
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

User avatar
Яadiøaktive TΣknik
Posts: 22
Joined: Sun 01 Jul 2012, 21:49
Location: Orlando, FL USA

#15 Post by Яadiøaktive TΣknik »

1. You can insert the commands in /etc/rc.d/rc.local. This is run in the final stages of sysinit, and run as root.
I looked at this and decided to put it in rc.local, which turns out to be a file with no commands to run, just commented out CPU throtting something-or-other. I put it at the end it totally works. Good info. Now my RAID volumes are automounted. See attached dm-1 and dm-2. Text added with mtpaint.
Best Regards,

Яadiøaktive TΣknik Яøøssell
***** Электроника ******

Post Reply