Puppy Linux Discussion Forum

[SBP]

Hi jbv

Thanks for your post. I have the same goal of a free debian based NAS, where also other "things" can be installed, like SqueezeboxServer from Logitech.

I have tried openmediavault, did you by any chance have a look at that? It seems very nice. http://www.openmediavault.org/ it is free and actually working fine already. It is a Debian OS with Openmediavault installed on top

If possible I would like a system which can be run in read-only mode, so that the OS can not be corrupted, and it can run without problems on a USB or CF-card, will you take that in consideration as well?

[Aitch]

jbv,sickgut, and saintless

I have followed your work and enthusiasm and wish you all well with the new project
It's always good to see similar ideas and energies merge into something where the sum is greater than the parts

Best of luck
...and sickgut...look after yourself...health first, hobby later!

Aitch

[jbv]

Hi sickgut, thanks. My absence (silence) over the last 6-8 weeks has been due to work more than anything else. I've still got a tough 1-2 weeks ahead, then hopefully things will return to normal and I may have a bit of time.

Hi SBP, running entirely from the USB/CF was/is one of my key requirements. In this regard puppy/pussy is perfect. I couldn't even guess at how many times I've just pulled the stick or turned the power off to save my bacon while playing. The system will only ever write back to the USB/CF when the sys-op performs an action that mandates it. This means more than just a configuration change. If you want a configuration change to "stick", then you will need to snapshot the change.

I've done quite a bit of research over the last 12-18 months. At first, I thought I was just going to buy some standard hardware and chuck linux on it, so I researched the hardware and also benchmarked hard-drives, USB sticks, Ethernet chips and quite a bit of other low-level stuff. Then I bought enough to build the first two boxes (without all of the hard drives). I did this while the hardware was cheap, our dollar was the highest it had been in years, and I had a few bucks I could spend. So, I've had the basic hardware now for about 12 months. Then started playing with various *nix systems and various packages such as freeNAS and pretty much everything else I could find, including systems that charged you for their distro. While doing this, I was learning about the limitations, quirks, and idiosyncrasies of things from the filesystem though to the RAID and *nix itself.

My hardware setup is an Atom dual-core 525 mainboard with 4Gb of RAM, 4 x SATA 3 ports, 8 x USB ports. With port-multipliers on the SATA ports, I can host 20 Hard Drives, although the cheap case I bought only accommodates 9 x 5.25" drives, so with my 5-in-3 drive bays, I can hold 15 drives in each box - I bought the case because it was very cheap for a 9 drive case and the power-supply is more than adequate for the task. At the time 2Tb drives were in the "sweet spot" and I figured that when I had more than 45Tb of data, I would just add another 2 boxes or change the case.

From a system software perspective, the goal is as follows:

Basically, the system will be (is) a pure and 100% clean Debian that runs fully from the USB stick. The drive arrays are setup so that a file is always placed on a single disc based on space balancing. The raid is provided by periodic snapshots that are taken of a cluster of drives with the raid/repair/rebuild info spread across the cluster so that any drive failure can be repaired/rebuilt from the snapshot data. This has a few advantages over a traditional real-time raid. For starters it lets you take out any individual disc and read it in any machine. If you lose a file or even an entire drive, the file or drive can be rebuilt and it can usually do this faster than a traditional raid. As the raid-data is freshened periodically (once a day) you can schedule this to occur late at night or any time that suits you, even though it doesn’t really impact performance that much. When reading or writing from/to the drive, you basically get the raw read/write speed of the drive. While a distributed or traditional raid array may provide a smidgeon faster reads if done with special hardware, software raid doesn’t really provide the same performance boost, so there is no real loss here.

On top of all this is a disk-pooling scheme that just lets you add drives and it sorts itself out. With a traditional raid, adding drives is a real pain and disk pooling becomes messy. This scheme does away with all of that. Just chuck in a drive and you don’t need to rebuild the array or worry about things. I ended up deciding on using the XFS filesystem instead of ZFS. I know ZFS has some great features and disc polling and raid are standard, but after digging into ZFS a bit more, while it does provide these things, making changes to a live system is almost impossible. Even adding a drive becomes an admin nightmare. With ZFS, if you have a major problem (which I know doesn’t happen very often) you are screwed because the data is in a non-standard format and spread across multiple drives. This basically means you have the potential to lose everything. With this scheme, in a total worst case scenario you will only ever lose the data-files that were on a single drive, although you can even rebuild this from the magic raid rebuild data.

You can't effectively backup this much data, so instead I am looking to provide as much redundancy as possible. In the first instance, I am thinking to use Rsync to sync two boxes (why I have 2 boxes already). The final "grand plan" is that the USB stick can simply be cloned and plugged into another box, change one or two things and hey presto, it just becomes another NAS in your network. Ive got some ideas on how this process could be somewhat auto-configuring, so that you just build the box, plug it in, it sorts itself out and then you just review the auto-configuration and if happy with it tell it to snapshot itself, but things like that are quite a way off.

As mentioned, I've already researched all of the core tools and components. I know what packages I intend to use. I've also used pussy to do quite a bit of testing to make sure I can build the stuff from source that I need to, that it works as intended, and produces the desired results and is solid. In a lot of instances, I haven't pushed the configuration stuff, so I still have quite a few black holes in that area, but the research I have performed before "locking in" the package or tool has given me a very high level of confidence that I will be able to get out of the hole easily.

My system won't really make much use of the desktop as it will be quite rare that anyone should need to touch it. If anything, this means the desktop and tools should look and feel natural which is why I'm keen to make it look and feel as much as possible like a Win-XP type system. While a lot of stuff will be done remotely with things like Webmin etc, I really do want the machine to look and feel lean and clean.

This is where saintless and I come together. He is looking at the top end, and I'm always thinking about what it is doing underneath. How is that going to affect my streaming/serving etc. and what sort of load is it going to have.

I'm also very concerned with support or the lack thereof, which is why I want things to be laid out properly. Here I am referring to support that I may need/require in the first instance, and more importantly the lack of support others (my friends) are going to get from me if they deploy this.

I also have a family with 3 children and hold a senior position professionally. This means that when things are going well at work, I can come home and mess around for a few hours and sometimes even spend the entire weekend on this. However, when things aren't or something special is taking place I can't. From my side, progress will be somewhat spasmodic and in all honesty it will be a few months before we have anything.

I know I have not listed the packages and/or libs I intend to use, and I won't do so here. Right now, I'm still working on the core and would like to get the basic desktop sorted. I will document some of my plans once we get a site/forum up ... sometime soon ... But without meaning to sound rude, I'm way past the "have you looked at" stage. If you've read the last 95 pages, I think you may have seen that when I start digging, I keep digging quite deeply and do my best to really understand what is going on and how it will fit into the "Grand Plan". Therefore, I'm at the point with my "concept" that if someone comes along with another idea or suggestion, they really will need to give me some serious meat, and prove to me that it is the right thing and also convince me that they are prepared to help or follow though. Basically, I think I've done enough digging and playing. I'm comfortable that if something is broken or doesn't work that we will have the ability to fix it, or we have a "Plan-B" we can switch over to.

This does not mean that we are not open to suggestions or discussion, but they will need to be well researched suggestions that can be substantiated with technical facts, and quite possibly the work required to implement it.

My view on the server vs desktop thing is quite simple. If you want the FileServer then just don't open the application software. Better yet, don't login, the box will just run and sort itself out (once you've set it up). If you want the desktop/application stuff, then just don't start the server services.

In the back of my mind is also the fact that if I live long enough to get this done, then my next project will be to take the same basic core (hardware and software) and make my own media-player, so once again the desktop / frontend comes into play.

Anyway, I hope this give you an idea of things in general from a system and what to expect point of view. As I/we don't want to hijack this thread, this will be my last post here regarding where we are headed.

Cheers

[jbv]

[saintless]

Thanks, Sickgut, Aitch...

Just like to add we are still here and we will help with answers, testing and fixes.
Almost all of the hard work for our new project lies on the shoulders of JBV, so I am the one with more spare time for PussyLinux for the moment.

Cheers

[atv]

Hi, Jbv:

You wrote to sickgut: ..[]..

I think I've sent you a script that will do the merge, or at least could be slapped into shape without much work... end quote

Could you post or IM me this script, as Would like to experiment with it.
regards, Andres

[sickgut]

just a little update

im in the process of getting things organized in my new more balanced lifestyle. Work will proceed reasonably soon but there has been a development that i didnt plan for....

the rasp pi has finally been released. I thought it would never happen in my lifetime but hey. I fully intend to support the raspberry pi.

After updating the current Pussy Xtra to its newer glory with bug fixes and improvements that have been posted on this forum, i will begin work on the Rasp Pi version of Pussy, so the new 64 bit version will have to wait.

Also to clarify some confusion related to the current Pussy Xtra (32bit) and the new 64bit fasterkill... the new 64 bit version will not be a carbon copy of the current 32bit Pussy Xtra but with 64 bit binaries etc... it will be a different OS and be aimed at a higher system spec. The new 64 bit fasterkill will also have everything in one squashfs and many many improvements at a base level that wherent included in the current Pussy Xtra.

The current Pussy Xtra will be kept updated so its usable but no more design changes.

32 bit computers are generally lower spec than 64 bit, so the 32 bit option for Pussy will remain the current Pussy Xtra that is designed to run on low end computers and the new 64 bit fasterkill is aimed at more modern computers with a higher system spec.

I do not know what desktop environment will be included in the Pussy for Rasp Pi, but initially it will be made similar to the current Pussy Server. This will give us a working reliable base that will let you add anything you like. After this has been completed i will do the same thing for the 64 bit version. When these are completed, anyone interested in developing for Pussy for the Raspi or the 64 bit platform will have all they need to begin with.

OK over and out

[update]
i didnt realize there would be a delay in rasp pi shipping, the 64 bit version will probably get worked on before the Rasp Pi version because of this *sigh*

[Lobster]

[sickgut]

[sickgut]

hello people

if your still comming back to this thread then i really do thank you for your for being patient.

Im still finding my feet after massively changing my lifestyle and i havent been able to make room for pussy development as yet. Pussy was something i did late at night and these days im actually going to bed at a normal hour rather than pulling all nighters every night then trying to make up for it with power sleeps.

I have come to a realization and a few directional changes that i feel are right for the distro itself and also that are actually attainable for me to do, and do well. There is so many linux distros out there and half assedness will get no one anywhere at all. I need to be realistic in my approach, have fragile health issues and this means i have problems keeping deadlines and also working as a team as to do so kinda of implies that i can do something within a certain time frame and i need to commit to it, but realistically this doesnt work at all with the type of issues i have as i have no real control over what happens to me on any given day. This means that i need to work at my own pace, dont promise things in advance that will be done at a given time frame and just work when i am able to do so.

Unfortunately this means that i need to be more realistic with what goals i can actually achieve, the current Pussy OS that is a server and desktop complete solution isnt practical for me as i personally have no handle whatsoever on asthetics, and desktop orientated things like what happens when you right click on something, auto mounting, window managers and other things. Its simply not been my thing and i have no real idea whatsoever how a normal person actually uses a computer at all.

Pussy will be changed from the current setup to something else. Currently we have the base iso and the Xtra iso and server iso. (ie. a puppy like OS, a complete solution and a OS for an insta out of the box server with auto network detection but no desktop environment).

I am going to drop development on the base iso, the xtra, and keep the server OS. This is what personally interests me and it is what i actually know and i am capable of doing. There are so many other linux OSes that are similar to the base iso (puppy!!) and the xtra ISO, and i feel that we will always just be a second rate OS compared to the already available solutions.

The desktop stuff is the real clincher. I am not suited for that and has been a source of stress and many sleepless nights with very little pay off because, lets face it. Its been done before.

When my raspberry PI arrives i will whip up a smaller sleeker desktop OS based on debian for this device but that will be the last time i make a general purpose OS with a desktop environment in the conventional sense. The current default debian image with lxde etc is inefficient in my opinion and our jwm and optional rox filer setup will run much faster.

After the Pussy Pi is completed, i will start work on the Pussy server and this will be released in non PAE CPU, PAE 32 bit 686 CPU, and also 64 bit CPU types as well as the Rasp Pi. The non PAE, 686 and 64 bit versions may very well be released on the same dvd or cdrom iso, while the Rasp Pi image would be released in the .img format for imaging to usb sticks and sd cards.

This new Pussy will be similar to the current pussy server iso except it will have an instantly accessible and custom help system that will inform the user about all the ins and outs and how to access the help system immediately after booting. There will also be beep codes for certain menu selections and boot sequences to aid with running on systems with no monitor.

Altho an out of the box server solution, this will be a very stable and sane base with which to add any kind of desktop environment that appeals to the user. I feel this new direction is well within my reach and i believe that i can produce the best out of the box server solution that is currently available. This kind of Pussy could be used as a tool by people wanting to make their OS but want to start with the smallest most efficient system that already has networking and drivers etc installed, inlike the mini (30mb) and standard (180mb) debian ISOs that require manual commandline configuration and commands just to connect to a network so that you can apt-get things as this is a real nightmare for new users.

OK people you will hear from me when i have something new to show you all.

[Lobster]

@ sickgut

Thanks for sharing all that info.
I like kicking and punching, occasionally I go and slap an old tree on the common (urban parkland] but I don't have a punch bag which would be a better option. My martial arts training is now centered around some rather unusual applications.
Poetry and music is good for ones need for expression. Coding till you drop is fine if you love coding.
Otherwise as you mention it is soul destroying.

I like your Pussy Console for the Rpi. Sounds a fun project.
I am happier when Puppys are happy, it produces a more mature distro for example the just released 5.3.3 is a 'just works' distro. No fuss, just works.

I think you understand, rightly, that drive comes from being happy with what one is doing. You will do good. I wish you well.

[saintless]

Glad to hear you found what you really want to make from pussy linux, Sickgut.

Sadly I think dropping the base and extra version at the current state is the death sentence for this thread, because I don't remember a single post here (except yours and mine) to show any interest for the server version. If you open different thread for the server version you will see what I mean. Anyway I hope I'm wrong.

Like Lobster wrote it is best to do what you really like and love to do.

Wish you luck with pussy server and raspberry project!

Cheers

[nooby]

I second what Lobster and Saintless tells you.
Always follow your inner best motivation.

To try to satisfy guys like Nooby is to head for disaster and bad health

I am too demanding. Devs should do what they get rewards from.
Not what selfish users wants them to do. Computing should be fun
and not a core.

The server version of Pussy most likely should have it's own thread. I agree.

Thanks for all the effort you did put into the base and extra.
Wish you all the best.

[Aitch]

Sickgut/others
maybe aside, but relevant

I don't know if you all are aware of the 2012 energy shifts at all, but see if any of this helps you understand the changes going on...
http://leeharrisenergy.blogspot.co.uk/
http://spiritlibrary.com/kate-spreckley/energy-report-for-may-2012
http://danamrkich.blogspot.com.au/2012/05/may-2012-monthly-visions-truth-and.html
Some readjustment going on in your lives may be just you tuning in to your true purpose, or may be just another delusion that will drop away as things become clearer - which they will
I can say with some certainty, not everyone is going through this easily...

Eat fresh live foodstuffs, drink plenty of fresh filtered water, and use Himalayan rock salt to restore your own body's natural system and immunity
http://kentfreedommovement.ning.com/profiles/blogs/salt-intake-is-a-global-deficiency
http://www.grandpappy.info/wwater.htm
http://www.youtube.com/watch?v=-PCoIJheRKc

A detox may help, too...I'm on my fifth already

Aitch

[sickgut]