Puppy Database

Under development: PCMCIA, wireless, etc.
Message
Author
User avatar
Springer
Posts: 52
Joined: Tue 22 Aug 2006, 16:25
Location: Austin, TX

#41 Post by Springer »

JohnM:

Just to clear up a misconception, NoSQL and its relatives (rdb, /rdb, starbase, etc.) ARE real relational databases. That's the idea - each of their application environments provides relational operators that work out-of-the-box on the tables, which happen to be (usually) TSV text files, sometimes with header lines.

IMHO, every well-read Unix hacker should at least have read this paper to see how Unix (the toolset, more than the OS proper) can be leveraged as an app platform, not just for underlying OS support: The Unix Shell as a Fourth Generation Language.

Stream/Operator databases will never flat outperform "real database engines", but they offer very real advantages, and in today's world, where the penalty for interpreted code is trivial and huge, fast harddisks and RAMdisks are dirt-cheap, the approach makes more sense than ever.

FWIW, here's a (slightly edited) post I made to the NoSQL mailing list a while back, when one of the new guys was wondering how to present NoSQL to his LUG as an alternative to "regular" SQL databases:
Well, Evan Schaffer's paper "The Unix shell as a 4GL" is a great place to start. Mainly, the advantage is not so much the database itself (DBs are just a way to hold and find stuff, after all) - it's the "stream-operator" paradigm that a database like /rdb, Starbase, or NoSQL makes possible.

This approach gives you awesome modularity and leverage, and lets non-DB people write serious DB apps without ever having to know all the arcane crap about how SQL works.

Further, in today's world where Gigabytes of RAM and RAMdisks are everywhere, speed isn't really the concern most people think it is. 4GB is actually a really LARGE database, and I'll stack up a RAMdisk against the latest snazzy FibreChannel SAN array any day! Another nice thing about using a filesystem-based table store is that you don't even need BLOBs: BLOBs are represented as the files they started out as - just drop them into the regular disk-based filesystem! If you like to work, you can keep a pointer to the BLOB's binary file in your tables, but even that's often not necessary if the directory structure and file naming scheme has been thought about very much.

Generally, people who say they need speed in databases need it to try to make up for extremely poor platform and/or application architectures - unfortunately, many of today's web apps fall into that category. Screaming DB speed isn't usually necessary, though - *good* design won't beat on the DB that hard in the first place. (A couple of years ago, I designed a high-performance deeply embedded web application server finely tuned for real-world I/O. Even though it ran on an *8-bit* microcontroller, with only about 3 KILObytes of RAM, it easily outperforms its Pentium-based competitors. That's not BS, it's true - good architecture really does make that much difference, and most modern, popular architectures are just flat *awful* from a performance point-of-view.)

Another reason this is an interesting idea: a number of years ago (literally around the dawn of Linux), RSW (Evan Schaffer's company, which wrote the commercial /rdb database that pretty much originated this whole idea) sold a product called Xrdb - a graphical (X/OpenWindows) interface to stream/operator programming. We bought a copy along with our /rdb license, and even though they kinda ripped us off becasue they never really finished the product, it was heading in a really cool direction - think of Yahoo's new Pipes, but building unix pipelines based on rdb (and other) operators, and you've got the idea. Of course, you could have separate scripts to perform certain data transformations (stored procedures, anyone?) that you could plug into the stream wherever you needed them, for stripping out phone numbers, formatting a report, whatever. Like I said, really cool, and it's about time somebody tried to build something like that again...

Really, I see stream-operator databases like NoSQL as mostly orthogonal to SQL-based RDBMSes, as both are orthogonal to specialized hybrid transactional systems such as Intersystems' M/Cache - they have different strengths, and there are good reasons for choosing each. Sadly, despite great potential, the rdb approach is the one that's still waiting for the world to notice - so go ahead and give a presentation or three...

(BTW, Carlo Strozzi, the author of NoSQL, replyed in enthusiastic agreement. He also recommended the following link on the effectiveness over efficiency approach of using text-based data: http://aplawrence.com/Opinion/textvsbinary.html)

User avatar
WhoDo
Posts: 4428
Joined: Wed 12 Jul 2006, 01:58
Location: Lake Macquarie NSW Australia

#42 Post by WhoDo »

JohnMurga wrote:As far as I am concerned it doesn't compare or compete - If you want a relational database, you use one.
Earlier in this thread I expressed what I thought we needed for Puppy, long and short term. Sqlite3, TkSqlite, your GUI frontend ... all offer a good short term solution for those familiar with SQL or not needing anything more powerful than an address book creator.

OTOH, long term we need an easy-to-use, fully relational database that can compete with 80% or more of what Microsoft Access can do and yet can be used by Window$ refugees who might not be able to use even Access but still need to create medium to large datasets. Something that does what FileMaker Pro does in Window$ and Mac environments. I believe GLOM is the answer for that, but compiling it for Puppy is a major task that is presently beyond my meager capabilities. What it offers is the ability to create SQL tables, forms and reports without knowing a single SQL command.

If someone of your stature and experience were to attempt it, we may stand a chance of success. Komodo stands willing to help. So do I. Maybe we could make this a new Puppy project. True relational database capability that is easy to use is all that stands between many Window$ users and a complete break from the Micro$oft stranglehold. What do you think?
[i]Actions speak louder than words ... and they usually work when words don't![/i]
SIP:whodo@proxy01.sipphone.com; whodo@realsip.com

DavidBell
Posts: 132
Joined: Fri 24 Nov 2006, 21:44

#43 Post by DavidBell »

WhoDo wrote:OTOH, long term we need an easy-to-use, fully relational database that can compete with 80% or more of what Microsoft Access...
I have never found an easy to use GUI based relational database for any OS. They all seem really easy to use until you try and do something not directly supported in the wizards. But I readily admit to being no expert on the subject and have very limited experience.... maybe Filemaker is the holy grail that I haven't tried. At the moment I use Access97 because it works and seems pretty sable, however trying to do some things in it can require a hell of a lot of hoop jumping. It's not intuitive, but in the end it works.

Higher up in this thread, I bagged OOo Base a bit because for me at least it, uuhhmm ... didn't work (and JRE is too big for basic puppy IMO). So the difference is Access is not that great or incredibly usable, it's just that it works.

So as I see it, sqllite engine is (already?) in puppy, it's the UI that's missing. Something that makes tables, queries, and reports just a little bit simpler is all that's needed. It doesn't have to be a complete solution, just a few integrated tools that make life a bit easier.

I'm not sure which of the packages mentioned higher up best suits this, but I think stability is more important than features with databases, because people with ???K records get spooked by even the hint of a failure. If you are targeting windows refugeees, then Access drivers are good, or something that can be used in both OSs and has data that can be manipulated in Access.

My 2c x 10

DB

User avatar
JohnMurga
Site Admin
Posts: 555
Joined: Wed 04 May 2005, 04:26
Location: Far to the east
Contact:

#44 Post by JohnMurga »

Springer wrote:JohnM:

Just to clear up a misconception, NoSQL and its relatives (rdb, /rdb, starbase, etc.) ARE real relational databases. That's the idea - each of their application environments provides relational operators that work out-of-the-box on the tables, which happen to be (usually) TSV text files, sometimes with header lines.
NO MISCONCEPTION my friend ;-)

I did read the docs and I am acutely familiar with your argument ...
I even worked on a few systems using this paradigm in the early 90s.

And ... Incidentally I found the following article very useful too :

http://www.troubleshooters.com/lpm/200704/200704.htm

BUT ...

It's a bit like calling my calculator a "Personal Computer" ...

In a way it is :-)

Just like you may be able to "pretend" noSql is a relational database by using the "sorttable/jointable" commands and the like, but at the end of the day it's missing fundamental relational database concepts like integrity constraints (most basically referential integrity) ...

As an example ... Foreign keys, the commonly accepted definition, is that these are integrity constraints enforcing that the value of the attribute set is drawn from a candidate key in another relation. This one of many things that noSql doesn't bother with, because to enforce you have to control ... And that's exactly what it DOESN'T WANT to do (well, as it happens you can't even define, but anyway). It breaks the relational paradigm, sorry ... The database itself is incapable of representing the relationships, you have to do it with code - The database itself is not relational.

And you can do it ... In code ... And its ugly ...

Furthermore in my tests it was pretty expensive to join data anyway.

To anyone that says noSql is relational I'd say :
  • * Referential integrity is explicitly not supported.
    * Nulls aren't treated uniformly.
    * Views can exist, but only as programs, not as 'virtual tables'.
    * Forget about data integrity, "data independence" and integrity constraints.
    * You have to relate data in code.
I guess you could call it quasi-relational ... :-)

And you can do neat stuff with it ...

I have nothing against noSql, but if you pitch it as something it isn't it'll fall flat on it's face as anything else would under the circumstances.

Cheers
John de Murga

User avatar
WhoDo
Posts: 4428
Joined: Wed 12 Jul 2006, 01:58
Location: Lake Macquarie NSW Australia

#45 Post by WhoDo »

DavidBell wrote:.... maybe Filemaker is the holy grail that I haven't tried.
Oh, it definitely is, and it beats Access 97 hands down, but it isn't available for Linux. I've used it for years and I've had it running under Wine but that's not the same thing.

I don't know how much like FileMaker is GLOM, but it looks interesting. I'd like to see someone with more knowledge than me compile it for Puppy. Then we might have a serious contender. Until then, we'll never know.
[i]Actions speak louder than words ... and they usually work when words don't![/i]
SIP:whodo@proxy01.sipphone.com; whodo@realsip.com

User avatar
JohnMurga
Site Admin
Posts: 555
Joined: Wed 04 May 2005, 04:26
Location: Far to the east
Contact:

#46 Post by JohnMurga »

WhoDo wrote:
JohnMurga wrote:As far as I am concerned it doesn't compare or compete - If you want a relational database, you use one.
Earlier in this thread I expressed what I thought we needed for Puppy, long and short term. Sqlite3, TkSqlite, your GUI frontend ... all offer a good short term solution for those familiar with SQL or not needing anything more powerful than an address book creator.

OTOH, long term we need an easy-to-use, fully relational database that can compete with 80% or more of what Microsoft Access can do and yet can be used by Window$ refugees who might not be able to use even Access but still need to create medium to large datasets. Something that does what FileMaker Pro does in Window$ and Mac environments. I believe GLOM is the answer for that, but compiling it for Puppy is a major task that is presently beyond my meager capabilities. What it offers is the ability to create SQL tables, forms and reports without knowing a single SQL command.

If someone of your stature and experience were to attempt it, we may stand a chance of success. Komodo stands willing to help. So do I. Maybe we could make this a new Puppy project. True relational database capability that is easy to use is all that stands between many Window$ users and a complete break from the Micro$oft stranglehold. What do you think?
Flattery will get you nowhere :-)

GLOM looks pretty amazing, but it's a bit heavy for Puppy (very), but the way Puppy's going you never know ...

My GUI front end is a demo for murgaLua that is constantly evolving ... And more gui like features are on the way. However, it is only ever going to support sqLite and it is a tiny part of a larger project with priorities that come from my main users.

As as far as my involvement with anything other than murgaLua at this point ... The fact remains that I've been homeless for a LONG TIME. I have found a house and will be moving in a couple of weeks, so once I have regular net access and don't have to do things from the hostels I've been staying in you might see a bit of more of me around here.

There are a LOT of things I want to do to Puppy :-)

Cheers
JohnM

User avatar
WhoDo
Posts: 4428
Joined: Wed 12 Jul 2006, 01:58
Location: Lake Macquarie NSW Australia

#47 Post by WhoDo »

JohnMurga wrote:Flattery will get you nowhere :-)
Ar, rats! :P
JohnMurga wrote:GLOM looks pretty amazing, but it's a bit heavy for Puppy (very), but the way Puppy's going you never know ...
Yep. That PostgreSQL backend is the killer. Something similar for sqLite would be nice though .... (hint, hint, hint) :wink:
JohnMurga wrote:My GUI front end is a demo for murgaLua that is constantly evolving ... And more gui like features are on the way. However, it is only ever going to support sqLite and it is a tiny part of a larger project with priorities that come from my main users.
Well it wouldn't take much to bring up to a similar standard to TksqLite, would it? At this point I'm for anything that offers an excuse for murgaLua to find its way into Puppy. That would mean we can have the u beaut Control Panel that dvw86 put together some time back:
http://www.murga-linux.com/puppy/viewto ... 53&t=15931
JohnMurga wrote:As as far as my involvement with anything other than murgaLua at this point ... The fact remains that I've been homeless for a LONG TIME. I have found a house and will be moving in a couple of weeks, so once I have regular net access and don't have to do things from the hostels I've been staying in you might see a bit of more of me around here.

There are a LOT of things I want to do to Puppy :-)
Ooohh! Good news! I'm really glad you've found a place to call home. Are you sure I can't sweet talk you into writing a complete GUI frontend for sqLite that a newbie could use? :wink: :roll:
[i]Actions speak louder than words ... and they usually work when words don't![/i]
SIP:whodo@proxy01.sipphone.com; whodo@realsip.com

User avatar
JohnMurga
Site Admin
Posts: 555
Joined: Wed 04 May 2005, 04:26
Location: Far to the east
Contact:

#48 Post by JohnMurga »

WhoDo wrote:
JohnMurga wrote:GLOM looks pretty amazing, but it's a bit heavy for Puppy (very), but the way Puppy's going you never know ...
Yep. That PostgreSQL backend is the killer. Something similar for sqLite would be nice though .... (hint, hint, hint) :wink:
And Python, otherwise you "could" try to replace the database backend.
WhoDo wrote:
JohnMurga wrote:My GUI front end is a demo for murgaLua that is constantly evolving ... And more gui like features are on the way. However, it is only ever going to support sqLite and it is a tiny part of a larger project with priorities that come from my main users.
Well it wouldn't take much to bring up to a similar standard to TksqLite, would it? At this point I'm for anything that offers an excuse for murgaLua to find its way into Puppy. That would mean we can have the u beaut Control Panel that dvw86 put together some time back:
http://www.murga-linux.com/puppy/viewto ... 53&t=15931
It has surprised me that murgaLua is a core part of DSL but completely unused in Puppy, but it doesn't bother me - Once I hit my 1.0 functionality I'll concentrate on apps, and them maybe someone here will be interested.

As for TksqLite ... I am the wrong person as I have been biased against anything TCL based since before Puppy (as I had to some professional work with it some time back and didn't enjoy it).
WhoDo wrote:
JohnMurga wrote:As as far as my involvement with anything other than murgaLua at this point ... The fact remains that I've been homeless for a LONG TIME. I have found a house and will be moving in a couple of weeks, so once I have regular net access and don't have to do things from the hostels I've been staying in you might see a bit of more of me around here.

There are a LOT of things I want to do to Puppy :-)
Ooohh! Good news! I'm really glad you've found a place to call home. Are you sure I can't sweet talk you into writing a complete GUI frontend for sqLite that a newbie could use? :wink: :roll:
That's something that will probably happen at some point, just don't expect it too soon as it is a small priority in a big world ;-)

Cheers
JohnM

User avatar
WhoDo
Posts: 4428
Joined: Wed 12 Jul 2006, 01:58
Location: Lake Macquarie NSW Australia

#49 Post by WhoDo »

JohnMurga wrote:It has surprised me that murgaLua is a core part of DSL but completely unused in Puppy, but it doesn't bother me - Once I hit my 1.0 functionality I'll concentrate on apps, and them maybe someone here will be interested.
I already am. How far away is that 1.0 functionality? We have guys here that are itching to get their teeth into programming applications for Puppy, but the breadth of development environments available seems to put off all but the most hardy.

Once I saw what dvw86 was able to quickly achieve with murgaLua, I was sold. It is only a matter of enthusing the "market". If what was done was designed to be compatible with IceWM, then I could introduce it through EZpup without impacting what happens in the mainstream version.

Frankly, I wish I knew more of programming ... especially in murgaLua. I'd have a lash at it myself. Unfortunately, my limited past experience with C and Basic barely qualifies me to write Hello World these days.

Keep plugging away John. Anything good is worth waiting for... just ask my wife! :wink:
[i]Actions speak louder than words ... and they usually work when words don't![/i]
SIP:whodo@proxy01.sipphone.com; whodo@realsip.com

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#50 Post by BarryK »

At this point I'm for anything that offers an excuse for murgaLua to find its way into Puppy.
MurgaLua is in Puppy, but as John says, it's not actually being used for anything ...yet. Looking at it from a fairly tough admin point of view, it would have to ultimately justify its inclusion, like some irresistable application that needs it.

raffy
Posts: 4798
Joined: Wed 25 May 2005, 12:20
Location: Manila

one application

#51 Post by raffy »

If we are looking at educating kids in cutting-edge programming, we don't even have to wait for version 1.0.

An example application or two that one can easily find in the Puppy menu and possibly modify at will should get things started. We can even call it "kiddie programming (KP)". (Then I can get my hands on it when no one is looking. :P )

How about a "create your Puppy" game? (create your unique Puppy from different colors and shapes)

andrei
Posts: 80
Joined: Wed 31 May 2006, 23:56

#52 Post by andrei »

The fonts in TkSqlite are way too small, very hard to read. I solved this problem by editing the configuration file:

/usr/lib/tile0.7.8/fonts.tcl

Now it looks much better.

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#53 Post by Flash »

How did you edit it? Inquiring minds want to know. :lol:

andrei
Posts: 80
Joined: Wed 31 May 2006, 23:56

#54 Post by andrei »

See the attached file. Changes are in the x11 section.

Notice that I use the Courier New font which I downloaded from the Internet. I understand that it is legal to use Courier New because it was made available by Microsoft at some point. But I am not a lawyer.
Attachments
fonts.tar
modified /usr/lib/tile0.7.8/fonts.tcl
(10 KiB) Downloaded 764 times

Igo AtM
Posts: 25
Joined: Thu 16 Nov 2006, 14:36
Location: UK

Working SQLiteDBMS

#55 Post by Igo AtM »

Have just uploaded SQLiteDBMS it is a puppy hacked version and was originally put together as a low resource solution for sqlite3 access and motionMon or litemon with the sqlite db back end. I had seen this topic previously so I am posting a link to those that wish to try it. It is far from a final package but is working and I would appreciate feedback as I am currently re-writing the client. I felt it may also contribute to the ongoing discussion here link is
http://www.murga-linux.com/puppy/viewto ... 180#159180

Yet another interesting sqlite db management option I am uploading is a Gecko based sqlite manager which is very good at the pure db access level the problem with it is it requires seamonkey with the embedded sqlite engine so version 1.1 and above but it adds more fuel to the debate. It is not a pet as it is a native seamonkey extension. You should just be able to upload it unpack it and drag n drop it onto the browser and it should auto install md5 is below
6027224839339bf3d3a34084c11c3d34 sqlite_manager-0.2.8.3.tar.gz
You will need to unpack it as for security can not upload xpi to the forum
Attachments
sqlite_manager-0.2.8.3.tar.gz
archive of sqlite manager for seamonkey (ver 1.1 and above )
(62.74 KiB) Downloaded 758 times

User avatar
Lobster
Official Crustacean
Posts: 15522
Joined: Wed 04 May 2005, 06:06
Location: Paradox Realm
Contact:

#56 Post by Lobster »

http://www.knoda.org/

it uses kde and python (too big for Puppy)
but may suit some . . .

Anyone use it?
Last edited by Lobster on Fri 10 Apr 2009, 11:29, edited 1 time in total.
Puppy Raspup 8.2Final 8)
Puppy Links Page http://www.smokey01.com/bruceb/puppy.html :D

User avatar
Lobster
Official Crustacean
Posts: 15522
Joined: Wed 04 May 2005, 06:06
Location: Paradox Realm
Contact:

#57 Post by Lobster »

This is very nice
but a database online?
What happens when they sell to Microsoft, Google, Yahoo, NSA etc?
http://www.blist.com/index.html
Puppy Raspup 8.2Final 8)
Puppy Links Page http://www.smokey01.com/bruceb/puppy.html :D

tlchost
Posts: 2057
Joined: Sun 05 Aug 2007, 23:26
Location: Baltimore, Maryland USA
Contact:

#58 Post by tlchost »

Lobster wrote:This is very nice
but a database online?
What happens when they sell to Microsoft, Google, Yahoo, NSA etc?
http://www.blist.com/index.html
Hopefully there will be an alternative made for Puppy by then. The signup process is simple and you can easily import a .csv file.

Took a quick look at the support forum....there are not many entries, and the project manager appears to be very active answering user questions.

Apparently one can share a database with others on a read-only basis via email.

muggins
Posts: 6724
Joined: Fri 20 Jan 2006, 10:44
Location: hobart

#59 Post by muggins »

There hasn't been much activity on this front but, even though I've no use for a database, I thought I'd post this link to various sqlite frontends:

http://www.sqlite.org/cvstrac/wiki?p=ManagementTools

nic2109
Posts: 405
Joined: Mon 01 Jan 2007, 20:24
Location: Hayslope, near Middlemarch, Midlands, England

#60 Post by nic2109 »

Another approach to this issue would be to use the XAMPP package (which is huge) and make it an .sfs file. This would add everything to make Puppy into a Web server running sqlite apache perl and php.

You will still need to add a GUI and/or Report writer, which is where Muggins' link will come into its own.
[color=darkblue][b][size=150]Nick[/size][/b][/color]

Post Reply