Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Thu 23 Nov 2017, 12:54
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
fields.awk database
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 2 of 7 [99 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7 Next
Author Message
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Tue 21 Jun 2016, 10:52    Post subject:  

Or if you prefer it in pdf format, it's attached.
Just remove the fake .zip file in this case.

I didn't include it in the zip file of the previous post as it took the file size to over the 256KB max.
UsingGeanyToConvertCSV.pdf.zip
Description 
zip

 Download 
Filename  UsingGeanyToConvertCSV.pdf.zip 
Filesize  171.95 KB 
Downloaded  84 Time(s) 
Back to top
View user's profile Send private message 
musher0


Joined: 04 Jan 2009
Posts: 11173
Location: Gatineau (Qc), Canada

PostPosted: Tue 21 Jun 2016, 13:58    Post subject:  

Thanks, Pete.
_________________
musher0
~~~~~~~~~~
"Logical entities must not be multiplied beyond necessity." | |
« Il ne faut pas multiplier les entités logiques sans nécessité. » (Ockham)
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Tue 21 Jun 2016, 14:27    Post subject:  

You're welcome musher0

Of course it's also just as easy to use geany to convert from fields.awk back to csv.
Back to top
View user's profile Send private message 
cabbie0

Joined: 23 Jun 2016
Posts: 22
Location: British Columbia, Canada

PostPosted: Thu 23 Jun 2016, 02:02    Post subject: Great Job Guys
Subject description: The author weighs in...
 

Hello Puppy Linux group,

Delighted to see you folks are finding such ingenious ways to make use of fields.awk.
Keep up the good work!

Just a quick correction in that musher0 somehow got my last name mixed up. It's Forsyth, not Stewart (though they're both Scottish lol).

The work on adaptation to barcode reader outputs and Arduino+sensor data is outstanding creativity. Kudos.

And yes, as you've discovered, using awk in subsequent scripts can produce handy analyses of the fields.awk database files. I do this on a regular basis, especially to keep track of running totals in a given group of items. For example if a simple database for musical items exists like so:

@@@DATE::STORE::ITEMS::TOTAL::
07-16-14::Wentworth Music::violin string set (Helicore 4/4 Light)::50.35::
11-14-14::Wentworth Music::Violin fine tuner (W1102)::4.43::
03-12-15::Wentworth Music::Pro-Arte Violin Set 4/4 (nylon core)::43.63::
05-09-15::London Drugs::Sony Memory Stick Pro Duo (4GB)::33.59::
05-15-15::Wentworth Music::violin string set (D'Addario helicore Heavy-tension 4/4)::50.35::
10-09-15::Wentworth Music::replacement "A" string (Precision A 4/4; 51 Medium, solid steel core, chrome wound)::14.51::
10-16-15::Wentworth Music::violin bow (4/4, carbon composite)::124.27::
10-30-12::Wentworth Music::HOHNER Pro Harp G::44.74::

Then I can enter the following command (of course give it a one-word handle) for the total,

awk -F:: '{print $4}' ~/DB_FILES/music_exp.txt | addcol 1
365.87

Note here the "addcol" command is yet another useful short script based on awk:

# ADDS THE NUMBERS IN COLUMN NUMBER SPECIFIED ON COMMAND LINE
# SYNTAX: addcol [number] [filename]

if [ $# -eq 0 ]
then echo USAGE: addcol [number] [filename]
exit 0
fi

awk -v colnum=$1 'BEGIN {t=0} {t=t+$colnum; print t}' $2 | tail -n1

exit 0

No doubt you may find many uses for that as well. I find it serves very well at tax time, so long as your records are tabulated. No more laborious checking columns with a calculator...


And a huge thank you to Pete for creating the FieldsAwk.png icon Smile Awesome!

Ian R. Forsyth
Back to top
View user's profile Send private message 
greengeek


Joined: 20 Jul 2010
Posts: 4719
Location: Republic of Novo Zelande

PostPosted: Thu 23 Jun 2016, 02:49    Post subject:  

Welcome!
I am just getting to grips with fields.awk and have a long way to go...

I enjoyed the youtube video mushero linked to. A good starting point.

I'm hoping to use this to help keep track of tax receipts throughout the year. I think the first hurdle is to decide how many fields to create. The temptation is to create a small number of fields to make it easy to get started - but it may well be better to add more fields than I think I need - in order to allow for future needs.

Or is there a way to add more fields later if necessary? Would that upset prior entries?
.
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Thu 23 Jun 2016, 05:27    Post subject:  

Hello cabbie0 (Ian) and a warm welcome to you.

Firstly many thanks for creating fields.awk, it's the perfect database, simple, fast, efficient, will work on just about any computer and no dependencies.

Glad you liked the icon, feel free to use it as you see fit.

Handy script "addcol", thanks.

Looking forward to more tips, suggestions and ideas from you.
Back to top
View user's profile Send private message 
musher0


Joined: 04 Jan 2009
Posts: 11173
Location: Gatineau (Qc), Canada

PostPosted: Thu 23 Jun 2016, 08:32    Post subject:  

Welcome aboard, Ian.

I'm very glad that you decided to join the forum. Thanks.

Sorry for confusing your last name with someone else's. (No doubt my most
memorable "oops" ever...) Embarassed Again, sorry about that.

I hope you enjoy your stay in "the kennels"!

Bye for now.

_________________
musher0
~~~~~~~~~~
"Logical entities must not be multiplied beyond necessity." | |
« Il ne faut pas multiplier les entités logiques sans nécessité. » (Ockham)
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Thu 23 Jun 2016, 13:35    Post subject:  

greengeek wrote:
.... is there a way to add more fields later if necessary? Would that upset prior entries?
.


Two options:

1) Open fields.awk the select (C) Change Editor, by default Ian has coded in Vim if memory serves me right, change it to nano or keep Vim or use another to your liking.
There are several reasons I chose nano, firstly Vim and I are not good friends, nano was already installed and it's CLI based.
Note that you can't use a GUI editor like geany else when you close it, it confuses fields.awk.

EDIT: musher0 has quite rightly pointed out that leafpad does work.

Once you have set the Editor, then you can edit/add/delete fields whilst still in fields.awk

2) If you are not in fields.awk, then you can edit the DB file in a GUI editor if you prefer.

Don't forget the :: at the end of your newly created field/s.

Perhaps Ian has more ideas.

Last edited by Pete on Thu 23 Jun 2016, 14:27; edited 1 time in total
Back to top
View user's profile Send private message 
musher0


Joined: 04 Jan 2009
Posts: 11173
Location: Gatineau (Qc), Canada

PostPosted: Thu 23 Jun 2016, 13:48    Post subject:  

Euh... about GUI editors: I had good results using fields.awk with leafpad. (FWIW)
_________________
musher0
~~~~~~~~~~
"Logical entities must not be multiplied beyond necessity." | |
« Il ne faut pas multiplier les entités logiques sans nécessité. » (Ockham)
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Thu 23 Jun 2016, 14:25    Post subject:  

Aha, and so it does work.
I tried only geany and completely forgot about leafpad.
Thanks musher0, will edit my post above.
Back to top
View user's profile Send private message 
musher0


Joined: 04 Jan 2009
Posts: 11173
Location: Gatineau (Qc), Canada

PostPosted: Thu 23 Jun 2016, 17:47    Post subject:  

Pete wrote:
Aha, and so it does work.
I tried only geany and completely forgot about leafpad.
Thanks musher0, will edit my post above.
My pleasure, Pete.

Yeah, geany is too cumbersome for such a job.

Back to command-line-interface editors, "joe's own editor" works fine with
fields.awk. On our Puppy forum, the presentation in English is here and the
pet archive is here (at the top of the post, the URL line with "adrive" in it).

I find "joe's own editor" fantastic: it has automatic back-up, can double-up as an
hex editor if you need one, has code-coloring, and can be paired with less --
among many other features. I call it my pseudo-GUI CLI editor!

There's no "line-up at the door" for it, so to speak; still 39 people went to the
trouble of downloading my pet archive of the "joe" editor from adrive.

If you'd like to use "joe" as the editor for fields.awk, 1) install joe from the URL
above ("Gee musher0, how obvious can you be at times!") Wink ; 2) specify
"joe" (without the quotation marks) in fields.awk under the "C" option.

But I'm getting a bit off-topic here. I just thought there was an opportunity to give
this very good "joe" editor a little push! (But nano is great too!)

BFN.

_________________
musher0
~~~~~~~~~~
"Logical entities must not be multiplied beyond necessity." | |
« Il ne faut pas multiplier les entités logiques sans nécessité. » (Ockham)

Last edited by musher0 on Thu 23 Jun 2016, 17:57; edited 1 time in total
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Thu 23 Jun 2016, 17:56    Post subject:  

Hi musher0

I was one of the 39 people that downloaded joe.
I do like it and use it as a hex editor.
Only thing that I have against it is the strange keys combinations which takes some getting used to and probably puts a few people off.
Back to top
View user's profile Send private message 
musher0


Joined: 04 Jan 2009
Posts: 11173
Location: Gatineau (Qc), Canada

PostPosted: Thu 23 Jun 2016, 17:59    Post subject:  

That's true.
The combination: Cntrl-K plus another key indeed takes some getting used to.

_________________
musher0
~~~~~~~~~~
"Logical entities must not be multiplied beyond necessity." | |
« Il ne faut pas multiplier les entités logiques sans nécessité. » (Ockham)
Back to top
View user's profile Send private message 
cabbie0

Joined: 23 Jun 2016
Posts: 22
Location: British Columbia, Canada

PostPosted: Thu 23 Jun 2016, 21:41    Post subject: Some suggestions  

Hi guys.

Regarding greengeek's question about adding new fields _after_ a database has been in use for a while:
Yes, could be done if done carefully by editing the database file itself with the editor of your choice which seems to be a matter of personal preference. I'm hooked on Vim, but realize that many don't care for it. It is very powerful once you get the hang of it but certainly not necessary for fields.awk stuff.
Now, I'd say that deciding carefully in the beginning as to how many fields you want is the better option as it will save much hassle; but should you want to add later I think that adding one or two new fields at the END of the first line would be least disruptive since those new fields would have numbers that would not confuse the earlier entries. If you added a new one somewhere else, then you'd have to add a new " ::" at the appropriate location in all the initial entries to prevent them from lining up the wrong data with the wrong field.

Not to worry musher0 ! I mix up (or rather forget) names all the time. Glad to see it's not me doing it this time he he.

Regarding an earlier concern of greengeek:
" If you try to read some data out of your data file the program asks you if you want to create an output file. If you say yes it will ask you what name to give that file. However - it will not write that file until after you complete the read function and quit back to the home screen. (File is not created till then)."
The reason for this is that is the output from the read function creates the content of the variable $OP, which feeds not only the standard output (to screen) but also the contents of the file to be saved. Prior to the read function happening $OP has no content (null string) and so cannot be used to create a file.

As for the other issue:
" After you create a new datafile you cannot immediately try to add a new record. It is necessary to type "S" first - this gives you an opportunity to select the datafile you want to work with"
This behavior can be altered if you want.
First, I'd like to just mention that originally fields.awk was simply used from the command line and the database file was specified in the launch command. As in: "fields.awk [path_to_datafile]" and the relevant file was selected. This behavior still works (i.e. backward compatible); you can try it. It works because the very first line of the script is still:
DATAFILE=$1; touch ~/tmp ~/output; TMP=~/tmp; OP=~/output
Note the first part that defines DATAFILE as $1 which in this case is the first option entered after the command itself.
The new default, where fields.awk is simply launched empty and all the datafiles are collected in a central repository from which they are selected by the "S" option, results in the variable "DATAFILE" initially being empty and therefore nothing can be done till one is selected.

However, in your special case of creating a new database file from the "N" option, even though it's only done once I suppose you're right that it would be more convenient if it were automatically selected immediately upon creation.
So, if you want to do that, go to the "new_data" function section of the script, near the end of the function where you see the line:
mv -f dbfile.tmp $DB_FOLDER/$NEW_DATAFILE (line 261 or thereabouts...)

Immediately after this line add:
DATAFILE=$DB_FOLDER/$NEW_DATAFILE
Then edit the output line,
echo "Your new database_file should be ready to select."
to read instead:
echo "Your new database_file is created and currently selected."
and either delete or comment out the line:
echo "Current contents of $DB_FOLDER folder: "; echo `ls -m $DB_FOLDER`
because you are no longer selecting from the list of files in the repository.

I tried this; it works. But of course always be careful editing a script as very small syntax issues can cause weird behavior.

That's about all I've got for now Smile
Have fun.
Back to top
View user's profile Send private message 
greengeek


Joined: 20 Jul 2010
Posts: 4719
Location: Republic of Novo Zelande

PostPosted: Fri 24 Jun 2016, 04:06    Post subject: Re: Some suggestions  

cabbie0 wrote:
Now, I'd say that deciding carefully in the beginning as to how many fields you want is the better option as it will save much hassle; but should you want to add later I think that adding one or two new fields at the END of the first line would be least disruptive since those new fields would have numbers that would not confuse the earlier entries.
Thanks. Good idea. I think I will make it a habit to add a couple of extra fields at the end. My last 3 fields will be Comments, Comments2 and Comments3 and I think I should be ok to rename the last two fields later if required.

Quote:
...it will not write that file until after you complete the read function and quit back to the home screen. (File is not created till then)."
The reason for this is that is the output from the read function creates the content of the variable $OP, which feeds not only the standard output (to screen) but also the contents of the file to be saved. Prior to the read function happening $OP has no content (null string) and so cannot be used to create a file.
Thanks for the explanation. Now that I've seen how it works it's easy to use.

Quote:
" After you create a new datafile you cannot immediately try to add a new record. It is necessary to type "S" first - this gives you an opportunity to select the datafile you want to work with"
This behavior can be altered if you want.....
I tried this; it works. But of course always be careful editing a script as very small syntax issues can cause weird behavior.
Thanks very much for the mod - that makes it even more useable. I think it is better for newbies this way - when I first tried it I spent quite a lot of time wondering how to make it work (obviously missed the top line prompt in the terminal) but in fact the programme is quite self-explanatory and I think this change makes it a no-brainer for the first time user. All straightforward.
cheers!
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 2 of 7 [99 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0617s ][ Queries: 12 (0.0062s) ][ GZIP on ]