Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Mon 20 Nov 2017, 19:19
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
fields.awk database
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 6 of 7 [99 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7 Next
Author Message
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Mon 27 Jun 2016, 18:59    Post subject:  

So now we know that fields.awk will also work under:

- MAC (OSX)
- BSD
- Android (with Term emulator)
- Windows (only tried 8.1 but no reason to believe that it won't work under 7 and XP).

Now only the Raspberry Pie left.

Perhaps others too like the Banana Pie and Beagle Board but I know nothing about them.
Back to top
View user's profile Send private message 
cabbie0

Joined: 23 Jun 2016
Posts: 22
Location: British Columbia, Canada

PostPosted: Mon 27 Jun 2016, 23:28    Post subject: Cross-platform fields.awk  

@Pete

I never thought I'd see fields.awk running on a Windows machine.
Wonders never cease.
It makes me curious how well/poorly it might be received by that new audience.
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Tue 28 Jun 2016, 04:43    Post subject: Re: Cross-platform fields.awk  

cabbie0 wrote:
@Pete

I never thought I'd see fields.awk running on a Windows machine.
Wonders never cease.


I have to admit that seeing the windows logo and things like /usr/sbin and awk in the same screenshot is a bit strange. Laughing

Quote:

It makes me curious how well/poorly it might be received by that new audience.


Good question.
My view is that Windows users may be even more reluctant than Linux ones when it comes to using the CLI.
Back to top
View user's profile Send private message 
cabbie0

Joined: 23 Jun 2016
Posts: 22
Location: British Columbia, Canada

PostPosted: Wed 29 Jun 2016, 00:23    Post subject: Search Output processing  

Quote:

My view is that Windows users may be even more reluctant than Linux ones when it comes to using the CLI.


@Pete
That's my gut feeling as well. Still it's interesting to see how adaptable it is to various versions of the bash shell environment.

@all
greengeek earlier brought up the issue of printing out a read search file containing ALL the records of a given database.
And I answered that hitting the ENTER key at the search prompt would accomplish this.
That's true, and you'll get the output in fields.awk's normal record-by-record display, including all fields in all records.

But what now if you want to create a list of just two or three fields of each record, without tedious editing?

For example, Suppose you have made a full printout of a database with field structure
FIELD1 || FIELD2 || FIELD3 || FIELD4 || FIELD5 || FIELD6 ||

and fields.awk creates a full-record printout of “n” number of records displaying all the fields, as usual..
You've named your output file fullrecs.txt.

But you want to produce an output with only a selected number of fields, say FIELD2 and FIELD6.
You can process your output file to do so with a single command using both egrep and awk:

egrep 'FIELD2|FIELD6' fullrecs.txt | awk '{gsub("FIELD2", "\nFIELD2"); print}' > selections.txt


Note: the egrep part selects for the two selected fields only.
You can include more fields in a multiple selective processing. (enclose the list in single quotes, separating the list items by the "pipe" symbol [|] )
The awk part adds a newline in front of the first field in each group, for easier readability.

Real world example:

database file = photobase.txt
field structure = FILENAME || SUBJECT || FILM+SPECS || DATE+LOCATION ||
Read search text = “Cassiar”
Output file = Cassiar_pics.txt (5 records):

==> Matching record 1 :
FILENAME = DSC01490
SUBJECT = Hwy 37 S of Kinaskan Lake (Cassiar Hwy)
FILM+SPECS = Kodacolor Gold ASA100
DATE+LOCATION = circa 1990
*******************************************
==> Matching record 2 :
FILENAME = DSC02243
SUBJECT = Boggy Forest with Mountain Backdrop
FILM+SPECS = Kodacolor Gold ASA100
DATE+LOCATION = Cassiar Hwy, northern BC; circa 1990
*******************************************
==> Matching record 3 :
FILENAME = DSC02245
SUBJECT = Lodgepole with pollen cones
FILM+SPECS = Kodacolor Gold ASA100
DATE+LOCATION = Along Cassiar Hwy, BC; circa 1990
*******************************************
==> Matching record 4 :
FILENAME = DSC02280
SUBJECT = Hwy 37 South of Kinaskan Lake
FILM+SPECS = Kodacolor Gold ASA100
DATE+LOCATION = Cassiar Highway, NW BC; circa 1990
*******************************************
==> Matching record 5 :
FILENAME = DSC02433
SUBJECT = Cassiar Highway
FILM+SPECS = Kodacolor Gold ASA100
DATE+LOCATION = Cassiar Hwy, NW BC; circa 1990

Now I want a list of only the FILENAME and DATE+LOCATION fields, and I want a blank line between each pair.

Command:

egrep 'FILENAME|DATE\+LOCATION' Cassiar_pics.txt | awk '{gsub("FILENAME", "\nFILENAME"); print}'
[Note the \ before the + symbol, as it must be escaped]

Output:

FILENAME = DSC01490
DATE+LOCATION = circa 1990

FILENAME = DSC02243
DATE+LOCATION = Cassiar Hwy, northern BC; circa 1990

FILENAME = DSC02245
DATE+LOCATION = Along Cassiar Hwy, BC; circa 1990

FILENAME = DSC02280
DATE+LOCATION = Cassiar Highway, NW BC; circa 1990

FILENAME = DSC02433
DATE+LOCATION = Cassiar Hwy, NW BC; circa 1990


So that's it. This is a bit of a revelation to me because I have never used grep to select multiple targets as is being done here.
Note that regular grep won't do the trick; it must be the "extended grep" or egrep.

Cheers,
ian
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Thu 30 Jun 2016, 17:01    Post subject:  

Hi cabbie0

That is real handy, gonna give it a try.
Thanks for sharing.
Back to top
View user's profile Send private message 
Pelo

Joined: 10 Sep 2011
Posts: 12591
Location: Mer méditerrannée (1 kms°)

PostPosted: Sat 02 Jul 2016, 00:04    Post subject: Databases ... for pleasure...  

"My view is that Windows users may be even more reluctant than Linux ones when it comes to using the CLI."
First CLI needs to speak english, what can eject 95% of people having a computer.

_________________
Passenger Pelo ! don't ask him to repair the aircraft. Don't use him as a demining dog .... pleeease.
Back to top
View user's profile Send private message Yahoo Messenger 
cabbie0

Joined: 23 Jun 2016
Posts: 22
Location: British Columbia, Canada

PostPosted: Sat 02 Jul 2016, 01:40    Post subject: Scripting LANGUAGE  

Pelo wrote:
Quote:
First CLI needs to speak english, what can eject 95% of people having a computer.

I hadn't thought of that, very true.

@musher0
BONNE FÊTE DU CANADA!
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Sat 02 Jul 2016, 06:09    Post subject:  

@Pelo

I do agree with you that the CLI can be rather cryptic at times.
One way to make it more "English" would be to use Aliases.

In general I don't like aliases as it makes your script less portable but
if it helps in the beginning then go for it.
You can always still use the original command as well.

You could even make it more "French" if you like. Smile

For example if you type this:

alias imprimer='echo'

Then you can use imprimer instead of echo, like so:

imprimer "Bonjour Monsieur Pelo"

Just don't use reserved words and to make the aliases persistent across reboots, place them all in ~/.bashrc or ~/.bash_aliases depending on your setup.
Back to top
View user's profile Send private message 
Pelo

Joined: 10 Sep 2011
Posts: 12591
Location: Mer méditerrannée (1 kms°)

PostPosted: Sat 02 Jul 2016, 14:52    Post subject: If English speaking people want to use a Database;  

it was just a remark. If English speaking people want to use a Database; it's a good idea.
As i Told somewhere, Puppy package manager is a database (data are stored in hidden .packages under root. You can use these data to test fields.awk.
separator | can easily be replaced by comma or semicolon

_________________
Passenger Pelo ! don't ask him to repair the aircraft. Don't use him as a demining dog .... pleeease.
Back to top
View user's profile Send private message Yahoo Messenger 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Sat 02 Jul 2016, 15:10    Post subject:  

@Pelo

OK, understood.
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Sat 02 Jul 2016, 16:19    Post subject:  

@cabbie0

I got my Android toy and downloaded and installed ZShaolin.
Works great and I like the feature that it has 4 terminals, but how does one enter
CNTL+C?
Sometimes it gets stuck and I need to terminate a script.
Tried volume down + c but no go.
Any ideas?

T.I.A.
Back to top
View user's profile Send private message 
cabbie0

Joined: 23 Jun 2016
Posts: 22
Location: British Columbia, Canada

PostPosted: Sun 03 Jul 2016, 03:29    Post subject: Zshaolin Special Keys Mapping  

@Pete

Hi,
Glad to hear you've managed to get Zshaolin installed on an Android device.
I use the Volume_Up button for CTRL (and Volume_Down for ESC).
But that is because I specifically mapped them to those keys.
In the startup screen for Zshaolin, you should see a 6-part menu:
[Start ZShaolin] [Install System] [Shutdown] [Keyboard] [Options] [Website]
Select [Options] (I know, you'd think it would be Keyboard....)
At the bottom of [Options], see [SPECIAL KEYS MAPPING]
There is has special options for both the CTRL and ESC keys you can select.
In each case the opions (on my version) are: Jog ball, @key, Left Alt, Right Alt, Vol Up, Vol Down, Camera Key, or None.

Hope that helps.
Back to top
View user's profile Send private message 
cabbie0

Joined: 23 Jun 2016
Posts: 22
Location: British Columbia, Canada

PostPosted: Sun 03 Jul 2016, 04:33    Post subject: Read Search Output file processing, continued.  

@all

Follow up to previous post on filtering "Read Search Output" files for displaying 2 or 3 selected Fields only.

Although the "egrep" command sequence works, it is a bit too complex to be practical
unless you could make a "one-liner" script which takes your selected fields in the form of command-line arguments and inserts them into the "egrep" sequence, so you don't have to remember the syntax every time.

I tried this, but found that transferring command line arguments as variables into such a script is VERY problematic.
The syntax section " '[SELECTION1]|[SELECTION2]' "
is the sticking point as the single quotes on either side render the variables ($SELECT1 etc) as meaningless text and if you escape the single quote with "\" then the syntax itself is rendered inoperable. -- "Catch22"

So instead I decided to use AWK for both sections of the script; that is, the the multiple-argument section and the blank line insertion section.
Two separate "one-liner" filter scripts called OPselect2 and OPselect3 for filtering out 2 or 3 FIELDS respectively.
The active code for each as follows:

Note each code string is two separate awk scripts; the output from the first (highlighted red) being piped as input to the second. Consequently the entire code string must be on a single line and not wrapped into two separate lines.
The first parts (in red) select only lines containing the FIELD names you enter on the command line.
The second part (in black) is the same for each script since it simply inserts a blank line in front of the first selection, for easier readability.
The command sequence is simple to remember:
OPselect2 [Field_selection1] [Field_selection2] [filename] (where filename is your output file generated by fields.awk)

EXAMPLE: Let's say we have a short output file from a database of tree types. Filename=treesout.txt
==> Matching record 1 :
COMMON NAME = Maple
LATIN NAME = Acer
FOLIAGE TYPE = deciduous
*******************************************

==> Matching record 2 :
COMMON NAME = Oak
LATIN NAME = Quercus
FOLIAGE TYPE = deciduous
*******************************************

==> Matching record 3 :
COMMON NAME = Pine
LATIN NAME = Pinus
FOLIAGE TYPE = evergreen
*******************************************

==> Matching record 4 :
COMMON NAME = Willow
LATIN NAME = Salix
FOLIAGE TYPE = deciduous
*******************************************

==> Matching record 5 :
COMMON NAME = Yew
LATIN NAME = Taxis
FOLIAGE TYPE = evergreen
*******************************************

This is a special case where the FIELD names are actually two separate words. The command line arguments can only be one-word, but the script only needs a recognizable tag so here we can just enter the first word from each FIELD as arguments:
OPselect2 COMMON LATIN treesout.txt

and the output should be:

COMMON NAME = Maple
LATIN NAME = Acer

COMMON NAME = Oak
LATIN NAME = Quercus

COMMON NAME = Pine
LATIN NAME = Pinus

COMMON NAME = Willow
LATIN NAME = Salix

COMMON NAME = Yew
LATIN NAME = Taxis

------------------------------------
Attaching a copy of each filter script for those who would like to try them.
Note these are FAKE zip files. Just rename them OPselect2 and Opselect3.

cheers
Opselect3.zip
Description  Filter script for selecting THREE fields from a fields.awk Read/Search output file.
Syntax = OPselect3 [Field_selection1] [Field_selection2] [Field_selection3] [filename]
zip

 Download 
Filename  Opselect3.zip 
Filesize  758 Bytes 
Downloaded  58 Time(s) 
Opselect2.zip
Description  Filter script for selecting TWO fields from a fields.awk Read/Search output file.
Syntax = OPselect2 [Field_selection1] [Field_selection2] [filename]
zip

 Download 
Filename  Opselect2.zip 
Filesize  664 Bytes 
Downloaded  61 Time(s) 
OPselect_code.jpg
 Description   One-liner awk scripts for filtering Read-Search output files.
 Filesize   72.63 KB
 Viewed   138 Time(s)

OPselect_code.jpg

Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Sun 03 Jul 2016, 05:12    Post subject: Re: Zshaolin Special Keys Mapping  

cabbie0 wrote:
@Pete
....
There is has special options for both the CTRL and ESC keys you can select.
In each case the opions (on my version) are: Jog ball, @key, Left Alt, Right Alt, Vol Up, Vol Down, Camera Key, or None.

Hope that helps.


Perfect, thank you.
I thought that the vol up/dn + letter key combo was an automatic Android thing.
I'm such an Android noob. Laughing

Last edited by Pete on Sun 03 Jul 2016, 05:43; edited 1 time in total
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Sun 03 Jul 2016, 05:33    Post subject:  

@cabbie0

Nice addition those two scripts.
A thought, since the syntax after the pipe command is identical for both one-liners, perhaps including a bit of code to detect how many arguments the user supplied and the script would automatically branch to either 2 or 3 fields.
The number of arguments available in $#.

Another idea, I'm thinking of creating a sort of launch script from where one could start fields.awk or addcols or your latest addition.
So this launch script could be a main menu with options something like:

(S) Start fields.awk
(A) Add columns
(E) Extended search - 2 or 3 fields
:
:
:
:
(X) Exit

This way we can add new functions/one-liners easily.
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 6 of 7 [99 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0636s ][ Queries: 12 (0.0073s) ][ GZIP on ]