Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Tue 21 Nov 2017, 10:04
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
fields.awk database
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 3 of 7 [99 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7 Next
Author Message
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Fri 24 Jun 2016, 11:07    Post subject:  

@cabbie0

Thanks Ian, that's a handy modification.
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Fri 24 Jun 2016, 11:50    Post subject:  

Several posts back, I included a one liner to convert a fields.awk DB to html.
It was limited in that in only converted to straight html with no regards for links.

One of my uses of fields.awk is to enter bits of info into a database so I can find them again.
It's a simple database with only 3 fields, Name URL and comments.
Here is an excerpt:

Code:

@@@Name::URL::Comments::
Google::http://www.google.com::search engine::
Puppy::http://www.murga-linux.com/puppy/index.php::Puppy forum index::
Puppy::http://www.murga-linux.com/puppy/viewtopic.php?p=909228::fields.awk thread::
Puppy::http://www.murga-linux.com/puppy/index.php?f=63::Puppt additional pets::
Fields.awk::https://www.youtube.com/embed/HC8Az9W8FV4?rel=0&autoplay=0::fields.awk video::


What I needed was a converter to view this in a web browser so I can click directly on the URLs and go to that web site.
So I came up with a Bash script called fields2html, attached, just remove the fake .zip and don't forget to set permissions.

Now what I do is open fields.awk, select my database described above then do a search/read, saving it to a file, not case sensitive then just pressing enter at the search criteria so that fields.awk will include everything.

Now I run that file generated by fields.awk (search1.txt in /root/.DB) thru fields2html:

Code:

fields2html /root/.DB/search1.txt


Result attached as a screenshot.
Note also that by clicking on a link, it opens in a new tab leaving the original tab still containing the list so that one can open multiple web sites instead of having to click on BACK all time.

Some notes/gotchas:

1) Don't parse the fields.awk DB directly to fields2html as the :: causes the links to become malformed, rather use the file created by fields.awk from a search/read.
Of course if you don't have any URLs in you database, then go for it.

2) The script also gives the user the option to open the newly created html file in the defaultbrowser.
If you are not running Puppy or prefer to open another web browser, modify the last line of the script accordingly.

3) The script creates a html file with the same name (plus .html extension) in the same directory as the input file.

Don't forget to remove the fake .zip in the attached script below.

Lastly, I decided to call the script fields2html and not text2html (which it technically is) as some of you may already have a script/pgm called that.
fields2html.zip
Description 
zip

 Download 
Filename  fields2html.zip 
Filesize  1.48 KB 
Downloaded  69 Time(s) 
ScreenshotF2H.jpg
 Description   
 Filesize   29.94 KB
 Viewed   265 Time(s)

ScreenshotF2H.jpg

Back to top
View user's profile Send private message 
greengeek


Joined: 20 Jul 2010
Posts: 4716
Location: Republic of Novo Zelande

PostPosted: Fri 24 Jun 2016, 16:19    Post subject:  

Pete wrote:
So I came up with a Bash script called fields2html,...
Now what I do is open fields.awk, select my database described above then do a search/read, saving it to a file, not case sensitive then just pressing enter at the search criteria so that fields.awk will include everything.

Now I run that file generated by fields.awk (search1.txt in /root/.DB) thru fields2html:

Code:

fields2html /root/.DB/search1.txt

Wow, thats a handy function! Given that ian says fields.awk can be cli driven I wonder if it would be possible to "pipe" the output of fields.awk direct to fields2html without manual manipulation of the database search?

I can't code to save my life but something like this? -

Code:
fields.awk [path_to_datafile] readall "*" /root/searchresult.txt | fields2html /root/searchresult.txt > links_out.html

or:
Code:
fields.awk [path_to_datafile] readall "*" | fields2html > links_out.html

The only thing missing from my example is accuracy and correct syntax
Smile
.
Back to top
View user's profile Send private message 
musher0


Joined: 04 Jan 2009
Posts: 11167
Location: Gatineau (Qc), Canada

PostPosted: Fri 24 Jun 2016, 17:05    Post subject:  

@Pete: that is indeed a handy addition!
_________________
musher0
~~~~~~~~~~
"Logical entities must not be multiplied beyond necessity." | |
« Il ne faut pas multiplier les entités logiques sans nécessité. » (Ockham)
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Fri 24 Jun 2016, 17:49    Post subject:  

@musher0
Thanks

@greengeek

Is this what you meant?

Version 2 of fields2html attached, remove fake .zip
Just delete the previous version and replace with this one.

One can now convert a fields.awk database directly without using fields,awk or any manipulation.

Code:

fields2html /path/to/database.txt


Of course you can also use the search file created by fields.awk if you wish.
ScreenshotF2H2.jpg
 Description   
 Filesize   22.77 KB
 Viewed   223 Time(s)

ScreenshotF2H2.jpg

fields2html.zip
Description 
zip

 Download 
Filename  fields2html.zip 
Filesize  1.61 KB 
Downloaded  66 Time(s) 
Back to top
View user's profile Send private message 
cabbie0

Joined: 23 Jun 2016
Posts: 22
Location: British Columbia, Canada

PostPosted: Sat 25 Jun 2016, 00:28    Post subject: Links in fields.awk database files  

@ greengeek, @ Pete
You're welcome for the mod.You're right, it's better that way.

Pete, Regarding your fields2html script:
Again, good work.

I was confused at first why you couldn't simply click on the links from directly within your terminal, until I realized your default is different from mine (in Debian, or now Devuan which is systemd free). I get the impression you have a default terminal (aside from xterm) of Urxvt, with the option of adding LXterminal. And I see from testing some link-containing data within them that they do not have that option. LXterminal comes closest in that the links will highlight (underlined) when the mouse is passed over them, but right clicking only gives the options to Copy URL, which you would then have to re-paste into your browser.

However, on my current system the default terminal is Xfce4-terminal which allows right clicking highlighted links for an "Open Link" option. Unfortunately that terminal is tied to the Xfce window manager and so isn't entirely stand-alone. But, there is another one called sakura which is standalone and has the same ability. Is sakura available from your Puppy Linux repository? One source for the tarball is here:
https://launchpad.net/sakura

If so, you can then run fields.awk from within sakura, and any link-containing search outputs displayed in the terminal can be launched directly in your browser with a right-click.

Alternatively, so long as fields2html could be made to create an output file automatically ( i.e. _without_ user-interaction regarding opening browser as it is in the download version), you could add one line at the end of the read_from function, something like:
fields2html ~/Desktop/$SAVEAS.txt > ~/Desktop/$SAVEAS.html

Presumably you'd end up with BOTH $SAVEAS.txt AND $SAVEAS.html in ~/Desktop at one go.

(But I have not tried this....)
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Sat 25 Jun 2016, 04:14    Post subject:  

Hi cabbie0

Yes I use LXTerminal by default, but as you say one can copy the URL but not open a browser directly by clicking it.
Will investigate Sakura.

I like your idea of adding the line fields2html ~/Desktop/$SAVEAS.txt > ~/Desktop/$SAVEAS.html

I don't think it will be a problem having a txt and html version of the same file in ~/Desktop.
The way I will go is make two versions of fields2html.

1) one with no stdout output to be used as above

2) A "stand alone" version (like it is now) with option to open a browser with the newly created html page as I'm finding fields2html quite useful as a general purpose text to html converter.
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Sat 25 Jun 2016, 04:38    Post subject:  

Hi all

For the modification that cabbie0 suggested, where fields.awk calls fields2html directly, two questions:

1) Where should this version of fields2html reside?
/root /usr/sbin or /usr/bin ?
My "vote" is for /usr/sbin, your suggestions are welcome.

2) Any preferences for the name of the new version of fields2html?
Something different to fields2html so as to allow two versions.

Or perhaps I should make just one version of fields2html but if the -q (quiet) option is passed to it, it suppresses output to stdout and no option to open newly created html file straight away.

Your thoughts?
Back to top
View user's profile Send private message 
greengeek


Joined: 20 Jul 2010
Posts: 4716
Location: Republic of Novo Zelande

PostPosted: Sat 25 Jun 2016, 05:36    Post subject:  

Hi Pete, could you just clarify the difference between the two versions please? I'm not sure I followed exactly - but i think from what you said version 1 creates an html file and requires manual opening of the browser and version 2 opens the info direct in a browser is that correct?

Maybe v2 could have a slightly different name like fields2browser?
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Sat 25 Jun 2016, 06:03    Post subject:  

greengeek wrote:
Hi Pete, could you just clarify the difference between the two versions please?


No worries, greengeek.

At first I thought of making two version of fields2html, one that fields.awk calls automatically and another one could use to convert manually.
The reason for the two versions was that fields2html also sends text to stdout in the form of a question asking you if you want to open the html file.

This would cause problems when called directly from fields.awk

However, I then thought of another, better way, have only one version but with the option of telling it to keep quiet (no text to stdout).

So if you call it manually and want the option to open the html file after creation, call it as before:

Code:

fields2html /path/to/database.txt


and when fields.awk calls it:

Code:

fields2html /path/to/searchfile.txt -q


The "-q" at the end is what tells fields2html not to send any text to stdout.

Hope it makes more sense now.

So now when you do a search in fields.awk, it creates a file in both txt and html automatically.
Back to top
View user's profile Send private message 
Pete


Joined: 02 Mar 2014
Posts: 675

PostPosted: Sat 25 Jun 2016, 11:36    Post subject:  

Latest version of fields2html attached (remove fake .zip).
Copy to /usr/sbin and set permissions.
Delete any previous versions you may have.

By placing it in /usr/sbin, it is accessible from anywhere when you call it when using it "manually" and fields.awk will also find it.
It's actually proven to be a handy general purpose text to html converter.

The latest version accepts a -q parameter (must be the second parameter) after the input text file.
This -q option suppresses any output to stdout and will just quietly convert your text file to html without asking you if you want to then open it.

Attached is also the latest modified version of fields.awk (again remove fake .zip) where it automatically calls fields2html to generate a txt and html version of your searches.
Fields.awk places these files in ~/Desktop directory.
fields2html.zip
Description 
zip

 Download 
Filename  fields2html.zip 
Filesize  2.04 KB 
Downloaded  68 Time(s) 
fields.awk.zip
Description 
zip

 Download 
Filename  fields.awk.zip 
Filesize  10.84 KB 
Downloaded  66 Time(s) 
Back to top
View user's profile Send private message 
cabbie0

Joined: 23 Jun 2016
Posts: 22
Location: British Columbia, Canada

PostPosted: Sun 26 Jun 2016, 00:58    Post subject: fields2html script  

Hi,

Congratulations Pete, that html conversion script seems to work fine in the fields.awk mod.
I like the approach of keeping the mods simple by calling the an outside script (such as fields2html or addcol) from within fields.awk, in order to keep the database script itself as true to the original as possible while still adding functionality.
This also seems in keeping with the basic Unix philosophy.

I have a question regarding your suggested placement of the fields2html script.
You recommend /usr/sbin but a regular user shouldn't be able to activate a script in /usr/sbin. Are you running fields.awk as root?
For a regular user I'd place the script in /usr/local/bin. That's what I did and it seemed to work fine.

Cheers
Back to top
View user's profile Send private message 
cabbie0

Joined: 23 Jun 2016
Posts: 22
Location: British Columbia, Canada

PostPosted: Sun 26 Jun 2016, 01:25    Post subject: Differing terminal behavior on fields.awk search outputs  

Hi,

Regarding my previous post indicating that links in a fields.awk search output should be "clickable" in some terminals but not others:

Note the following differences of behavior in 4 different terminals displaying the same fields.awk search output.
Case 1 = xterm
Case 2 = LXterminal
Case 3 = Xfce4-terminal
Case 4 = sakura

The images were taken with a digital camera to include the mouse-click drop down menus, since I don't see any way to include that with normal screen capture. Sorry for poor quality resulting from size reductions.

Hope this helps.
DSC05145.jpg
 Description   Xterm: right-clicking a link in the search output has no effect. (Not much use)
 Filesize   118.51 KB
 Viewed   94 Time(s)

DSC05145.jpg

DSC05149.JPG
 Description   LXterminal: Right-clicking a link in the search output includes an option to copy URL, but no direct opening of the link. (Good but not quite there)
 Filesize   224.17 KB
 Viewed   95 Time(s)

DSC05149.JPG

DSC05148.JPG
 Description   Xfce4-terminal: Right-clicking a link in the search output includes an option to directly open the link. (Yes!)
 Filesize   223.38 KB
 Viewed   94 Time(s)

DSC05148.JPG

DSC05150.JPG
 Description   Sakura: Again, right-clicking a link in the search output includes an option to directly open the link. This time, the terminal is a standalone item. (Yes!)
 Filesize   157.8 KB
 Viewed   98 Time(s)

DSC05150.JPG

Back to top
View user's profile Send private message 
mcewanw

Joined: 16 Aug 2007
Posts: 3194
Location: New Zealand

PostPosted: Sun 26 Jun 2016, 03:21    Post subject: Re: Differing terminal behavior on fields.awk search outputs  

cabbie0 wrote:
Hi,

Regarding my previous post indicating that links in a fields.awk search output should be "clickable" in some terminals but not others:


Hello Ian,

Just trying out your fields.awk database. Very nice to play with and a very useful and easy to customise commandline script.

I am using it in XenialDog, which is a Ubuntu 16.04 Xenial Xerus fully compatible system which has been special crafted by Fredx181 to be very "puppy-like" in use and size but fully apt/dpkg/synaptics capable:

http://www.murga-linux.com/puppy/viewtopic.php?t=106696

XenialDog doesn't have gawk installed by default, but your script seems to work fine so far simply by creating simlink gawk pointing to existing /usr/bin/awk (which turns out to itself actually be a symlink that ends up at /usr/bin/mawk...).

Main news is that XenialDog uses Roxterm by default and, in that terminal, right-clicking on url links in Roxterm allows either direct opening in browser or in copying to pasteboard. Great!
(EDIT: That's a facility provided by the terminal itself of course, rather than internally by fields.awk; any url seen in a Roxterm window, manually typed or created by any program output will offer the click-to-open functionality).

William

EDIT: of course, if using mawk instead of gawk proves problematic, I can always install gawk via apt or synaptics in XenialDog, but so far no problems anyway.

_________________
SomeOfMyWork with links:
dCoreDog
weX, scrox, Precord, Premote, fokSyfEyeR, xhippo-mod, flite_hts_pet

Last edited by mcewanw on Sun 26 Jun 2016, 04:04; edited 3 times in total
Back to top
View user's profile Send private message Visit poster's website 
greengeek


Joined: 20 Jul 2010
Posts: 4716
Location: Republic of Novo Zelande

PostPosted: Sun 26 Jun 2016, 03:40    Post subject:  

Pete wrote:
One can now convert a fields.awk database directly without using fields,awk or any manipulation.

Code:

fields2html /path/to/database.txt

Hi Pete, I just gave this a try and it gave me the correct output (and asked me if i wanted the browser opened) but the urls do not appear as underlined. Do you think that might be something to do with the way my browser is configured? (see pic)

Also - after the browser displayed the output I couldnt tell what to do next with the terminal output so just did a ctrl C and the terminal closed - but so did both instances of Firefox that I had open at the time. (Just thought I'd mention that in case there is a way to only close the PID of the browser window that was started by the terminal rather than the original I already had open)
browser_view.jpg
 Description   
 Filesize   18.95 KB
 Viewed   83 Time(s)

browser_view.jpg

Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 3 of 7 [99 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0677s ][ Queries: 12 (0.0056s) ][ GZIP on ]