Looking for a simple, fast routine to display text, images

For discussions about programming, programming questions/advice, and projects that don't really have anything to do with Puppy.
Message
Author
User avatar
linuph
Posts: 128
Joined: Mon 04 Jun 2012, 02:29
Location: Philippines

Looking for a simple, fast routine to display text, images

#1 Post by linuph »

Attached a snapshot of a car GPS program that I'm working on. It runs on an old laptop with Dpup Exprimo 5.X.3.6 installed.

The program is a Bash (sh) script that makes extensive use of awk to read and process GPS text strings. Update of most of the white text and the GPS quality indicators (mode, sats, pdop) are updated synchronously with the GPS data, that is, every second. The system indicators (memory, disk, load) are updated every couple of minutes or so.
Top left is a live video from a webcam. All of it is on top of a static background picture. Not shown here are messages, e.g. lost GPS signal, that are shown on top of the video.
Last but not least: a video screen grabber runs in the background and saves everything live to disk.

The program is not yet operational: I still have to do a lot of additions, corrections and refinements but the principle works.

To display all dynamic text and indicators I use the dzen2 utility (see https://sites.google.com/site/gotmor/dzen). Several dzen2 instances run continuously in the background, called from awk:

Code: Select all

awk -F","  '{
        if ( /\$GPGGA/) program stuff
            print stuff "dzen2 ........."
        else if (/\$GPRMC/) other program stuff
            print other stuff | "dzen2 ........."
            print other stuff | "dzen2 ........"
        else more program stuff
            print more program stuff | "dzen2........."
}' GPS_data
I use awk for a couple of reasons, the main ones being that it can do trigonometry and math (cos, sin, atan, sqrt to name a few) to calculate a.o. distances and bearings between geocoordinates, it is fast and small.

A complication however is that awk runs in it's own shell, making passing of variables to the bash shell and back rather 'awk'ward. It becomes more complicated because dzen2 requires a lot of arguments, like text, colors, font, geometry etc. leading to a lot of work and lots of mistakes, corrections when optimizing the layout of the display. And then, I don't need all the dzen2 options and I do not use the menu option. Of course, dzen2 was not designed for this (it's a taskbar/tray/menu application) but it is the only usable application that I could find.

Now to my question: can anyone advise on a small, fast program that can be called from awk and prints text and images as per the attached image? I guess something in C, but I don't know how to do that (I had quite some trouble already with the little C-routine that provides the background image....).

What I need to handle:
- text
- text fg color
- font/fontsize
- images
- screen position x,y

Optional:
- text bg color
- blinking text
- .....

Stuff should stay on display until refreshed or killed. Small footprint is needed because several instances have to run at the same time

Some notes:

I can't use the usual gtk kind solutions . Reason is that the car display is only 800x600 (no touch) and of limited resolution. A rather large font is used (DejaVu Sans Bold 24) for clear visibility when driving. There isn't room for frames, decorations, dialogs and the like.

It is surprising that awk can do all that processing in almost no time on my old P3 1GHz 512 Mb laptop. Also thanks to Puppy, of course.

dzen2 is a great little program. It is flexible and general purpose. Each instance requires only about 1.1Mb. I have 10 instances running in my program.

I look forward to ideas.
Attachments
image.jpg
(69.57 KiB) Downloaded 571 times

seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

#2 Post by seaside »

linuph.

This looks like an interesting project.

I wasn't really sure what you're after here.

A dedicated screen divided as shown or a single program that would look like your screen image.

If it's a divided screen, then as you know, there are many tiled window managers that you could use with conky or dzen. On the other hand, if you interested in a single program, that would be much more involved and in that case you might look at zoneminder here http://www.zoneminder.com/ . You many be able to adapt that to your needs.

Cheers,
s

User avatar
linuph
Posts: 128
Joined: Mon 04 Jun 2012, 02:29
Location: Philippines

#3 Post by linuph »

Seaside, thanks.

The image you see is dedicated and fully "handmade". There's no window manager involved. I did look into a tiled wm (like i3, I posted about that) but I found that it doesn't fit to my purpose.

There's a substantial difference in what my program and applications like Conky and Xmobar do:

-
Conky runs by it's own, backgrounded, picks up system data and displays them. What is displayed is initiated by Conky.

-
My program calls dzen2 to display data the very moment GPS data is available and processed, so everything is initiated by my program.

Theoretically I could have my program write processed GPS data to a file or memory and then have it picked up by Conky, however, that would lead to delays and, probably, other issues.

Here is a snippet of my code:

Code: Select all

#!/bin/sh

awk -F"," -v dlati=$dlat -v dloni=$dlon -v hlati=$hlat -v hloni=$hlon '{
if ( $1 == "$GPGGA" && $7 == 0 ) {
	############################################
	print "No valid GPS data. Wait or press 'Esc' to quit." | \
	"dzen2 -fn \"-*-DejaVu Sans-*-r-*-*-18-*-*-*-*-*\" \
	-fg red -bg black -h 18 -ta c -x 0 -y 342 -w 480" }
	############################################
else if ( $1 == "$GPGGA" && $7 == 1 ) {
	#get GPS time
	gpshrs=substr($2,0,2)
	gpsmin=substr($2,3,2)
	gpssec=substr($2,5,2)
	if ( gpshrs > 15 ) gpshrs=0
		localtime=sprintf("%03s%03s%02s",gpshrs":", gpsmin":", gpssec)
		print localtime | \
		###############################################
		"dzen2 -fn \"-*-DejaVu Sans-*-r-*-*-22-*-*-*-*-*\" \
		-bg black -fg white -h 22 -ta r -x 600 -y 135 -w 180"
###############################################
	#convert current coordinates from decimal
	#into degrees, minutes, seconds and print
	clatd=int($3/100)
	clatm=int($3-clatd*100)
	clats=int(($3-clatd*100-clatm)*60)
	clatp=sprintf("%04s%03s%03s%1s", clatd"°",clatm"ʹ",clats"ʺ",$4)
	clond=int($5/100)
	clonm=int($5-clond*100)
	clons=int(($5-clond*100-clonm)*60)
	clonp=sprintf("%04s%03s%03s%1s", clond"°",clonm"ʹ",clons"ʺ",$6)			
	#get some more GPS data
	elev=int($10);sats=$8
	############################################
	if ( sats < 2 ) {
		print clatp" "clonp | \
		"dzen2 -fn \"-*-DejaVu Sans-*-r-*-*-22-*-*-*-*-*\" \
		-fg red -bg black -h 22 -ta l -x 5 -y 410 -w 345"
		print "████" | \
		"dzen2 -fn \"-*-DejaVu Sans-*-r-*-*-18-*-*-*-*-*\" \
		-fg red -bg black -h 18 -ta l -x 609 -y 200 -w 100"
		print elev"m" | \
		"dzen2 -fn \"-*-DejaVu Sans-*-r-*-*-22-*-*-*-*-*\" \
		-fg red -bg black -h 22 -ta l -x 435 -y 410 -w 125" }
	else if ( sats == 2 ) {
		print clatp" "clonp | \
		"dzen2 -fn \"-*-DejaVu Sans-*-r-*-*-22-*-*-*-*-*\" \
		-fg yellow -bg black -h 22 -ta l -x 5 -y 410 -w 345"
		print "████" | \
		"dzen2 -fn \"-*-DejaVu Sans-*-r-*-*-18-*-*-*-*-*\" \
		-fg yellow -bg black -h 18 -ta l -x 609 -y 200 -w 100"
		print elev"m" | \
		"dzen2 -fn \"-*-DejaVu Sans-*-r-*-*-22-*-*-*-*-*\" \
		-fg yellow -bg black -h 22 -ta l -x 435 -y 410 -w 125" }
	else {
		print clatp" "clonp | \
		"dzen2 -fn \"-*-DejaVu Sans-*-r-*-*-22-*-*-*-*-*\" \
		-fg white -bg black -h 22 -ta l -x 5 -y 410 -w 345"
		print "████" | \
		"dzen2 -fn \"-*-DejaVu Sans-*-r-*-*-18-*-*-*-*-*\" \
		-fg green -bg black -h 18 -ta l -x 609 -y 200 -w 100"
		print elev"m" | \
		"dzen2 -fn \"-*-DejaVu Sans-*-r-*-*-22-*-*-*-*-*\" \
		-fg white -bg black -h 22 -ta l -x 435 -y 410 -w 125" }
###############################################
			
} }' /dev/rfcomm0
Between the long '####'s displaying of data takes place.

Maybe it explains better what I mean: I 'just' need a simple, fast and small routine to display text in position x,y with definable (large) font and color. Icons and transparency are nice options.

seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

#4 Post by seaside »

linuph wrote: Maybe it explains better what I mean: I 'just' need a simple, fast and small routine to display text in position x,y with definable (large) font and color. Icons and transparency are nice options.
linuph,

Interesting, it would seem that dzen is exactly what you want.

I guess gtk programs using svg files that you've experimented with are not up to the task? It seems that a one second refresh would be ok for even gtkdialog.

Cheers,
s

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#5 Post by greengeek »

So you are saying that dzen does what you want, but is more complex than you need, so you want a dzen replacement?

And you would prefer not to specify font/position etc from awk? Could you do it the other way round and call awk from the display program, so that you already define where you want the output of awk to appear before awk is running?

(forgive my ignorance - I am not a programmer. This just looks an interesting project).

User avatar
linuph
Posts: 128
Joined: Mon 04 Jun 2012, 02:29
Location: Philippines

#6 Post by linuph »

Yes, dzen2 does what I need but in some respects it's a bit cumbersome in the way I use it. And yes, I'm looking for something that is simpeler, programming wise.

In the mean time I looked into Cairo and Pango. A dedicated application can be written with it, but I quickly got lost in C programming :roll: I guess I have to get serious about learning C sometime...



greengeek:
Dzen2 does nothing by itself. It just displays stuff that you feed into it. It cannot call other programs.

To explain a bit more:

Every second a couple of comma separated strings arrive at the /dev/rfcomm0 port that must be parsed (read/processed). Awk is very efficient and fast in parsing such strings because it was designed to do exactly that.

Calling the awk routine from another program would be inefficient.

Mostly, GPSD and GPSPIPE are used to read GPS data. You can even use cat. The program structure would then be:

-read (gpspipe or cat)
-pipe
-process (awk)
-print (dzen2)

Directly with awk:
-parse
-print (dzen2)

GPSD and GPSPIPE are defacto standards. It is universal and allows various GPS hardware to communicate with various applications in various programming languages. However, I don't need it since I write a dedicated application.

BTW:
I use a Bluetooth GPS module. That's why you see '/dev/rfcomm0'. The rfcomm application takes care of the wireless connection with the Bluetooth GPS module and puts the GPS data stream on the serial port designated '/dev/rfcomm0'.

seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

#7 Post by seaside »

linuph wrote: BTW:
I use a Bluetooth GPS module. That's why you see '/dev/rfcomm0'. The rfcomm application takes care of the wireless connection with the Bluetooth GPS module and puts the GPS data stream on the serial port designated '/dev/rfcomm0'.
linuph,

Ah.... does the bluetooth device input data as a stream (continuously) or a discrete package. If it's a stream, you'll need to read a discrete block of data at a time.

Cheers,
s

User avatar
linuph
Posts: 128
Joined: Mon 04 Jun 2012, 02:29
Location: Philippines

#8 Post by linuph »

It's streamed packages, sort of.

Every second a comma separated string is sent. In each string are headers like $GPGGA or $GPRMC or $GPGSA (and more, depends on the GPS module). After the header follows particular data, ending with a \n.

So, yes, you have to separate the packages from the stream and process them. That's what awk is very good in.

The specification is according to the NMEA0183 standard. Here's documentation about that: http://aprs.gids.nl/nmea/

seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

#9 Post by seaside »

linuph wrote: So, yes, you have to separate the packages from the stream and process them. That's what awk is very good in.
linuph,

It appears that you know how to parse the data with awk. Another possibility I noticed in searching around for GPS data collection, was this video using python.
http://www.youtube.com/watch?v=d9hsQ_OhD20

Possibly the entire process could be done with python.

Cheers,
s
Last edited by seaside on Mon 28 Jan 2013, 20:19, edited 1 time in total.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#10 Post by greengeek »

seaside wrote:
linuph wrote: So, yes, you have to separate the packages from the stream and process them. That's what awk is very good in.
linuph,

It appears that you know how to parse the data with awk. Another possibility I noticed in searching around for GPS data collection, was this video using python.
http://www.youtube.com/watch?v=d9hsQ_OhD20

Possibly the entire process could be done with python.

Cheers,
s
seaside - your post would not display because of an extra space at the end of the url

seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

#11 Post by seaside »

greengeek wrote: seaside - your post would not display because of an extra space at the end of the url
greengeek, Thanks, I was wondering what happened?
Now fixed.
s

User avatar
linuph
Posts: 128
Joined: Mon 04 Jun 2012, 02:29
Location: Philippines

#12 Post by linuph »

Yes, the process can be done entirely in Python. However, I stay away from it because such Python applications generally require large libraries. That doesn't do well on my old laptop with limited resources.

It's a hobby and challenge to do things off the beaten paths of Python, Java, Ruby, Lua, MatLab and what more. I would make an exception for C if I knew how.

The funny thing is that you can do relatively complex things on an outdated computer with your Puppy Linux as is, n.l. with bash and awk. But you can't display anything graphically. That's where dzen2 comes in as an extra.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#13 Post by technosaurus »

you can generate svg images with text and other included images directly from awk or shell (pretty much anything really) and use this:
http://murga-linux.com/puppy/viewtopic.php?t=80352

I have posted a lot of svg generation tips here:
http://www.murga-linux.com/puppy/viewtopic.php?t=76431
Last edited by technosaurus on Thu 31 Jan 2013, 02:46, edited 1 time in total.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#14 Post by greengeek »

technosaurus wrote:I have posted a lot of svg generation tips here: http://www.murga-linux.com/puppy/viewtopic.php?t=7643
Seems to be the wrong link? Typo?

EDIT: Looks like it should be topic 76431

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#15 Post by technosaurus »

Thanks, fixed.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
linuph
Posts: 128
Joined: Mon 04 Jun 2012, 02:29
Location: Philippines

#16 Post by linuph »

Thanks, technosaurus.

I'm woking on improving the awk parsing routine. One of the problems is handling errors when the GPS data stream is killed or interrupted. I'll find a way...

I do believe that svg and gtk/C are the way to go. I will pick up on that soon. Your input is very valuable.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#17 Post by technosaurus »

if you want it to stop on a timeout after 9 seconds

gpscommand | while read -t9 LINE; do
...do stuff on $LINE
done
... it must have timed out if we are here
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

#18 Post by seaside »

technosaurus wrote:if you want it to stop on a timeout after 9 seconds

gpscommand | while read -t9 LINE; do
...do stuff on $LINE
done
... it must have timed out if we are here
linuph,

Good tip by technosaurus.

From my recent understanding of the sent GPS sentences, a timeout on the read occurring before the sentence was completely read, would result in a partial sentence.

Also, you may find more control by reading one char at a time and testing for EOL.

Cheers,
s

User avatar
linuph
Posts: 128
Joined: Mon 04 Jun 2012, 02:29
Location: Philippines

#19 Post by linuph »

Thanks for the tip! It started me thinking. I apologize for the somewhat lengthy post below but it helped me my thoughts organized. Please bear with me.

I indeed think that counting the amount of GPS sentences with invalid or no information is a way to detect an error. However, error checking outside the parsing routine won't work well with my program. It has to do with the way I designed it and that has to do with speed.

I'll try to explain as far as I understand all this.

The 'kernel' of the program is the awk parsing routine. Remember that awk is a programming language and runs in it's own shell. It is fast, according to some faster than a similar program in C or C++. Calling routines from awk outside it's own shell may slow down the parsing process.

Very simplified, my program structure is, in order:

bash shell - ensure Bluetooth and monitor availability, set background
awk shell & - parse GPS data, import system data, display data (dzen2)
bash shell - grab video from display &, display webcam (Mplayer)

awk is pushed into the background ('&'), otherwise you're stuck in the awk shell and you can't easily startup the video screen grabber (also pushed into the background) or Mplayer.

awk runs happily in it's own shell but has to import/export data. Exporting text to dzen2 is no problem since dzen2 is fast enough not to cause delays, no matter how many dzen2 instances.

Importing data from /dev/rfcomm0 is no problem either because awk is infinitely faster than the speed with which GPS data comes in.

But noticeable speed issues arise when calling some of the system functions. Most evident is calling the system time:

Code: Select all

cmd="date +"%T""; cmd | getline system_time
close(cmd)
The time displayed freezes, skipping seconds at a time, is irregular. In stead I use the GPS time, convert it to local time (GPS time is always UTC) and format:

Code: Select all

local_time_hours=substr(gps_time,0,2)+timezone
if ( local_time_hours > 23 ) local_time_hours=0
local_time=local_time_hours":"gps_time_minutes":"gps_time_seconds
This is much faster than calling 'date +"T%"'.

(this, by the way, un-synchronizes time and local system date but there are solutions for that).

Similarly, calling a 'while-do' loop outside the awk shell puts a handbreak on awk. That's undesired and a waste of awk's capabilties.

Concluding this: I do not want to rely on routines outside the awk shell to recognize errors in the GPS datastream. It must be done entirely within the parsing routine itself.

That brings us to details of the GPS datastream. It looks like this (example):

--------------------------------------------------------------------------
$GPGGA,043428.163,,,,,0,00,50.0,,M,,,,0000*23

$GPGGA,043429.163,,,,,0,00,50.0,,M,,,,0000*22

$GPRMC,043429.163,V,,,,,,,020213,,,N*43

$GPVTG,,T,,M,,N,,K,N*2C
---------------------------------------------------------------------------
$GPGGA,043430.163,,,,,0,00,50.0,,M,,,,0000*2A

$GPGSA,A,1,,,,,,,,,,,,,50.0,50.0,50.0*05

$GPGSV,2,1,08,15,63,310,,09,55,132,29,26,45,016,23,02,43,129,*7F

$GPGSV,2,2,08,24,37,178,,29,36,271,,05,34,039,,21,07,321,*7B

$GPRMC,043430.163,V,,,,,,,020213,,,N*4B

$GPVTG,,T,,M,,N,,K,N*2C
----------------------------------------------------------------------------
$GPGGA,043431.163,,,,,0,00,50.0,,M,,,,0000*2B

$GPRMC,043431.163,V,,,,,,,020213,,,N*4A

$GPVTG,,T,,M,,N,,K,N*2C
-----------------------------------------------------------------------------
$GPGGA,043432.163,,,,,0,00,50.0,,M,,,,0000*28

$GPRMC,043432.163,V,,,,,,,020213,,,N*49

$GPVTG,,T,,M,,N,,K,N*2C
-----------------------------------------------------------------------------

(the '-------' lines are to indicate one second data blocks but actually they are empty sentences).

When the GPS signal is interrupted (.e.g. when you drive through under a bridge) there should be more empty sentences. However, the GPS receiver keeps a buffer of x seconds, so the GPS data is retained for a while. Interruptions longer than x seconds (e.g. when driving through a long tunnel) result in empty sentences only should result in a 'lost signal' error.

The parsing routine ignores empty sentences (including the ones in a normal situation as illustrated above) and thus, sofar, doesn't know the difference between intermittent empty sentences and empty sentences forever. awk just keeps looking at the /dev/rfcomm0 port with nothing to parse.

That is where the technosaurus' tip comes in: I should detect empty sentences in the parsing routine within a time frame and decide there what to do with it.

I have been looking up a lot about awk programming. Mostly, manuals and examples are about parsing text documents. awk is mostly treated as a utility alongside with grep and sed (which are programming languages too) and, worse, in combination of those and, much worse, with cat.
awk can do that all by itself. It can do al lot more than simple document processing. I read somewhere that even an OS was built with awk, but so far did not find the details.

I'll keep in touch on progress.

Thanks guys.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#20 Post by greengeek »

Have you tried a puppy with a RealTime kernel (like those used for audio/studio recording)? Things I've read suggest they are quicker and maybe the "read current time" issue (or "get GPS time") could be sped up by that?

Post Reply