Looking for a simple, fast routine to display text, images

[linuph]

Thanks, technosaurus.

I'm woking on improving the awk parsing routine. One of the problems is handling errors when the GPS data stream is killed or interrupted. I'll find a way...

I do believe that svg and gtk/C are the way to go. I will pick up on that soon. Your input is very valuable.

[technosaurus]

if you want it to stop on a timeout after 9 seconds

gpscommand | while read -t9 LINE; do
...do stuff on $LINE
done
... it must have timed out if we are here

[seaside]

if you want it to stop on a timeout after 9 seconds

gpscommand | while read -t9 LINE; do
...do stuff on $LINE
done
... it must have timed out if we are here

linuph,

Good tip by technosaurus.

From my recent understanding of the sent GPS sentences, a timeout on the read occurring before the sentence was completely read, would result in a partial sentence.

Also, you may find more control by reading one char at a time and testing for EOL.

Cheers,
s

[linuph]

Thanks for the tip! It started me thinking. I apologize for the somewhat lengthy post below but it helped me my thoughts organized. Please bear with me.

I indeed think that counting the amount of GPS sentences with invalid or no information is a way to detect an error. However, error checking outside the parsing routine won't work well with my program. It has to do with the way I designed it and that has to do with speed.

I'll try to explain as far as I understand all this.

The 'kernel' of the program is the awk parsing routine. Remember that awk is a programming language and runs in it's own shell. It is fast, according to some faster than a similar program in C or C++. Calling routines from awk outside it's own shell may slow down the parsing process.

Very simplified, my program structure is, in order:

bash shell - ensure Bluetooth and monitor availability, set background
awk shell & - parse GPS data, import system data, display data (dzen2)
bash shell - grab video from display &, display webcam (Mplayer)

awk is pushed into the background ('&'), otherwise you're stuck in the awk shell and you can't easily startup the video screen grabber (also pushed into the background) or Mplayer.

awk runs happily in it's own shell but has to import/export data. Exporting text to dzen2 is no problem since dzen2 is fast enough not to cause delays, no matter how many dzen2 instances.

Importing data from /dev/rfcomm0 is no problem either because awk is infinitely faster than the speed with which GPS data comes in.

But noticeable speed issues arise when calling some of the system functions. Most evident is calling the system time:

Code: Select all

cmd="date +"%T""; cmd | getline system_time
close(cmd)

The time displayed freezes, skipping seconds at a time, is irregular. In stead I use the GPS time, convert it to local time (GPS time is always UTC) and format:

Code: Select all

local_time_hours=substr(gps_time,0,2)+timezone
if ( local_time_hours > 23 ) local_time_hours=0
local_time=local_time_hours":"gps_time_minutes":"gps_time_seconds

This is much faster than calling 'date +"T%"'.

(this, by the way, un-synchronizes time and local system date but there are solutions for that).

Similarly, calling a 'while-do' loop outside the awk shell puts a handbreak on awk. That's undesired and a waste of awk's capabilties.

Concluding this: I do not want to rely on routines outside the awk shell to recognize errors in the GPS datastream. It must be done entirely within the parsing routine itself.

That brings us to details of the GPS datastream. It looks like this (example):

--------------------------------------------------------------------------
$GPGGA,043428.163,,,,,0,00,50.0,,M,,,,0000*23

$GPGGA,043429.163,,,,,0,00,50.0,,M,,,,0000*22

$GPRMC,043429.163,V,,,,,,,020213,,,N*43

$GPVTG,,T,,M,,N,,K,N*2C
---------------------------------------------------------------------------
$GPGGA,043430.163,,,,,0,00,50.0,,M,,,,0000*2A

$GPGSA,A,1,,,,,,,,,,,,,50.0,50.0,50.0*05

$GPGSV,2,1,08,15,63,310,,09,55,132,29,26,45,016,23,02,43,129,*7F

$GPGSV,2,2,08,24,37,178,,29,36,271,,05,34,039,,21,07,321,*7B

$GPRMC,043430.163,V,,,,,,,020213,,,N*4B

$GPVTG,,T,,M,,N,,K,N*2C
----------------------------------------------------------------------------
$GPGGA,043431.163,,,,,0,00,50.0,,M,,,,0000*2B

$GPRMC,043431.163,V,,,,,,,020213,,,N*4A

$GPVTG,,T,,M,,N,,K,N*2C
-----------------------------------------------------------------------------
$GPGGA,043432.163,,,,,0,00,50.0,,M,,,,0000*28

$GPRMC,043432.163,V,,,,,,,020213,,,N*49

$GPVTG,,T,,M,,N,,K,N*2C
-----------------------------------------------------------------------------

(the '-------' lines are to indicate one second data blocks but actually they are empty sentences).

When the GPS signal is interrupted (.e.g. when you drive through under a bridge) there should be more empty sentences. However, the GPS receiver keeps a buffer of x seconds, so the GPS data is retained for a while. Interruptions longer than x seconds (e.g. when driving through a long tunnel) result in empty sentences only should result in a 'lost signal' error.

The parsing routine ignores empty sentences (including the ones in a normal situation as illustrated above) and thus, sofar, doesn't know the difference between intermittent empty sentences and empty sentences forever. awk just keeps looking at the /dev/rfcomm0 port with nothing to parse.

That is where the technosaurus' tip comes in: I should detect empty sentences in the parsing routine within a time frame and decide there what to do with it.

I have been looking up a lot about awk programming. Mostly, manuals and examples are about parsing text documents. awk is mostly treated as a utility alongside with grep and sed (which are programming languages too) and, worse, in combination of those and, much worse, with cat.
awk can do that all by itself. It can do al lot more than simple document processing. I read somewhere that even an OS was built with awk, but so far did not find the details.

I'll keep in touch on progress.

Thanks guys.

[greengeek]

Have you tried a puppy with a RealTime kernel (like those used for audio/studio recording)? Things I've read suggest they are quicker and maybe the "read current time" issue (or "get GPS time") could be sped up by that?

[linuph]

Mmm...I didn't know that there is such a thing as a 'RealTime kernel', let alone a Puppy with that. Which Puppy would that be?

[greengeek]

The first place worth a look is here:
http://www.linuxmusicians.com/viewtopic.php?f=4&t=6994

There is also a puppy called Turbopup Xtreme which I think was in part designed/configured by the same people and was well on the way to being an rt puppy. You can find the Turbopup thread here:
http://www.murga-linux.com/puppy/viewtopic.php?t=40477

My understanding of the "rt" part is that the kernel was altered to prevent distractions from un-needed processes, however I'm not clued up on how this was done or how effective it is, although I have read some comparative analyses of response times / latencies during audio recording sessions and it appears to be true that an rt kernel is faster. (Don't know how much of this is hype or how effective it may actually be)

There was also puppy studio 3.1 which is available somewhere on the net, (EDIT:http://sourceforge.net/projects/puppystudiorbd/) and a semi-commercial version called Studio 4 which I think has now been released as freeware due to being superceeded by a newer version called Studio 1337. I will add some other links if I find them. A word of caution though - the developer of these puppies ran into a lot of flak on the Murga forum for various reasons and some posts went missing so info may be fragmented.

Might be worth a try though. (If you trial Turbopup be warned that it boots so fast you don't get time to add any boot codes unless you madly type a key as it starts to boot - I think it's the "P" key you have to hit but I need to research it again. EDIT: yep it's the p key: http://208.109.22.214/puppy/viewtopic.p ... &start=237). My advice is don't boot Turbopup from CD on a system containing live data or savefiles, or it may behave differently than it should. Try it first up on a blank system.

EDIT: Comment about Turbopup Xtreme ram footprint here:
http://208.109.22.214/puppy/viewtopic.p ... &start=223
Also, this guy says he made a realtime puplet but I haven't tried it:
http://www.murga-linux.com/puppy/viewto ... 74&start=8
Interesting comment from Jamesbond. rt good for some uses, unstable for others:
http://208.109.22.214/puppy/viewtopic.p ... 85&start=8

[linuph]

Hi greengeek, thanks for looking up Puppies with RTkernel.

I will to try out either of them as soon as I have stabilized my program.

These Puppies are anyway very interesting if I ever pick up again on a neglected hobby: composing electronic music. I have some synthesizers, keyboards and other midi stuff laying around.

In the mean time, I have discovered some possibilities run to system calls (like df) outside the awk parsing routine.

One of them is the nifty 'wait' command. It's native and most distro's should have it. It's hardly mentioned anywhere. Well known is the similar 'cron' command but 'wait' works better in this case.