Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Tue 21 Oct 2014, 17:45
All times are UTC - 4
 Forum index » Off-Topic Area » Programming
Verify if a string is a valid UTF-8 string? [SOLVED]
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
Page 1 of 2 Posts_count   Goto page: 1, 2 Next
Author Message
Argolance


Joined: 06 Jan 2008
Posts: 1835
Location: PORT-BRILLET (Mayenne - France)

PostPosted: Tue 05 Mar 2013, 12:40    Post_subject:  Verify if a string is a valid UTF-8 string? [SOLVED]  

Hello,
Don't know if it is specific to my NVIDIA graphic card, but running a script (report-video) that intend to get the product label, I get this:
Code:
(gtkdialog:28206): Pango-WARNING **: Invalid UTF-8 string passed to pango_layout_set_text()

(gtkdialog:28206): Gtk-WARNING **: Failed to set text from markup due to error parsing markup: Erreur à la ligne 5, caractère 2 : Codage UTF-8 non valide dans le nom - « 
 oem: NVIDIA
product: BIOS-P/N@N3932:0 GW-CLK@\x84\u0003\x8a\u0002\xe8\u0003\xbc\u0002\xb0\u0004R\u0003

 » n'est pas valide

So, I would like to verify if the string is a valid UTF-8 string, then display, for example "unknown" if not, because this makes the script not to be displayed properly ("strange" characters and wrong text using markup) and consequently xerrs.log (bip!) not to be editable/readable.

Thank you.

Cordialement.
173343_476x436_easyshot.jpg
 Description   
 Filesize   31.42 KB
 Viewed   374 Time(s)

173343_476x436_easyshot.jpg

172904_476x455_easyshot.jpg
 Description   
 Filesize   36.3 KB
 Viewed   380 Time(s)

172904_476x455_easyshot.jpg


Edited_time_total
Back to top
View user's profile Send_private_message Visit_website 
amigo

Joined: 02 Apr 2007
Posts: 2257

PostPosted: Tue 05 Mar 2013, 13:55    Post_subject:  

I found this:
https://github.com/zeevt/validate_utf8

and this:
http://lua-users.org/lists/lua-l/2011-01/msg00503.html

validate_utf8 seems to work here.
Back to top
View user's profile Send_private_message 
Argolance


Joined: 06 Jan 2008
Posts: 1835
Location: PORT-BRILLET (Mayenne - France)

PostPosted: Tue 05 Mar 2013, 14:40    Post_subject:  

Thank you!
I found such very complicated stuffs too but it is probably very difficult to use inside a simple bash script (for me!). I thought of something simpler. I am wondering if it should not be more convenient to use a function with grep command to know if there is any error message inside xerrs.log file concerning this string, then, if yes and so on?

Cordialement.
Back to top
View user's profile Send_private_message Visit_website 
don570


Joined: 10 Mar 2010
Posts: 3337
Location: Ontario

PostPosted: Tue 05 Mar 2013, 15:52    Post_subject:  

I would check if there is an error message

Code:
report-video  2>/tmp/error$$


Barry Kauler should be warned about this.
Back to top
View user's profile Send_private_message 
Argolance


Joined: 06 Jan 2008
Posts: 1835
Location: PORT-BRILLET (Mayenne - France)

PostPosted: Tue 05 Mar 2013, 18:30    Post_subject:  

Hello don570,
Quote:
Barry Kauler should be warned about this.
I tried to let a message on bkhome but I forgot my password Embarassed ... Please, could you do it?

Thanks.

Cordialement.
Back to top
View user's profile Send_private_message Visit_website 
don570


Joined: 10 Mar 2010
Posts: 3337
Location: Ontario

PostPosted: Tue 05 Mar 2013, 18:42    Post_subject:  

I posted it on his blog for you.

You should run Tazoc's SYS-INFO to generate a report and post it
on this thread.


By the way I see a gtkdialog error in your image so I don't think
that Barry K's simple script to find the driver that loaded is causing
this problem. I believe he added in a gtkdialog script in recent
builds.

Perhaps you could download the source code of gtkdialog
and compile a version specifically for your puppy.
/usr/sbin/gtkdialog4 is the application and /usr/sbin/gtkdialog
is the link. Zigbert had problems with his
packages because gtkdialog wasn't installed correctly.

http://code.google.com/p/gtkdialog/downloads/detail?name=gtkdialog-0.8.3.tar.gz


.
Back to top
View user's profile Send_private_message 
Argolance


Joined: 06 Jan 2008
Posts: 1835
Location: PORT-BRILLET (Mayenne - France)

PostPosted: Wed 06 Mar 2013, 05:57    Post_subject:  

Hello,
Quote:
I posted it on his blog for you.

Thanks.
Quote:
By the way I see a gtkdialog error in your image so I don't think
that Barry K's simple script to find the driver that loaded is causing
this problem. I believe he added in a gtkdialog script in recent
builds.
The first picture above shows the video-report GUI when the wrong UTF-8 line inside /tmp/video-report is deleted and replaced with "unknown", then saved as /tmp/report-video-test. Inside the original GUI, I replaced "report-video" with "report-video-test" for testing... and this works fine, including additional "text use-makup" and so on - for my own.
Quote:
export RV="
<window title=\"$(gettext 'VIDEO REPORT')\" >
<vbox>
<vbox border-width=\"30\">
<pixmap><width>48</width><input file>/usr/share/pixmaps/report-video.png</input></pixmap>
<text use-markup=\"true\">
<label>\"$(cat /tmp/report-video-test)\"</label>
</text>
</vbox>
<hbox>
<button tooltip-text=\"$(gettext 'Edit /tmp/report-video file')\"><input file stock=\"gtk-edit\"></input>
<action>defaulttextviewer /tmp/report-video &</action></button>
<button><input file stock=\"gtk-close\"></input></button>
</hbox>
</vbox>
</window>"

$GTKDIALOG --program=RV

The second picture above shows what happens when report-video runs with the normal (wrong) /tmp/report-video.

When using the Lignthouse GUI (picture below), text window is blank.
Code:
Video-Info 1.5.1 - mer. 6 mars 2013 on ToOpPy Precise 5.4.3 - Linux 3.2.29 i686
 
Chip description:
  0.0 VGA compatible controller: NVIDIA Corporation G94 [GeForce 9600 GT] (rev a1) 
  oem: NVIDIA
  product: BIOS-P/N@N3932:0 GW-CLK@è¼°R
 
X Server: Xorg  Driver used: vesa

X.Org version: 1.11.3
  dimensions:    3200x1200 pixels (956x354 millimeters)
  depth of root window:    24 planes

   ...the above also recorded in /tmp/root/ as video-info,
and archived with xorg.conf and Xorg.0.log as video-info-full.gz

(gtkdialog4:18443): Gtk-CRITICAL **: gtk_text_buffer_emit_insert: assertion `g_utf8_validate (text, len, NULL)' failed

Cordialement.
105654_686x467_easyshot.jpg
 Description   
 Filesize   25.07 KB
 Viewed   292 Time(s)

105654_686x467_easyshot.jpg

Back to top
View user's profile Send_private_message Visit_website 
Argolance


Joined: 06 Jan 2008
Posts: 1835
Location: PORT-BRILLET (Mayenne - France)

PostPosted: Wed 06 Mar 2013, 09:45    Post_subject:  

Hello don570,
... Solved!
The reason why I got this UTF-8 issue: I was using/making enhancement from your "report-video 1.0" (now "report-video 1.1") which had something wrong somewhere...
Using the latest BK's version (from Puppy Precise 5.4.95) and building a GUI with, all works perfectly.
I visited bkhome thread about "report-video" and read the answer given to "our" post:
Quote:
Posted on 6 Mar 2013, 10:15 by BarryK
Re report-video
No, that is not my 'report-video' script.

There are two other report-video PETs, sys-info by TaZoc and another by sc0ttman.

My Wary/Racy 5.5, and I think the latest Precise, only have my report-video, which is commandline only, does not use pango/gtk. The Forum thread does not identify which "report-video" is causing the problem.

By the way, I asked the authors to change the name of their programs so as not to be the same as my 'report-video'.

This is quite right and I personally will do it! Wink
So, all my apologies to BK for having taken this feather out of its cap. Embarassed

Cordialement.
Back to top
View user's profile Send_private_message Visit_website 
Argolance


Joined: 06 Jan 2008
Posts: 1835
Location: PORT-BRILLET (Mayenne - France)

PostPosted: Wed 06 Mar 2013, 12:53    Post_subject:  

Hello,
Here is a nth version of report-video named "2P Report Video" Wink
pot file is in /usr/share/doc/nls/2P_report-video for bold text to be taken into account and some strings that are not translated in the original BK script...

Cordialement.
2P_report-video-1.0.pet
Description 
pet

 Download 
Filename  2P_report-video-1.0.pet 
Filesize  8.31 KB 
Downloaded  109 Time(s) 
173258_731x463_easyshot.jpg
 Description   
 Filesize   49.52 KB
 Viewed   256 Time(s)

173258_731x463_easyshot.jpg

173313_463x538_easyshot.jpg
 Description   
 Filesize   41.51 KB
 Viewed   248 Time(s)

173313_463x538_easyshot.jpg

Back to top
View user's profile Send_private_message Visit_website 
L18L

Joined: 19 Jun 2010
Posts: 2525
Location: Moved from Hosla to www.eussenheim.de

PostPosted: Wed 06 Mar 2013, 13:38    Post_subject: nth version of report-video named "2P Report Video"  

Hello Argolance,
you wrote:
...
pot file is in /usr/share/doc/nls/2P_report-video for bold text to be taken into account and some strings that are not translated in the original BK script...

1,
Why do you share 2P_report-video.pot? Does momanager, the tool for translations with puppy not work with 2P_report-video Question
2,
Which are the strings that are not translated in the original BK script?

Cordialement.
Back to top
View user's profile Send_private_message 
Argolance


Joined: 06 Jan 2008
Posts: 1835
Location: PORT-BRILLET (Mayenne - France)

PostPosted: Wed 06 Mar 2013, 14:20    Post_subject:  

Hello L18L,
Quote:
1,
Why do you share 2P_report-video.pot? Does momanager, the tool for translations with puppy not work with 2P_report-video
I suppose yes, but I admit I never used momanager: Just little command lines and poeditor... Sorry!
Quote:
2,
Which are the strings that are not translated in the original BK script?
just one: Line 43 "Requested by /etc/X11/xorg.conf:" and the 2 words "millimeters" and "planes" that my GUI translate using gettext too, directly inside /tmp/report-video and /tmp/report-video_ (these 2 files are different because one is displayed inside the window GUI with bold characters - text use-markup - and the other without, in the default text viewer).

Cordialement.
Back to top
View user's profile Send_private_message Visit_website 
L18L

Joined: 19 Jun 2010
Posts: 2525
Location: Moved from Hosla to www.eussenheim.de

PostPosted: Thu 07 Mar 2013, 06:34    Post_subject:  

Argolance wrote:
...
Quote:
2,
Which are the strings that are not translated in the original BK script?
just one: Line 43 "Requested by /etc/X11/xorg.conf:" and the 2 words "millimeters" and "planes" that my GUI translate using gettext too, directly inside /tmp/report-video and /tmp/report-video_ (these 2 files are different because one is displayed inside the window GUI with bold characters - text use-markup - and the other without, in the default text viewer)....

Thanks, now I see too:
-Requested by /etc/X11/xorg.conf:
-millimeters
-planes
and
pixels Rolling Eyes

with / without <b>

why not change from
Quote:

echo "$(gettext '<b>VIDEO REPORT:</b>') ${DISTRO_NAME}, $(gettext 'version') ${DISTRO_VERSION}"...

to
Quote:
echo "<b>$(gettext VIDEO REPORT:')</b> ${DISTRO_NAME}, $(gettext 'version') ${DISTRO_VERSION}" ....

?
Smile
Back to top
View user's profile Send_private_message 
Argolance


Joined: 06 Jan 2008
Posts: 1835
Location: PORT-BRILLET (Mayenne - France)

PostPosted: Thu 07 Mar 2013, 08:06    Post_subject:  

Hello L18L,
Quote:
why not change from
Quote:

echo "$(gettext '<b>VIDEO REPORT:</b>') ${DISTRO_NAME}, $(gettext 'version') ${DISTRO_VERSION}"...

to
Quote:
echo "<b>$(gettext VIDEO REPORT:')</b> ${DISTRO_NAME}, $(gettext 'version') ${DISTRO_VERSION}" ....

I got into this habit because, in most of the cases I had to handle, there were several marks, for example:
Quote:
echo "$(gettext '<b><span size='large'>VIDEO REPORT:</span></b>is a <i>GUI</i> for <u><i>BK</i> command line script</u>')
and I noticed it is simpler this way than repeat $(gettext '') for each marked word. Wink
But in the case of "report-video", I admit it would be better to make otherwise.
Quote:
-Requested by /etc/X11/xorg.conf:
-millimeters
-planes
and
pixels

... "forgot" this one, because it is the same word in most of the languages. But maybe you are right, we should not forget the "others", even if they are a minority... Wink

EDIT: I was wrong, I would have say for most of the European languages
بكس Arabian
пиксель Russian
ピクセル Japanese
...
So, you are right: this has to be done!

Cordialement.

Edited_time_total
Back to top
View user's profile Send_private_message Visit_website 
L18L

Joined: 19 Jun 2010
Posts: 2525
Location: Moved from Hosla to www.eussenheim.de

PostPosted: Thu 07 Mar 2013, 11:13    Post_subject:  

Argolance wrote:
...I would have say for most the European languages.

So you have been thinking at our Greeks too! Cool
-------
http://forum.slitaz.org/topic/i18n-rules-tips-tricks
Back to top
View user's profile Send_private_message 
Argolance


Joined: 06 Jan 2008
Posts: 1835
Location: PORT-BRILLET (Mayenne - France)

PostPosted: Thu 07 Mar 2013, 13:15    Post_subject:  

Greeks, Chinese (not exactly a "minority"), Bretons.... Wink
Back to top
View user's profile Send_private_message Visit_website 
Display_posts:   Sort by:   
Page 1 of 2 Posts_count   Goto page: 1, 2 Next
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
 Forum index » Off-Topic Area » Programming
Jump to:  

Rules_post_cannot
Rules_reply_cannot
Rules_edit_cannot
Rules_delete_cannot
Rules_vote_cannot
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1065s ][ Queries: 12 (0.0050s) ][ GZIP on ]