localizing-shell-scripts-without bashisms, gettext or ...

Under development: PCMCIA, wireless, etc.
Message
Author
User avatar
L18L
Posts: 3479
Joined: Sat 19 Jun 2010, 18:56
Location: www.eussenheim.de/

Re: still willing to test

#16 Post by L18L »

ferro10n wrote:...
Success this time around! Didn't have to change puppy version, it's Wary 5.2 running entirely from RAM. And I learned some German on the way :lol: Now testing pt-BR
...
Don't be disappointed if something will not work with pt_BR
Then this will be my fault.
Be patient with me, I am a slow learner.

Muchas gracias :D :D :D

ferro10n
Posts: 106
Joined: Wed 15 Jun 2011, 20:18
Contact:

#17 Post by ferro10n »

Yeah, I forgot this isn't meant to work with other languages yet :lol: but despite I still saw the GUI in English, the locale for pt_BR was generated the same. I also noticed that when I was asked to select a locale it appeared as 'pt', not pt_BR. I can still test that part, and then Italian and that's all my knowledge. I'm glad to help wherever I can. :D
A beginner Puppy Linux user
Intel(R) Celeron(R) CPU 2.13GHz 512MB
LiveCD user

User avatar
L18L
Posts: 3479
Joined: Sat 19 Jun 2010, 18:56
Location: www.eussenheim.de/

autogenerating

#18 Post by L18L »

technosaurus wrote:using L18L's naming method, you can quickly autogenerate a template:

Code: Select all

#!/bin/bash
for x in $@ ; do
while read -d$ A || [ "$A" ]; do
case "$A" in
*{_M_*)B=_M${A##*_M};B=${B%%\}*}";echo ${B//:-/="};;
esac
done < $x >$x.i18n
done
usage:
mki18n list of scripts

hmm, even though I used a bashism (read -d) it generates the entire template in less time than gettext takes to do a single translation

... it may be beneficial to change >$x.i18n to |sort -u >$x.i18n if you are planning to use a VAR multiple times
Technosaurus,
thank you, much appreciated :D

So this does confirm that we do not need to store English language files? They can be autogenerated very fast!

Having tried to really understand your mki18n I had to run it. :)
and "hacked" this variant:
#!/bin/bash
[ -f all_i18n ] && rm all_i18n # all in one
for x in $@ ; do
while read -d$ A || [ "$A" ]; do
case "$A" in
*{_M_*) B=_M${A##*_M};B=${B%%\}*}";echo ${x}:${B//:-/="};;
esac
done < $x |sort -u >$x.i18n >>all_i18n
done
# time mki18n askpass alsawizard

real 0m0.213s
user 0m0.076s
sys 0m0.024s
#

Stuff for thoughts...

Naming method
(for the record) _M_
short, unique, restricted to letters and underscore, easy to grep

Location of language files
Maybe a directory of their own is better?
M under /usr/share/locale/<language>
LANGPATH would become 2 bytes longer
But anything inside is _M_ method

I am thinking that "all in one" can be helpful for our PuppyMaster.
GUI: search a word and get all phrases containing this word
and maybe use an existing phrase in a new script...
...if so then 1 translation less

Or using just one language file for all the basic Puppy scripts...

The locale is really only needed when using date, time, currency..
Not for "just language"

I know that you know it (reminding your example of de_JA :D )

User avatar
L18L
Posts: 3479
Joined: Sat 19 Jun 2010, 18:56
Location: www.eussenheim.de/

#19 Post by L18L »

ferro10n wrote:Yeah, I forgot this isn't meant to work with other languages yet :lol: but despite I still saw the GUI in English, the locale for pt_BR was generated the same. I also noticed that when I was asked to select a locale it appeared as 'pt', not pt_BR. I can still test that part, and then Italian and that's all my knowledge. I'm glad to help wherever I can. :D
Yes, this is Cutting Edge
I am working on pt_BR...or generally enabling "regional variants"

I will make versions (not soon, maybe tomorrow) and post on page 1 (that is the way one should do it from beginning :oops: )

Thank you and good night (it is late here in Europe now)
:)

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#20 Post by technosaurus »

I am posting from my Droid so the following syntax will be "off"

[ -f /.../$LANG/app ] && LOCALES=/.../$LANG/app ¦¦ [ -f /.../${LANG%%_*} ] && LOCALES=/.../${LANG%%_*}
. $LOCALES

Edit: haven't tested above because I was doing this:

Code: Select all

curl -A firefox \
  -F "file=@trans.txt;type=text/plain" \
  -F "sl=en" \
  -F "tl=es" \
  http://translate.googleusercontent.com/translate_f | while read A || [ "$A" ]; do
    case $A in 
      "<meta"*)continue;;
      *"<pre>"*)echo ${A##*"<pre>"};;
      *"</pre>"*)echo ${A%%"</pre>"*};;
      *)echo $A;;
    esac
  done
replace trans.txt with your english file and tl=es with tl=??
where ?? is
  • es>Spanish
    yi>Yiddish
    cy>Welsh
    separator disabled>—
    af>Afrikaans
    sq>Albanian
    ar>Arabic
    hy>Armenian
    az>Azerbaijani
    eu>Basque
    be>Belarusian
    bn>Bengali
    bg>Bulgarian
    ca>Catalan
    zh-CN>Chinese(Simplified)
    zh-TW>Chinese(Traditional)
    hr>Croatian
    cs>Czech
    da>Danish
    nl>Dutch
    en>English
    et>Estonian
    tl>Filipino
    fi>Finnish
    fr>French
    gl>Galician
    ka>Georgian
    de>German
    el>Greek
    gu>Gujarati
    ht>Haitian
    Creole
    iw>Hebrew
    hi>Hindi
    hu>Hungarian
    is>Icelandic
    id>Indonesian
    ga>Irish
    it>Italian
    ja>Japanese
    kn>Kannada
    ko>Korean
    la>Latin
    lv>Latvian
    lt>Lithuanian
    mk>Macedonian
    ms>Malay
    mt>Maltese
    no>Norwegian
    fa>Persian
    pl>Polish
    pt>Portuguese
    ro>Romanian
    ru>Russian
    sr>Serbian
    sk>Slovak
    sl>Slovenian
    es>Spanish
    sw>Swahili
    sv>Swedish
    ta>Tamil
    te>Telugu
    th>Thai
    tr>Turkish
    uk>Ukrainian
    ur>Urdu
    vi>Vietnamese
    cy>Welsh
    yi>Yiddish
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
L18L
Posts: 3479
Joined: Sat 19 Jun 2010, 18:56
Location: www.eussenheim.de/

#21 Post by L18L »

technosaurus wrote:I am posting from my Droid so the following syntax will be "off"

[ -f /.../$LANG/app ] && LOCALES=/.../$LANG/app ¦¦ [ -f /.../${LANG%%_*} ] && LOCALES=/.../${LANG%%_*}
. $LOCALES
Seen this before your "edit"

That is: your one-liner can replace my two-lines code for enabling "regional variants" of language.
And LOCALES replaces LANGPATH.
_____________
edited later
and LOCALES is avoiding need of M/ under LOCALE :)
_______________

Very nice. Thank you
Last edited by L18L on Fri 18 Nov 2011, 14:31, edited 2 times in total.

User avatar
L18L
Posts: 3479
Joined: Sat 19 Jun 2010, 18:56
Location: www.eussenheim.de/

#22 Post by L18L »

technosaurus wrote:...

replace trans.txt with your english file and tl=es with tl=??
where ?? is
...
g** wrote: Sorry, the page you requested contains a file type (application/octet-stream) we are unable to translate.
:cry:
my script wrote:#!/bin/sh
echo -n > ./t12s_g_de
curl -A defaultbrowser \
-F "file=/dev/shm/t12s_askpass_en;type=text/plain" \
-F "sl=en" \
-F "tl=de" \
http://translate.googleusercontent.com/translate_f | while read A || [ "$A" ]; do
case $A in
"<meta"*)continue;;
*"<pre>"*)echo ${A##*"<pre>"} >> ./t12s_g_de;;
*"</pre>"*)echo ${A%%"</pre>"*} >> ./t12s_g_de;;
*)echo $A >> ./t12s_g_de;;
esac
done
g has been my friend until this happened
http://murga-linux.com/puppy/viewtopic. ... 9&start=33

Anyhow nice idea

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#23 Post by technosaurus »

Ok, I've merged them together to get autotranslated locale files but all my trial and error caused me to exceed the max queries temporarily. Can someone try this on a _M_ formatted script:

Code: Select all

#!/bin/bash
FROM=/C/
[ ! -d /usr/share/locales/C/ ] && mkdir /usr/share/locales/C/
for x in $@ ; do
	TEMPLATE=/usr/share/locales/C/${x##*/}
	while read -d$ A || [ "$A" ]; do
		case "$A" in
			*{_M_*)B=_M${A##*_M};B=${B%%\}*};echo ${B//:-/=};;
		esac
	done < $x >$TEMPLATE

for x in es yi cy sq ar hy az eu be bn bg ca zh-CN zh-TW hr cs da nl en et tl fi fr gl ka de el gu ht iw hi hu is id ga it ja kn ko la lv lt mk ms mt no fa pl pt ro ru sr sk sl es sw sv ta te th tr uk ur vi cy yi ; do
TO=/${x//-/_}/
[ ! -d ${TEMPLATE%%/C/*}/${x//-/_} ] && mkdir ${TEMPLATE%%/C/*}/${x//-/_}
curl -A firefox \
	-F "file=@$TEMPLATE;type=text/plain" \
	-F "sl=en" \
	-F "tl=$x" \
	http://translate.googleusercontent.com/translate_f | \
	while read A || [ "$A" ]; do
		case $A in
			"<meta"*)A="";;
			*"<pre>"*)A=${A##*"<pre>"};;
			*"</pre>"*)A=${A%%"</pre>"*};;
		esac
		[ "$A" ] && echo ${A// = /=\"}\"
	done > ${TEMPLATE//$FROM/$TO}
done
rm $TEMPLATE
done
Note: may need to have a .txt extension on the submitted file
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
L18L
Posts: 3479
Joined: Sat 19 Jun 2010, 18:56
Location: www.eussenheim.de/

Starting translation supported by g

#24 Post by L18L »

Success :D
technosaurus wrote:...
Note: may need to have a .txt extension on the submitted file
# TEMPLATE=/usr/share/locales/C/${x##*/}
TEMPLATE=/usr/share/locales/C/${x##*/}.txt

I have run it for 6 languages
time
real 0m8.659s
user 0m0.292s
sys 0m0.076s


Spanish:
_M_1="Esta aplicación necesita para funcionar:"
_M_2="La contraseña del administrador se requiere:"
_M_3="Por favor introduzca la contraseña de administrador para ejecutar esta aplicación:"
_M_4 Tipo="admin contraseña necesaria para ejecutar esta aplicación:"
_M_5 =... No"
ferro10n, este esta bueno?
As a start for a good translation, better than English!

Wondering where the Tipo is coming from :roll:
:) :D

ferro10n
Posts: 106
Joined: Wed 15 Jun 2011, 20:18
Contact:

Re: Starting translation supported by g

#25 Post by ferro10n »

L18L wrote:Success :D
technosaurus wrote:...
Note: may need to have a .txt extension on the submitted file
# TEMPLATE=/usr/share/locales/C/${x##*/}
TEMPLATE=/usr/share/locales/C/${x##*/}.txt

I have run it for 6 languages
time
real 0m8.659s
user 0m0.292s
sys 0m0.076s


Spanish:
_M_1="Esta aplicación necesita para funcionar:"
_M_2="La contraseña del administrador se requiere:"
_M_3="Por favor introduzca la contraseña de administrador para ejecutar esta aplicación:"
_M_4 Tipo="admin contraseña necesaria para ejecutar esta aplicación:"
_M_5 =... No"
ferro10n, este esta bueno?
As a start for a good translation, better than English!

Wondering where the Tipo is coming from :roll:
:) :D
This is from Google Translate, isn't it? :lol: Well, here's what the lines are supposed to be in Spanish:

Code: Select all

_M_1="Esta aplicación quiere ejecutar:"
_M_2="Requiere contraseña de administrador:"
_M_3="Por favor introduzca la contraseña de administrador requerida para ejecutar esta aplicación:"
_M_4="Escribir contraseña de administrador requerida para ejecutar esta aplicación:"
_M_5="... Falló"
The 'Tipo' comes from 'Type', I think, which in that line should be 'escribir'. We don't have a special word for 'to write by keyboard' in Spanish, although there are heretic people who translate it as 'tipear'. :roll:
A beginner Puppy Linux user
Intel(R) Celeron(R) CPU 2.13GHz 512MB
LiveCD user

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#26 Post by technosaurus »

Noone expects Google to get all the translating 100%, but at least its better than 0%.
Note that I use the Google translate site directly masked as FireFox with curl (wget cannot do right metadata for files), since the translate api is deprecated... So as long as the site itself is fairly stable, it should continue to work.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

ferro10n
Posts: 106
Joined: Wed 15 Jun 2011, 20:18
Contact:

#27 Post by ferro10n »

technosaurus wrote:Noone expects Google to get all the translating 100%, but at least its better than 0%.
Note that I use the Google translate site directly masked as FireFox with curl (wget cannot do right metadata for files), since the translate api is deprecated... So as long as the site itself is fairly stable, it should continue to work.
Sorry, I didn't read the full posts :wink: You're right, the translations may not be the most accurate, but they're understandable. As you said, it's better than 0%. You should make a PET :D
A beginner Puppy Linux user
Intel(R) Celeron(R) CPU 2.13GHz 512MB
LiveCD user

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#28 Post by technosaurus »

Now if I can just figure out how to connect piece G to happy.

I havent ironed out or commented on possible oopses or known issues ... For instance it won't work on translation variables that contain variables due to using $ as a separator... ${_M_1:-$hello world} would break the translation
... Still needs a bit of testing and documentation before people go installing it Willy nilly from a pet without looking at the code.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
L18L
Posts: 3479
Joined: Sat 19 Jun 2010, 18:56
Location: www.eussenheim.de/

oopses

#29 Post by L18L »

@ferro10n,
please forget what I have announced about another version (enabling pt_BR) for now. There is enough other stuff to be done before next version.

http://www.phrases.org.uk/meanings/willy-nilly.html :) ________________
technosaurus wrote:Now if I can just figure out how to connect piece G to happy.

I havent ironed out or commented on possible oopses or known issues ... For instance it won't work on translation variables that contain variables due to using $ as a separator... ${_M_1:-$hello world} would break the translation
... Still needs a bit of testing and documentation before people go installing it Willy nilly from a pet without looking at the code.
Known issue:
- wrong placement in variable name
_M_4 Tipo="admin contraseña necesaria para ejecutar esta aplicación:"

- other languages:
--Greek
_M_5 =...

ferro10n
Posts: 106
Joined: Wed 15 Jun 2011, 20:18
Contact:

#30 Post by ferro10n »

A beginner Puppy Linux user
Intel(R) Celeron(R) CPU 2.13GHz 512MB
LiveCD user

User avatar
L18L
Posts: 3479
Joined: Sat 19 Jun 2010, 18:56
Location: www.eussenheim.de/

localizing-shell-scripts-without bashisms, gettext or ...

#31 Post by L18L »

Thank you, let us continue here.
In another thread fero10n wrote:You guys are so computer-skilled, while all I can do is translate between five languages
Our computer skills are very different. Technosaurus is master of hacking... while I am just trying to walk in some of his foot steps :)
but you are also a software tester now :D (Willy nilly :wink: )

-----
Edited
pt_BR should work now
If you first translate just some messages for pt (not all)
then these should be preloaded when starting pt_BR
Confirm?
----
Once more edited

:oops:
If you do this then pt is killed.
Change script t12s line 234 from
mv ${T}/${x}/${app} ${T}/${this_LANG}/${app}_tmp
to
cp ${T}/${x}/${app} ${T}/${this_LANG}/${app}_tmp

That is: copy instead of move and pt will continue to live
And soon you will become a coder too :D

ferro10n
Posts: 106
Joined: Wed 15 Jun 2011, 20:18
Contact:

Re: localizing-shell-scripts-without bashisms, gettext or ...

#32 Post by ferro10n »

L18L wrote:you are also a software tester now :D
Yes, sir! 8)
L18L wrote: Edited
pt_BR should work now
If you first translate just some messages for pt (not all)
then these should be preloaded when starting pt_BR
Confirm?
----
Once more edited
If you do this then pt is killed.
Change script t12s line 234 from
mv ${T}/${x}/${app} ${T}/${this_LANG}/${app}_tmp
to
cp ${T}/${x}/${app} ${T}/${this_LANG}/${app}_tmp

That is: copy instead of move and pt will continue to live
And soon you will become a coder too :D
Done. Here are the results:

:arrow: Tested t12s with mentioned line modified.
:arrow: Translation was generated properly with no more question marks where special characters should be, for example 'n?o' instead of 'não'.
:!: The adding of a new translated line caused the whole list of translatable lines to repeat below, making the list longer than it actually is.
:!: At times I submitted a translation and then saw the line blank where the new translation should be, I had to Undo, Quit and start the script again to be able to submit the translation. Finally, as the bug continued, I had to edit the generated locale file to manually complete the translation process.

Generated translations:

Code: Select all

_M_10="AJUDA"
_M_11="Arquivo novo:"
_M_12="Arquivo novo ou .delta:"
_M_13="Arquivo velho:"
_M_14="por favor espere..."
_M_15="Por favor espere, procurando arquivo velho..."
_M_16="Desculpe não achous-se ${OLDFILENAME}. Terá de arrastá-lo manualmente na caixa ´Arquivo velho´"
_M_17="SUCESSO! ${SOURCEFILE_DELTA} gerado"
_M_18="SUCESSO! ${SOURCEFILE_NEW} gerado"
_M_19="O autor do Xdelta é Joshua MacDonald."
_M_20="Primeiro deve-se instalar o pacote ´xdelta´"
_M_21="Interface escrita por Barry Kauler. Licença GPL v2."
_M_22="O algoritmo Rsync, que inspirou o algoritmo delta central, deve-se a Andrew Tridgell e Paul Mackerras."
_M_23="Istas mudanças (deltas) são similares à saida do programa ´diff´ em que podem-se utilizar para armazenar e transmitir somente as mudanças entre arquivos. Porém, ao contrário do diff, a saida do Xdelta não é expressada em um formato legível pelos humanos (além disso Xdelta pode aplicar estes deltas a uma copia do arquivo original). Xdelta utiliza um rápido algoritmo linear e desenvolve-se bem tanto em arquivos binários quanto em arquivos de texto."
_M_24="Para criar um arquivo ´.delta´, você precisa dois arquivos originais, um arquivo ´velho´ e um ´novo´. O arquivo delta gerado terá uma extensão ´.delta´."
_M_25="Para gerar um arquivo ´.delta´, arraste os arquivos velho e novo aqui, ou para recriar o arquivo novo arraste os arquivos velho e ´.delta´ aqui"
_M_26="Bemvindo/a, ista é uma interface para a ferramenta ´xdelta´, para administrar mudanças entre dois arquivos"
_M_27="Xdelta é um programa de aplicação desenhado para computar mudanças entre dois arquivos."
_M_28="Você pode recriar o arquivo ´novo´ original, enquanto você tenha o arquivo ´velho´ original e o arquivo ´.delta´. Este frontend de interface faz com que isto seja fácil: somente arraste o arquivo .delta na caixa, e o programa procurará e localizará o arquivo velho original. Ou, somente clique em um arquivo .delta em uma janela do ROX-Filer para re-gerar o arquivo novo original."
_M_3="Erro, ´Arquivo novo´ não existe"
_M_4="Erro, ´Arquivo velho´ não existe"
_M_5="ERRO! falhiu a geração do arquivo .delta. Mensagem de erro:"
_M_6="ERRO! falhiu a recriação do arquivo novo. Mensagem de erro:"
_M_7="ERRO: ${SOURCEFILE} não existe"
_M_8="GERAR"
A beginner Puppy Linux user
Intel(R) Celeron(R) CPU 2.13GHz 512MB
LiveCD user

User avatar
L18L
Posts: 3479
Joined: Sat 19 Jun 2010, 18:56
Location: www.eussenheim.de/

Re: localizing-shell-scripts-without bashisms, gettext or ...

#33 Post by L18L »

ferro10n wrote: :!: The adding of a new translated line caused the whole list of translatable lines to repeat below, making the list longer than it actually is.
:!: At times I submitted a translation and then saw the line blank where the new translation should be, I had to Undo, Quit and start the script again to be able to submit the translation. Finally, as the bug continued, I had to edit the generated locale file to manually complete the translation process.
Thank you
fixed :)
Attachments
t12s.gz
bugfix version 0.4.3
Rename it to t12s (without .gz)
(13.09 KiB) Downloaded 478 times

User avatar
01micko
Posts: 8741
Joined: Sat 11 Oct 2008, 13:39
Location: qld
Contact:

#34 Post by 01micko »

Hi L18L

First, great job :)

I have 3 queries before I embark on my first attempt. (later, answered myself, app link at bottom of post :) )

1) Does this method handle strings with a newline char? ( \n )

example:

Code: Select all

echo -en "Choices \n1. Firefox \n2. Seamonkey \n\nChoose 1 or 2 to download \n"
A: Yes

2) Can we use escaped quotes?

example:

Code: Select all

echo "get_mozilla_browser-0.1
Invoke with \"get_mozilla_browser\" and you are offered a choice of 
\"firefox\" or \"seamonkey\" latest stable browsers direct from mozilla 
in your default locale if it exists.(snip)"
A: Yes

3) Can we use variables inside a string?

example:

Code: Select all

echo "$PACKAGE en-US will be downloaded "
A; No
-

If the answer to any or all of these is "no" it isn't a problem as I can split up the strings.

TIA

EDIT: app here!
Last edited by 01micko on Thu 01 Dec 2011, 05:47, edited 1 time in total.
Puppy Linux Blog - contact me for access

ferro10n
Posts: 106
Joined: Wed 15 Jun 2011, 20:18
Contact:

#35 Post by ferro10n »

This time around the t12s script worked nicely :D
I've even been able to change language so that I can make a new locale for the same file just after I finished the one I've worked on previously. However, I had to check that the locale folders were in place -- as the 'locales' folder was empty, I had to create individual folders for the locales I usually make (es, pt_BR, it, fr) so that the translations were generated there. If there wasn't a folder for the locale I wanted to work with, upon selecting the script to translate I got an empty list with no lines to translate, so I quit, created the locale folders and started t12s again, then I just wrote the folder name (es, it, fr, etc.) and got the list of translatable lines.
I think I'm ready to learn how to make the scripts translatable myself :D . Who knows, maybe I can make a localized pupplet, something like 'Wary EPI (= Español-Português-Italiano, the languages with less pupplets available that are based on newer puppy versions)'
Last edited by ferro10n on Sun 18 Dec 2011, 19:29, edited 2 times in total.
A beginner Puppy Linux user
Intel(R) Celeron(R) CPU 2.13GHz 512MB
LiveCD user

Post Reply