Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Thu 18 Sep 2014, 19:50
All times are UTC - 4
 Forum index » Advanced Topics » Puppy Derivatives
voxpup - a voice controlled puppy
Moderators: Flash, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 3 [35 Posts]   Goto page: 1, 2, 3 Next
Author Message
greengeek

Joined: 20 Jul 2010
Posts: 2542
Location: New Zealand

PostPosted: Fri 22 Nov 2013, 07:32    Post subject:  voxpup - a voice controlled puppy  

Would you like to control puppy by voice command only? Voxpup is my attempt to assemble a puppy that responds to your voice and allows you to start programs, open menus, browse, type (slowly) and basically tell puppy what to do, all without using the mouse.

Download here:

http://www.meownplanet.net/puppylinuxstuff/greengeek/voxpup001.iso
(thanks russoodle for the hosting!!)
or here: http://www.mediafire.com/download/8umzu2v94wvi2yp/voxpup001.iso
voxpup001.iso md5 2b07e99ed3f1c99e0b2656c605e0e98d
(NOTE :This iso is modified so that it ignores savefiles during booting. This is my safety feature to avoid complications for those trialling different pups)

This alternative iso uses standard puppy behaviour and will look for previous savefiles:
http://www.mediafire.com/download/w4cjopop7sw1ecm/voxpup001_find_savefiles.iso
md5=1dbda82b6a6d2c0531abe5b703b52083


Voxpup will run out-of-the-box on some machines, but will need microphone and/or capture settings to be adjusted on others.

Handy Hint ; Read the voxmain005commands.txt file that you can find in /root. This will show you what commands to use to control voxpup.

Voice decoding accuracy will depend on various adjustable factors:

- microphone and/or capture volume settings are the most critical factor in making the voice recognition work well. Generally the mic/capture volumes need to be set much lower than you would expect (however voxpup001 sets the volume low by itself, so it may work ok without tinkering...)
- microphone type. Expensive mics are not required - cheap ones work fine but the wiring config needs to match your soundcard's needs (some want a mono mic and some want stereo), and the shielding has to be good enough to avoid picking up electrical noise and hum from mains power adapters etc.. - try what you've got and see if it works...
- correct microphone port needs to be selected (some machines have two mic selections to choose from in the mixer settings)
- your accent. The voice recognition program is written for the American accent, but seemed to work almost perfectly for my NZ (kiwi) accent. (I have made adjustments / duplications within the 'dictionary' so that users have a better chance of getting recognition integrity).

You will probably get best performance from the voice recognition system if you use headphones so that the speaker output does not feed back into the microphone and confuse the voice detection. (I found it worked very well when I used a cheap multimedia 'computer headset' - the type that has headphones and mic built into one headpiece)

I will be posting some tutorials and troubleshooting info over the next few days, but in many cases the user will be able to get started by just plugging in their headset and following the onscreen instructions that appear after boot/startup has finished.

The user will find a couple of "voxmain" documents in /root, which will assist in understanding what commands are available. The voxmain menu will have other submenus added later.

There will also be other versions of voxpup with greater functionality as I get to grips with better coding.

Other Information
voxpup is built on Pemasu's Upup Raring 3.9.9.2
If you want more word processing functionality than the inbuilt Abiword offers, forum member vicmz has listed some good LibreOffice versions here: http://murga-linux.com/puppy/viewtopic.php?t=85915
(I am currently using v4.03 successfully with voxpup)
The voxpup voice recognition module is based on the "pocketsphinx_continuous" .pet offered by technosaurus here:
http://www.murga-linux.com/puppy/viewtopic.php?t=88095&start=27

Known bugs:
The "close window" command sometimes closes more than the one window I want it to. (something to do with child/parent processes?). This will be fixed in later releases.
.
voxpup screenie.jpg
 Description   
 Filesize   47.68 KB
 Viewed   879 Time(s)

voxpup screenie.jpg


Last edited by greengeek on Sat 08 Mar 2014, 17:51; edited 10 times in total
Back to top
View user's profile Send private message 
greengeek

Joined: 20 Jul 2010
Posts: 2542
Location: New Zealand

PostPosted: Fri 22 Nov 2013, 07:36    Post subject:  

Basic HowTo:

Mouse control is (at this stage) very basic - you can tell it to move ten, fifty or one hundred pixels left, right, up or down.

Most positioning is best done with the cursor left, right, up or down commands, the arrow left, right, up or down commands, or by the tab key commands.

Choose carefully when you want to use the enter or return key (relating to CURSOR position) or click/rightclick etc (which relate to MOUSE position).

Spelling/typing is by use of the NATO phonetic alphabet (with some modifications/alternatives). See the command list in /root (speaking the command "VOICE-CONTROL-SHOW-COMMAND-LIST" will get this document displayed up on screen)

When using voxpup it is best to issue the commands in a normal speaking voice - just speak at normal volume and speed. No need to shout or stress each word.
Also - if you want to have a conversation with others in the room and want to put voxpup to sleep for a minute just issue the following command:
"VOICE-CONTROL-MICROPHONE-MUTE-ONE-MINUTE"

Important notes :
- Pocketsphinx (which is the program which decodes the speech) only starts decoding when you reach the end of your command phrase and it hears some silence. If the room is too loud it will keep listening but not start decoding. Stop speaking for a couple of seconds after issuing each command...
- voxpup will often show lots of decoded 'junk words' that seem to have no relevance. Don't panic about this - it only happens when your command (or background conversation) falls outside of it's special command list. As long as it is correctly acting upon the commands in the official list that is what we are looking for.

At this stage voxpup is not intended to be capable of handling full vocabulary dictation (such a module may follow later...). If you need dictation functions I recommend looking at ETP's Obedient pup and it's online ability to access the Google dictation services.
.

Last edited by greengeek on Fri 22 Nov 2013, 14:02; edited 3 times in total
Back to top
View user's profile Send private message 
greengeek

Joined: 20 Jul 2010
Posts: 2542
Location: New Zealand

PostPosted: Fri 22 Nov 2013, 07:36    Post subject:  

Troubleshooting

**** If you are having problems getting things to work, I recommend doing your troubleshooting in a quiet room with no background noise. This will help provide stable results ****

1) After booting you should see a terminal showing the output of the voice decoder. The normal startup prompt is:
READY....

This means that the program is ready for you to begin speaking.

2) If the startup prompt shows:
READY...
Listening...

This means that it can already detect incoming sound through the microphone and is collecting the voice data. If the program says "Listening..." when you are not yet talking then this is a sign that the microphone/capture levels are set too loud and the mic is picking up background noise, electrical hum, fan noise etc etc.

3) If the "Listening..." prompt shows up only while you are talking that is a good sign. Once you stop talking it should then decode the command that you were speaking.

4) If the "Listening.." prompt does NOT show up while you are talking then that indicates that your mic is unplugged, broken, or that the mixer settings are set for too low mic/capture volume, or that the capture setting is selecting the wrong source (eg mix or line instead of mic), or that the wrong mic is selected in the mixer (eg some PCs allow switching between front or rear mics, or mic1, mic2 or inbuilt).

The actual mixer controls available will vary from machine to machine so if you are having problems please post in this thread and we will try to help.

Last edited by greengeek on Sat 23 Nov 2013, 15:49; edited 1 time in total
Back to top
View user's profile Send private message 
ally


Joined: 19 May 2012
Posts: 846
Location: lincoln

PostPosted: Fri 22 Nov 2013, 07:43    Post subject:  

mirrored here:

http://archive.org/details/Puppy_Linux_voxpup


Smile
Back to top
View user's profile Send private message Visit poster's website 
greengeek

Joined: 20 Jul 2010
Posts: 2542
Location: New Zealand

PostPosted: Fri 22 Nov 2013, 07:47    Post subject:  

Thanks Ally!
Back to top
View user's profile Send private message 
edoc


Joined: 07 Aug 2005
Posts: 4368
Location: Southeast Georgia, USA

PostPosted: Sat 23 Nov 2013, 15:24    Post subject:  

This could be really useful for mobile operation of Amateur Radio equipment.

It may need to interface with fldigi flrig etc. to take advantage of their existing interfaces to different equipment.

I will link this at linuxham@yahoogroups.com so that they are aware of it.

Thanks for building and sharing this!

_________________
Thanks! David
Home page: http://nevils-station.com
Don't google Search! http://duckduckgo.com
Puppy upup Raring 3992 & Lighthouse64-b602
Back to top
View user's profile Send private message Visit poster's website 
RSH


Joined: 05 Sep 2011
Posts: 2420
Location: Germany

PostPosted: Tue 26 Nov 2013, 20:06    Post subject:  

Hi.

This sounds pretty cool.

Would it be possible to modify this for DE users?

Would you like to provide the packages used for this to be downloaded as well?

RSH

_________________
LazY Puppy
RSH's DNA
SARA B.
Back to top
View user's profile Send private message 
greengeek

Joined: 20 Jul 2010
Posts: 2542
Location: New Zealand

PostPosted: Wed 27 Nov 2013, 02:02    Post subject:  

RSH wrote:
Would it be possible to modify this for DE users?

I do know how to make a special dictionary file for specific commands in English, but I don't know if the same method will work for the German language.

If you could provide me with a DE translation of the following commands I will make a small DE trial dictionary and see if it works:

CENTERCLICK
CLICK
CLICKHOLD
DOUBLECLICK
RELEASE
RIGHTCLICK

Maybe we can teach it some German!

Basically this method would be trying to 'fool' the voice recognition into learning some new, previously unknown (but actually German) words. It might work so I will give it a try. (Unfortunately I do not speak German so even if I teach it German words I might be doing so in a hideous Americo-Kiwi-Germanic accent that destroys the ears...)

The voice control program used in voxpup is called 'pocketsphinx_continuous' which was offered as a .pet by technosaurus here:
http://www.murga-linux.com/puppy/viewtopic.php?t=88095&start=27

As far as totally translating the vocal recognition capability of pocketsphinx to a different language is concerned - I do not know if that will be possible. I do know that they are working on extending the language capability but I do not know how far they have got.

see here for a comment about pocketsphinx language modules:
http://www.speech.cs.cmu.edu/tools/lmtool.html
(NOTE: when pocketsphinx tutorials talk about 'language modules' they really mean 'grammar modules' or 'dictionary files' - they don't really refer to the ability to speak other languages).

When I was building my English trial dictionaries I found a LOT of confusing information on the internet (most of it referring to older methods of getting pocketsphinx set up). In the end I had better success doing it my own way. (maybe I just didn't understand the other ways...)

There is quite a bit of info regarding attempts to get German modules running, but maybe these methods are confusing dead ends, I don't know. Anyway, here are some DE links:
http://cmusphinx.sourceforge.net/wiki/sphinx4:germanacousticmodel
http://www.voxforge.org/de/forums
http://www.voxforge.org/home/forums/other-languages/german/testing-german-with-pocketsphinx-hardly-no-hits/13

Approx 1/3 of the way down this page:
http://cmusphinx.sourceforge.net/wiki/faq
I did find the following quote which suggests that German should be possible:
voxpupGerman.jpg
 Description   
 Filesize   100.35 KB
 Viewed   695 Time(s)

voxpupGerman.jpg

Back to top
View user's profile Send private message 
greengeek

Joined: 20 Jul 2010
Posts: 2542
Location: New Zealand

PostPosted: Thu 28 Nov 2013, 05:10    Post subject:  

Hi RSH - I have tried using google translate to get a German translation of the mouse commands I listed above and I have made a .pet based on these commands. It seems to work well, but obviously my accent will not be authentic so I really need a German tester to confirm.

Do these translations seem ok as a starting point?

CENTERCLICK - zentrum klick
CLICK - klicken
CLICKHOLD - klicken Sie auf hold
DOUBLECLICK - klicken Sie doppelt
RELEASE - Freisetzung
RIGHTCLICK - Rechtsklick

Would you have time to test the .pet for me?

gg
Back to top
View user's profile Send private message 
RSH


Joined: 05 Sep 2011
Posts: 2420
Location: Germany

PostPosted: Thu 28 Nov 2013, 12:34    Post subject:  

Quote:
Would you have time to test the .pet for me?

Hi greengeek.

Thanks for all the information in above posting.

Of course I will have some time to do some testings.

But first, let's check, if I do get the your commands used in the right direction:

CENTERCLICK --> clicking the centered/middle button of the mouse
CLICK --> clicking the left button of the mouse
CLICKHOLD --> clicking the left button of the mouse and hold it pressed down
DOUBLECLICK --> double-clicking the left button of the mouse
RELEASE --> (left) button of the mouse not longer pressed down
RIGHTCLICK --> clicking the right button of the mouse

If the above was explained (how I do understand this) the right way, I would use German commands as listed below.

Usually we would translate such EN text into full sentences, e.g.:

CENTERCLICK --> drücken Sie die mittlere Maustaste
CLICK --> drücken Sie die linke Maustaste
CLICKHOLD --> klicken und halten Sie die linke Maustatste gedrückt
DOUBLECLICK --> klicken Sie die linke Maustatste doppelt
RELEASE --> lassen Sie die linke Maustaste los (or: linke Maustaste loslassen)
RIGHTCLICK --> klicken Sie die rechte Maustaste

But we would need some kind of short and unique commands, right?

CENTERCLICK --> Maus-Mittelklick
CLICK --> Maus-Linksklick
CLICKHOLD --> Maus-Linksklick-halten
DOUBLECLICK --> Maus-Doppelklick
RELEASE --> Mausklick-freigeben
RIGHTCLICK --> Maus-Rechtsklick

or even

CENTERCLICK --> Mittelklick
CLICK --> Linksklick
CLICKHOLD --> Links-Dauerklick
DOUBLECLICK --> Doppelklick
RELEASE --> Dauerklick-Ende (or: Freigeben, or: Loslassen)
RIGHTCLICK --> Rechtsklick

Just one question: what .pet are you referring to?

_________________
LazY Puppy
RSH's DNA
SARA B.
Back to top
View user's profile Send private message 
greengeek

Joined: 20 Jul 2010
Posts: 2542
Location: New Zealand

PostPosted: Thu 28 Nov 2013, 14:09    Post subject:
Subject description: Test pet only for RSH trials with DE
 

**** I don't think I made it clear that this DE test with RSH is only using a small portion of the voxpup commands - it is only testing mouse click functions. If this test works then maybe it can be expanded ****

*** Ignore these tests if you do not use DE *****

RSH wrote:
But first, let's check, if I do get the your commands used in the right direction:

CENTERCLICK --> clicking the centered/middle button of the mouse
CLICK --> clicking the left button of the mouse
CLICKHOLD --> clicking the left button of the mouse and hold it pressed down
DOUBLECLICK --> double-clicking the left button of the mouse
RELEASE --> (left) button of the mouse not longer pressed down
RIGHTCLICK --> clicking the right button of the mouse

Yes, that is the correct understanding.

Quote:
Just one question: what .pet are you referring to?

The pet I mentioned is one I was testing last night to see if I could "trick" the voice control into recognising German words. It does work ok, but I used my 'google translate' words which are not quite the same as the phrases you suggested. If you want to try my first attempt at this .pet it is available here:

http://www.mediafire.com/download/u9s0vakud2po41j/voxclick004DE.pet
Or, if you want an sfs (which I think you normally prefer...) try this:
http://www.mediafire.com/download/rwkko582mo5oceo/voxclick004DE_3.9.9.2.sfs (It was made using Upup 3.9.9.2 so I have no idea if it will suit your system - I tried it on ThinSlacko and it worked fine...)

Best to use the pet or sfs with another puppy, not voxpup - otherwise it will conflict with the programs already running in voxpup. I recommend trying in a fresh install or Live session without savefile. Load the pet then restart x.
(Should be straightforward but let me know if problems...)

NOTE: on some systems a mouse freeze might occur but this is simply fixed by a change to the /root/.jwmrcpersonal file to change 'window move' and 'resize' modes to 'opaque' (This was the original reason why I released voxpup as a complete OS rather than just a pet or sfs... The full voxpup is already configured to avoid this)


Quote:
Usually we would translate such EN text into full sentences, e.g.:

CENTERCLICK --> drücken Sie die mittlere Maustaste
CLICK --> drücken Sie die linke Maustaste
CLICKHOLD --> klicken und halten Sie die linke Maustatste gedrückt
DOUBLECLICK --> klicken Sie die linke Maustatste doppelt
RELEASE --> lassen Sie die linke Maustaste los (or: linke Maustaste loslassen)
RIGHTCLICK --> klicken Sie die rechte Maustaste
I assume that these sentences would be too long to be used all day by the user? You would get lip fatigue if I use these sentences in the program yes? Smile

Quote:
But we would need some kind of short and unique commands, right?

CENTERCLICK --> Maus-Mittelklick
CLICK --> Maus-Linksklick
CLICKHOLD --> Maus-Linksklick-halten
DOUBLECLICK --> Maus-Doppelklick
RELEASE --> Mausklick-freigeben
RIGHTCLICK --> Maus-Rechtsklick

or even

CENTERCLICK --> Mittelklick
CLICK --> Linksklick
CLICKHOLD --> Links-Dauerklick
DOUBLECLICK --> Doppelklick
RELEASE --> Dauerklick-Ende (or: Freigeben, or: Loslassen)
RIGHTCLICK --> Rechtsklick


Yes, those look like good options - thanks. If you find that my test pet works I can then change the keywords to match your suggested words.

The pocketsphinx software has an interesting characteristic - it is more likely to give an incorrect decode if the command is too short - so there is a benefit in using longer phrases. However there has to be a balance between long phrases that are good for decoding, but short enough phrases that the user can actually be productive.

Last edited by greengeek on Tue 03 Dec 2013, 19:23; edited 1 time in total
Back to top
View user's profile Send private message 
RSH


Joined: 05 Sep 2011
Posts: 2420
Location: Germany

PostPosted: Sun 01 Dec 2013, 11:00    Post subject:  

Quote:
If you want to try my first attempt at this .pet it is available here:

http://www.mediafire.com/download/u9s0vakud2po41j/voxclick004DE.pet
Or, if you want an sfs (which I think you normally prefer...) try this:
http://www.mediafire.com/download/rwkko582mo5oceo/voxclick004DE_3.9.9.2.sfs (It was made using Upup 3.9.9.2 so I have no idea if it will suit your system - I tried it on ThinSlacko and it worked fine...)

Thanks for the Links.

Yes, I do prefer SFS Modules for the use of programs.

Though, in fact of that I want to have DE menu entries existing for each application, I do manually edit each sort of a package and rebuild everything to a full featured SFS P.L.U.S. SFS Module (those with the 'LP2_' Prefix).

Ok, downloaded.

Will return after doing some testings - at first I have to set up my computer...

RSH

_________________
LazY Puppy
RSH's DNA
SARA B.
Back to top
View user's profile Send private message 
greengeek

Joined: 20 Jul 2010
Posts: 2542
Location: New Zealand

PostPosted: Sun 01 Dec 2013, 14:18    Post subject:  

Just for interest - here are some links regarding the current state of voice controlled computing:

http://info.legalsolutions.thomsonreuters.com/signup/newsletters/practice-innovations/2013-jan/article4.aspx
(commentary on overall options/platforms).

http://theubi.com/
(interesting device).

http://www.e-speaking.com/free_pc_voice_control.htm
(Windows based. Free for 30 days, then only $14 to purchase).
.
Back to top
View user's profile Send private message 
greengeek

Joined: 20 Jul 2010
Posts: 2542
Location: New Zealand

PostPosted: Wed 11 Dec 2013, 12:12    Post subject:  

samstake wrote:
I try e speaking what do you think which one is more better for vice control..?
I have not tried espeak - I just included the link as an example of other voice control programs that are available.

I think voxpup is better because it is free Smile

Voxpup could also have it's functionality extended by the modification of the control script (voxmain) or the addition of other subscripts. I don't know if espeak allows the user to do that.

Plus epeak runs on Windows - voxpup uses puppy linux Cool
Back to top
View user's profile Send private message 
puppyluvr


Joined: 06 Jan 2008
Posts: 3200
Location: Chickasha Oklahoma

PostPosted: Wed 25 Dec 2013, 19:55    Post subject:  

Very Happy
Hello,
IDK about ESpeaking above, but espeak is linux and I use it on my puppies.

_________________
Close the Windows, and open your eyes, to a whole new world
http://puppylinuxstuff.meownplanet.net/puppyluvr/
Puppy Linux Users Group on Facebook

Puppy since 2.15CE...
Back to top
View user's profile Send private message Visit poster's website 
Display posts from previous:   Sort by:   
Page 1 of 3 [35 Posts]   Goto page: 1, 2, 3 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Puppy Derivatives
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1223s ][ Queries: 12 (0.0035s) ][ GZIP on ]