Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Thu 18 Sep 2014, 20:11
All times are UTC - 4
 Forum index » Advanced Topics » Puppy Projects
Need on-screen keyboard with mouse functions
Moderators: Flash, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 9 [134 Posts]   Goto page: 1, 2, 3, ..., 7, 8, 9 Next
Author Message
standard7452

Joined: 25 Feb 2013
Posts: 15

PostPosted: Mon 25 Feb 2013, 19:59    Post subject:  Need on-screen keyboard with mouse functions
Subject description: Puppy for the severely disabled
 

Hello, my name is Will and apologize in advance if this is not in the right part of the forum.

I pored over this to try to make it as concise as possible, as I do not want anyone to tl;dr it.

I am looking for help with crafting a solution for a few friends and people I volunteer my time for who have no use of their arms and legs.

These folks are in wheelchairs and interact with their environment on a "single switch" basis, which means they can use their heads or chins to press a button.

Solutions exist for Windows that allow for them to interact with the computer, but the software and hardware costs are astronomical.

My goal is to give people who can only access the computer with a single switch (quadriplegics/people with cerebral palsy/motor control skills) a computer of their own that they can take with them, so that they can communicate with their friends and family both online, and when they go home via the PC.

Sounds like I am dreaming I know, but after 3 years of working with these folks, I have determined what I need to help them.

I need an on screen keyboard that also has the ability to manipulate the mouse.

I hate to ask for help, but I really need it. I am not a programmer, but I am willing to learn whatever I need to.

To make it short, I need an onscreen keyboard (like onboard) that also has mouse functions (like GOK).

I have found the xdotool command to move the mouse appropriately, but I have no idea how to integrate everything together.

I am probably stupid, and someone could probably look at this and give me a 5 minute solution.

I just don't know where to start or who to ask.

Thanks for your time.
Back to top
View user's profile Send private message 
greengeek

Joined: 20 Jul 2010
Posts: 2542
Location: New Zealand

PostPosted: Mon 04 Mar 2013, 22:39    Post subject:  

Would they have the ability/dexterity to distinguish between a short press and long press of the button?
Back to top
View user's profile Send private message 
Ted Dog


Joined: 13 Sep 2005
Posts: 2330
Location: Heart of Texas

PostPosted: Tue 05 Mar 2013, 00:50    Post subject: Can they talk?  

There are speech controlled software for computers, saw some pets I think MU was working on.. Also would be interested in helping you if special drivers or complex open source compiling is needed. As a person with a slow form of Muscular Dystrophy myself, I'm constantly displeased with the extreme high cost and lack of quality hardware. There is a brand of joystick (thrust master i think) that can be programed as a keyboard and mouse and cost hundreds of dollars less.
I'm thinking a keyboard overlay with a hot zone for switching between mousemode(as a joystick) and keyboard mode, should be due-able. There are on screen keyboards that come with http://www.mythbuntu.org/ by default try that out and see if that type of onscreen keyboard would work,
Back to top
View user's profile Send private message 
Ted Dog


Joined: 13 Sep 2005
Posts: 2330
Location: Heart of Texas

PostPosted: Tue 05 Mar 2013, 01:36    Post subject: GOK with percise puppy  

GOK is availble for Ubuntu apt-get GOK, so it would be available for precise-puppy new version is in Beta testing phase. Also, since I'm in the MDA program already, if you would give me models/controls you already work with I could get the same, if available, from my local support network.
Back to top
View user's profile Send private message 
mikeslr


Joined: 16 Jun 2008
Posts: 781
Location: Union New Jersey USA

PostPosted: Tue 05 Mar 2013, 01:54    Post subject: Onboard at least can be created for one Pup, maybe others  

Hi stardard7452,

Hoping to attract more attention, I posted on the "users" subforum: http://murga-linux.com/puppy/viewtopic.php?p=689498#689498. And got to thinking, and tinkering. So I know that Onboard, at least, can be built to run on LxPup-aptget-test. See the following link for the process for creating an SFS --a squash file you load/unload rather than install. http://murga-linux.com/puppy/viewtopic.php?p=689611#689611

I also provide instructions for creating (I hope) an installable pet. My guess is that it should work on any precise-pups to which python has been installed. First install python and then install an onboard pet or load an onboard sfs which will overwrite any conflicting python libraries. Also, now that we know what debs are necessary, once python is installed, it should be possible to download the Maverick version of those debs and create a pet or sfs which would run under Lupu/Lucids. But see below, and note that precise-pangolin pets/sfses are not always backward compatible: won't run on Lupu/Lucid --different libraries were used.

Regarding gok, I'll take a stab at it, but hope someone who actually knows what he's doing will also. I couldn't find a listing for gok for Ubuntu pangolin, only for Ubuntu Maverick. While I mention that my experience is that apps built for Lupu/Lucid --which correspond more or less to Maverick-- will almost alway run in a precise-pup, apps built for Maverick don't always run under Lupu/Lucids. Only about 2/3 the binaries used in building Lupu/Lucid were from Ubuntu. And sometimes, even though something may not appear on the dependency list of a Ubuntu app, such lists presumes certain base files built into Ubuntu which may not have been included in pups: i.e., Ubuntu presumes a full gnome environment, which Pups don't use because of its bloat. But one never knows until one tries.

Good luck on your project. Don't hesitate to email me if the instructions on the above post need further explanation.

mikesLr
Back to top
View user's profile Send private message 
8-bit


Joined: 03 Apr 2007
Posts: 3368
Location: Oregon

PostPosted: Tue 05 Mar 2013, 12:13    Post subject:
Subject description: Puppy files found
 

See my post here for a link to another post containing an on screen keyboard PET for Puppy.
Back to top
View user's profile Send private message 
mikeslr


Joined: 16 Jun 2008
Posts: 781
Location: Union New Jersey USA

PostPosted: Tue 05 Mar 2013, 12:42    Post subject: Please continue discussions HERE  

Hi All,

Nice work 8-bit. Hope it suites standard7452's needs, at least in part.

I found Ubuntu's repo listing for gok under pangolin. https://launchpad.net/ubuntu/precise/i386/gok/2.30.0-1ubuntu1 Suggest we concentrate on it as it was what standard7452 actually said he needed.

Sorry to report Onboard wasn't as successful as I hoped. Although it works fine under jejy69's lxpup-aptget-test, the SFS created under it did not run in pemasu's upup372 even though it had python installed. Starting it from a terminal at /usr/bin indicated that "gi.repository" could not be found. Googling revealed that gi.repository was something like --sorry it was late-- an experimental now being combined under something else. Pfind "python" in lxpup...test showed over 500 files> too many to manually copy into a package for use in other Pups. Typing python in synaptic generated a list which ran off the page, maybe thirty. Doable, but the description of many on the list suggests they wouldn't be needed or duplicate others on the list. Really needs someone who understands python.

mikesLr
Back to top
View user's profile Send private message 
Ted Dog


Joined: 13 Sep 2005
Posts: 2330
Location: Heart of Texas

PostPosted: Tue 05 Mar 2013, 16:40    Post subject:  

I fell asleep getting the mouse camera sofware to work,

Python was never a strong point since I never valued its appeal as yet another silly hack crutch that adds bloat on bloat.

But I have the source code and the dependencies are thankfully less than ten. Gnome is not even required, just gtk. so this is hopeful together with 8bits onsreen keyboard would make puppy more appealing to disabled folks.
Back to top
View user's profile Send private message 
Keef


Joined: 20 Dec 2007
Posts: 628
Location: Staffordshire

PostPosted: Tue 05 Mar 2013, 17:47    Post subject:  

Another onscreen keyboard option is WriteAid
It needs Java, but I've tried it with Shinobar's JRE sfs (jre-1.7.0.09.sfs) and it is working well in Wary.
Also has a 'rolling' mode, so if someone has a problem using the mouse, it will scroll through all the letters, and they can be selected with any key press. The speed can be altered.
The selected text appears in the bottom window, but is not very big though - no option to increase the size.
WriteAid.jpg
 Description   
 Filesize   18.71 KB
 Viewed   1946 Time(s)

WriteAid.jpg

Back to top
View user's profile Send private message 
mikeslr


Joined: 16 Jun 2008
Posts: 781
Location: Union New Jersey USA

PostPosted: Sat 09 Mar 2013, 12:38    Post subject: Xdotools to simulate mouse
Subject description: GTK conflict attempting GOK under Puppy
 

Hi standard7452 and All;

Spent a couple of days trying to jury-rig gok into Puppy's environment. I can't do it, and I suspect it can't be done after having built both an SFS using pangoln required debs and another using the maverick required debs. When both sfses loaded but did not run, I typed "gok" in a terminal at /usr/bin --where its executable is located-- and received a report of a conflict in gtk's. I think ubuntu switched over to gtk+ 3, while Puppy is still using gtk +2.

With a couple of possibilities having been suggested to provide an onscreen keyboard, I wondered if there was a way to simulate mouse movements using such keyboard. Googling I discovered xdotools. From what I gather, xdotools uses X itself. So it should be, more or less, distro independent. To install in jejy69's precise-apttest required download of only two debs. The following is the link to Ubuntu pangolin's repo.
http://packages.ubuntu.com/precise/xdotool
According to above, libc6 is required. But Puppy already has it so I'd first test for compatibility before installing ubuntu's, especially as ubuntu's libc6 itself has dependenceis, which may have dependencies and so on until you end up with having built Ubuntu itself.
More information and download link can be found here:
http://ashu-geek.blogspot.com/2012/01/how-to-install-configure-xdotool-on.html
I couldn't figure out how to configure it, but then I never use hot-keys or similar. But the following is documentation for it:

http://www.semicomplete.com/projects/xdotool/xdotool.xhtml

And apparently there's a gui making it easy to configure.

http://xdotool-gui.sourceforge.net/

I regret that I could not figure out how to upload the debs to the forum.

As muggins's virtual keyboard, http://www.murga-linux.com/puppy/viewtopic.php?p=261712#261712 suggested by 8-bit also is for the X environment (ergo distro independent) and provides a keyboard with numeric keypad, function keys, and special keys, I suggest an experiment of a combination of it and xdotool. Again, my apologies for not having the expertise to conduct such experiment.


mikesLr
Back to top
View user's profile Send private message 
greengeek

Joined: 20 Jul 2010
Posts: 2542
Location: New Zealand

PostPosted: Sat 09 Mar 2013, 14:22    Post subject:  

I got the impression from standard's post that the user would not be able to make keyboard taps so I think that means xdotool may not be a usable option. I think that the "rolling mode" keef mentions would probably be a pre-requisite if the user can only press a separate single button.

If they could use a joystick there would be lots more possibilities but I got the feeling that's not possible. I hope we do hear back from standard as this seems like a very valuable project. I'm still wondering if the user would be able to control "press duration" of the button as that might open some other options.
Back to top
View user's profile Send private message 
mikeslr


Joined: 16 Jun 2008
Posts: 781
Location: Union New Jersey USA

PostPosted: Sat 09 Mar 2013, 15:36    Post subject: Explanation of Xdotools employment  

Hi greengeek,

You may be right that physical limitations could inhibit the use of xdotool's full capabilities. But what I had in mind was PRE-CONFIGURED pet assigning to xvkbd's Function and other special keys mouse movements generated by arguments assigned through xdotools. Perhaps not perfect, and certainly looses xdotool's flexible customization. But, if possible, and someone were to built such pet, it would thereafter be easy to deploy, and anyone limited to single switch capabilities would only have to learn which special keys produced which mouse movements. Additionally, with the wealth of special keys available via xvkbd's GUI, the need for mouse movements could be limited. For example, a key to expand the menu, menu scroll keys up, down, left & right, and perhaps keys to place the cursor at the start of the address box of a browser, and the start of the search box of a search engine.
Actually, I just realized that the last two suggestions aren't necessary. I've gotten so used to relying on a mouse that I forgot --just tested-- that at least in opera and firefox the tab key will move thru webpage elements. Perhaps other "Standard" applications also "retain" such tab functionality.

By the way, unless I'm misrecollecting or misunderstood, GOK is structured to use 2 mice so may, in fact, present problems for those limited to single switch capabilities.

mikesLr
Back to top
View user's profile Send private message 
greengeek

Joined: 20 Jul 2010
Posts: 2542
Location: New Zealand

PostPosted: Sat 09 Mar 2013, 15:52    Post subject:  

I'm thinking something like this Magoo pet:
http://www.murga-linux.com/puppy/viewtopic.php?p=668558#668558
combined with WriteAid in rolling mode might somehow be configurable to offer reasonable access to basic functionality through a single switch.

I'm trying to get java installed so I can try WriteAid, buy not having much success yet. (I usually prefer to stay away from java due to security issues but this seems like a worthy project...)

I guess the basic screen would have to scroll through the Magoo tabs (allowing the user to click on the preferred application to open), then WriteAid (or similar) would have to scroll through the tabs within the app, and bring up a keyboard for scrolling through characters to be entered. I wonder if there is an onscreen keyboard that scrolls through an app list as well as keyboard characters?

I wonder if there is some way to set up xdotool so that it keeps scrolling through the Magoo tabs as soon as the PC powers up. Then maybe a single click of a button (grafted into mouse button 1) would activate the chosen tab as it became "scroll focused".

(question: would xdotool interpret WriteAid key depressions the same way it would interpret actual keyboard depressions? eg: if so then xdotool could be configured to interpret some of the WriteAid onscreen characters ("non A-Z" and "non numeric") as up/down/R/L mouse movements)

Last edited by greengeek on Sat 09 Mar 2013, 21:03; edited 3 times in total
Back to top
View user's profile Send private message 
8-bit


Joined: 03 Apr 2007
Posts: 3368
Location: Oregon

PostPosted: Sat 09 Mar 2013, 15:53    Post subject:  

It would be nice if stardard7452 checked back in so we would know our efforts are being seen and this is just not a futile effort.
Back to top
View user's profile Send private message 
mikeslr


Joined: 16 Jun 2008
Posts: 781
Location: Union New Jersey USA

PostPosted: Sat 09 Mar 2013, 21:35    Post subject: Sent email to xdotools group  

Hi All,

Patience 8-bit. Sad to say but some people have a real life which from time to time interferes with Puppy activity.

greengeek raised a good point, especially as I have been making suggestions without solid experiential support. So I've emailed the xdotools usergroup the following:

"Hello everyone.

First off, I'm way over my head but have gotten involved in a worthwhile project: enabling those with cerebral palsy, quadriplegics or otherwise limited to "single switch" mobility to use computers. The project was suggested by Will, known on the Puppy Linux Forum as standard7452. The thread he initiated, currently only 11 posts, can be quickly read here: http://murga-linux.com/puppy/viewtopic.php?t=84617. There are a couple more posts on a thread I opened hoping to attract more technically astute users of Puppy Linux. Although we have identified several onscreen keyboards which could be employed, emulation of mouse functions or a work-around remain elusive. I had suggested that perhaps xdotools might provide such functionality, http://murga-linux.com/puppy/viewtopic.php?p=690708#690708, in combination with Xvkbd: Virtual keyboard, http://www.murga-linux.com/puppy/viewtopic.php?p=261712#261712 but greengeek --suggesting a different approach-- asked a question pertinent to both: whether xdotools would interpret the "pressing" of onscreen keys the same way as actual keyboard key depressions?
That prompted this inquiry to people who may know.
Please feel free to join the discussion on the Puppy Forum (registration required --everyone is welcome, but especially those with technical knowledge) or to respond to this email if for no other reason than to save us some time attempting the impossible. If you have alternative suggestions they would be welcomed as well.

Thanking you in advance for your courtesy and cooperation,

mike kessler"

mikesLr
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 9 [134 Posts]   Goto page: 1, 2, 3, ..., 7, 8, 9 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Puppy Projects
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1123s ][ Queries: 12 (0.0143s) ][ GZIP on ]