Need on-screen keyboard with mouse functions

A home for all kinds of Puppy related projects
Message
Author
User avatar
Ted Dog
Posts: 3965
Joined: Wed 14 Sep 2005, 02:35
Location: Heart of Texas

#21 Post by Ted Dog »

Ah, you got around the expensive switch, I'd like to see a video setup.

Also do you know if you keep a lazer mouse still and just move the pad its the same and moving the mouse, would a upside down mouse and a padded semi ridged chin strip made of mousepad work. Also the mouse buttons are easier to reuse over a keyboard.
Just a thought.

I asked my local chapter of the MDA if there was any interest in a project like this... She was not expecting this type of topic so she will get back with me in a few weeks, after the Shamrock for MDA fund drive is over. :wink:

While on the topic, MDA fund drive is underway, please donate,
MDA saved my life, I was going down hill with a form of MD that is very rare. Normal doctors where unable to classify or treat my condition for 12 yrs. When I visited the MDA clinic they where able to id the problem and I had a workable prescription before I left, also since it was rare they had me stay the rest of the day and be seen by everybody in the department and a few students.
If I was seen by that many doctors and had all the expensive tests run it would have easily been 10,000 dollars, my cost $86 rest covered my MDA.

starhawk
Posts: 4906
Joined: Mon 22 Nov 2010, 06:04
Location: Everybody knows this is nowhere...

#22 Post by starhawk »

A thought that occurs to me --

Could these people manipulate a directional-pad sort of setup, like on a Game Boy? If so, a special switch (not expensive!), an Arduino Leonardo chip (the whole board is not necessary), and a PS/2 cable is all you'd need.

But they'd need to be able to set their chin on the switch actuator and then move the chin slightly to make the circuit do its thing. They'd also need to be able to move to a second and third switch for left- and right-click.

Directional switch like this --> http://www.sparkfun.com/products/10063
Other two switches as needed, maybe like this? --> http://www.sparkfun.com/products/9337
Arduino chip --> http://www.sparkfun.com/products/10524
28pin DIP socket (for the chip) --> http://www.sparkfun.com/products/7942
...as a note, that is an utterly atrocious price. Mouser Electronics sells those for less than $0.25! (But the added shipping cost isn't worth it.)

The Arduino would need to be programmed, using the ps2dev library here as a starting point --> http://playground.arduino.cc/componentLib/Ps2mouse

A further note -- the switch I mention is going to be a real doozy to solder, simply because it's surface-mount. Probably the best way to deal with this would be to use this "breakout board" that puts things into through-hole friendly form. You would not need to buy the switch separately -- the "breakout" board includes everything needed here. There is an additional "select" output on that switch which would not be used -- that can be implemented in other buttons. (This is likely to be far, far less frustrating than "press the button and push, but don't push too hard or you'll click on something!")

So, per device, you'd be looking at a cost of $12.50 plus shipping, labor, and whatever sort of PCB you'd want it mounted on -- Radio Shack actually has a pretty nifty "perfboard" selection (perfboard is a PCB with plated holes spaced at 0.1" intervals, primarily for prototyping and for hobbyists who can't afford custom PCBs)... I'd recommend their Catalog # 276-158 board, at about 2.8"x3.7". See here --> http://www.radioshack.com/product/index ... Id=2103798

I daresay that, given some experimentation and such, I could quite likely build one of these myself as a prototype. I don't think you'd want me building a lot of them for you (I've got motivation issues, among other things) but I'd be happy to assemble a prototype if I can scrape together the money...

starhawk
Posts: 4906
Joined: Mon 22 Nov 2010, 06:04
Location: Everybody knows this is nowhere...

#23 Post by starhawk »

I have a much better design for this. Let me know if you're interested, and if so, I'll send you the plans.

Total cost should be under $30 including all parts and shipping. (Labor is extra.) It is 100% workable, with all off-the-shelf components, and can be assembled by an experienced maker in probably about an hour. (That means it'll take me two hours at least!)

I still have to write the programming to make it work -- but it WILL work!

What's more -- if the system this is used on has an integrated keyboard (i.e. is a laptop), there is no need for any other input device so long as an onscreen keyboard is readily available! That is -- this can REPLACE the single-button keyboards entirely.

For systems that will not boot up without a keyboard present -- simply find a PS/2 keyboard, open it up, and pull out the electronics inside. You only need the PCB and PS/2 cable to remain intact -- discard the rest. Wrap it up tight with electrical tape, hook it up, and stash it somewhere. Presto! One keyboard that will never experience another keypress, but fool the computer into thinking all is well. Simple and brutal, but remarkably effective.

standard7452
Posts: 20
Joined: Mon 25 Feb 2013, 23:11

#24 Post by standard7452 »

greengeek wrote:
standard7452 wrote:if I could add six buttons to any onscreen keyboard that has the scanning feature mentioned above that ran a script for xdotool to move the mouse in a direction until the user hits the predefined physical keyboard button again. and also have a button for right, left and doubleclicks that would be it.
So is this similar to what you mean?

Programmatically assign the following keys with the attached functions as shown:
F4 = "slowly scroll left till user hits switch"
F5 = "slowly scroll up till user hits switch"
F6 = "slowly scroll down till user hits switch"
F7 = "slowly scroll right till user hits switch"
Yes! But, the keyboard has to be "scannable" as well so that the person can first select the f4 key using row, column deduction, and then the mouse would start, and then the mouse would stop when they hit the switch again.

Functionally, this is exactly what it would be with 3 other keys, left click, right click, and double click.

When I am back at work tomorrow (I got the stomach bug my kids got), I will take a video of someone using a super expensive solution just so you guys see what the end functionality is.

Thank you again for your help!

I will post the video tomorrow which I think will clear up the details of what I am looking to do (obviously with much help, ty)

Will

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

Re: Delay

#25 Post by greengeek »

standard7452 wrote:The fundamental problem I found with GOK is that the original developers assigned all of the scanning functionality to any one of the mouse buttons. This creates a conflict with the pointer, because every time you click to make it scan it also clicks wherever the pointer is.
I would just like to clarify this if possible - is the conflict caused because the pointer MOVES when you click the mouse button (because of the mechanical pressure on the mouse), or do you mean that the conflict is that you cannot trigger the scanning unless the mousepointer is in one correct place only? (and you want the mouse to be elsewhere at the moment you trigger scanning...)

standard7452
Posts: 20
Joined: Mon 25 Feb 2013, 23:11

Re: Delay

#26 Post by standard7452 »

greengeek wrote:
standard7452 wrote:The fundamental problem I found with GOK is that the original developers assigned all of the scanning functionality to any one of the mouse buttons. This creates a conflict with the pointer, because every time you click to make it scan it also clicks wherever the pointer is.
I would just like to clarify this if possible - is the conflict caused because the pointer MOVES when you click the mouse button (because of the mechanical pressure on the mouse), or do you mean that the conflict is that you cannot trigger the scanning unless the mousepointer is in one correct place only? (and you want the mouse to be elsewhere at the moment you trigger scanning...)
http://m.youtube.com/#/watch?v=Mn-HLmEP ... n-HLmEPI_s

This is a video of what I mean by "scanning" that uses the win7 OSK.

In the first part he uses the "Point and Click" method where he selects the character or key he wishes to use with the mouse pointer. In the second part he uses the "Scanning" mode to allow him to press a predefined button like "space" to get a row selected, and then three characters, then the final character the user wants.

GOK has a keyboard layout that included mouse functionality like you we're talking about with xdotool. There is buttons (like enter, space, letters, buttons), except they are for mouse up (xdotool mousemove_realtive 0,5 on some sort of loop) and mouse down, etc. these buttons allow for the user to access both the keyboard and mouse using a single switch and the deductive logic part.

GOK's conflict lies in the fact that they used a mouse button exclusively to initiate this scanning process. So if you want to move the mouse using the OSK with GOK, every time you want to change direction or open a context menu or whatever, the core pointer itself clicks.

[/url]

standard7452
Posts: 20
Joined: Mon 25 Feb 2013, 23:11

#27 Post by standard7452 »

starhawk wrote:A thought that occurs to me --

Could these people manipulate a directional-pad sort of setup, like on a Game Boy? If so, a special switch (not expensive!), an Arduino Leonardo chip (the whole board is not necessary), and a PS/2 cable is all you'd need.

But they'd need to be able to set their chin on the switch actuator and then move the chin slightly to make the circuit do its thing. They'd also need to be able to move to a second and third switch for left- and right-click.

Directional switch like this --> http://www.sparkfun.com/products/10063
Other two switches as needed, maybe like this? --> http://www.sparkfun.com/products/9337
Arduino chip --> http://www.sparkfun.com/products/10524
28pin DIP socket (for the chip) --> http://www.sparkfun.com/products/7942
...as a note, that is an utterly atrocious price. Mouser Electronics sells those for less than $0.25! (But the added shipping cost isn't worth it.)

The Arduino would need to be programmed, using the ps2dev library here as a starting point --> http://playground.arduino.cc/componentLib/Ps2mouse

A further note -- the switch I mention is going to be a real doozy to solder, simply because it's surface-mount. Probably the best way to deal with this would be to use this "breakout board" that puts things into through-hole friendly form. You would not need to buy the switch separately -- the "breakout" board includes everything needed here. There is an additional "select" output on that switch which would not be used -- that can be implemented in other buttons. (This is likely to be far, far less frustrating than "press the button and push, but don't push too hard or you'll click on something!")

So, per device, you'd be looking at a cost of $12.50 plus shipping, labor, and whatever sort of PCB you'd want it mounted on -- Radio Shack actually has a pretty nifty "perfboard" selection (perfboard is a PCB with plated holes spaced at 0.1" intervals, primarily for prototyping and for hobbyists who can't afford custom PCBs)... I'd recommend their Catalog # 276-158 board, at about 2.8"x3.7". See here --> http://www.radioshack.com/product/index ... Id=2103798

I daresay that, given some experimentation and such, I could quite likely build one of these myself as a prototype. I don't think you'd want me building a lot of them for you (I've got motivation issues, among other things) but I'd be happy to assemble a prototype if I can scrape together the money...
This sounds interesting! I need to take a video of how they access so that you could look...as far as prototypes I would fund it, but first take a look to at the vid to see if it is applicable....I should have it n tomorrow,

starhawk
Posts: 4906
Joined: Mon 22 Nov 2010, 06:04
Location: Everybody knows this is nowhere...

#28 Post by starhawk »

I eagerly await your video... ;)

Minus the programming part (which will be hard for me, but I think I can do it) the entire design is basically done. It uses parts from Mouser Electronics (www.mouser.com) and Radio Shack. There will be one part that must be scrounged (the PS/2 cable) but that's not a terribly big deal IMO. Nearly anyone should be able to build it in an afternoon -- so it doesn't need factories and mass-production, it can be built by hobbyists and makers the world over! (This also lowers the cost of things dramatically.)

I have a couple of things I'd want to talk to you about, to get the design spot-on, but at this point I'm just doing fine-tuning.

The problem with the programming is that I'm used to a very old programming language that the world has largely moved on from -- QuickBASIC -- and the Arduino uses a subset of C/C++, which is completely different. I'll have some learning to do!

I would like to open-source this design if at all possible... I'll be glad to assemble one or two for you, and we can figure out how to tweak them, but as I said above -- if you can hold a drill, and if you can solder, then you can build this thing yourself.

I will also mention -- if you need to use a USB-to-PS/2 adapter with this, for computers that do not have PS/2 ports, you need to steer clear of these two styles of adapter --
Passive, wires-only adapters, like this one.
Active but super-cheap adapters like this one.

The one in the first link won't work, and it could damage things. Those rely on the mouse being able to detect what kind of port it's attached to, and switch by itself. This device can't do that!

The one in the second link will "sort of" work -- those have HORRIBLE filtering (if any at all) in them -- I had one and it was so jittery that I just got rid of it. The mouse cursor dances a jig even when the mouse ISN'T moving, and when it is moving the dancing gets worse. By the way, these come in both beige and black -- so don't be fooled, lol.

I can't speak to how well the following adapters work, but money provided I'll gladly test them --
http://www.ebay.com/itm/280775192717
http://www.ebay.com/itm/111011014372

standard7452
Posts: 20
Joined: Mon 25 Feb 2013, 23:11

#29 Post by standard7452 »

I made a couple of videos to explain what the project entails...


...I apologize if I seem sassy on the video, I had to do it in the early morning, and I am not a morning person.

I just went over the basics, not because I think anyone is dumb, it just is such a weird thing to explain to folks....

Plus this is the first time I have ever made a vid like this so I didn't want to look too stupid....

Thanks for your help!

part one - http://www.youtube.com/watch?v=BaKmdhGexVk

part two - http://www.youtube.com/watch?v=6b857d5G2Qs

User avatar
Ted Dog
Posts: 3965
Joined: Wed 14 Sep 2005, 02:35
Location: Heart of Texas

#30 Post by Ted Dog »

Thanks, DL them now, internet is slow for me.... :(

User avatar
mikeslr
Posts: 3890
Joined: Mon 16 Jun 2008, 21:20
Location: 500 seconds from Sol

Rethinking the Keyboard for single-switch

#31 Post by mikeslr »

Hi All,
The following was written before I watched Will's video. But I think most of it holds up. So I'm submitting it "as is" or "was."
The discussion now appears to be entering into a technical area beyond my comfort zone. Except for checking in once and awhile, and seeing if I can recruit others having needed expertise, this may be my last post on this thread.
Although I have always been driven curiosity about everything, shortly after WWII, when I was about 6 and TVs had yet to become mainstream, I took apart the family's radio. No one knew how to fix it and its replacement cost about a week's savings. To say the discussion which followed was not an encouragement of my tinkering with appliances would be euphemistic.
Now that I better understand the problem, at present I have only one thing to add. It will probably complicate things.
We are creatures of habit. Much --perhaps most of what we do, or at least how we do things-- is the result of our internalization of our culture's adaption to its environment, including the existent technology. We learn it, do it, it becomes the way we do it and, thereafter, we don't think about it. Having habits is efficient as anyone will attest who has had to wait while someone else tries to locate keys or eye-glasses “just left someplace.
Attachments
Keyboard.jpg
Possible Keyboard Arrangement
(59.23 KiB) Downloaded 644 times

starhawk
Posts: 4906
Joined: Mon 22 Nov 2010, 06:04
Location: Everybody knows this is nowhere...

#32 Post by starhawk »

One more question before I give up on my hardware solution.

Can these people reliably tilt their chin forwards, backwards, left, and right, about a half-inch, without problems?

If this gets into pure software I'll have to bow out. I can probably learn arduino code, but even a linux shell script is likely beyond me right now (although I'll have a look if it becomes necessary).

EDIT: even tilting the chin 1/4 or 3/8 inch, would be enough... just need to be able to do it enough to bump into something is all.

standard7452
Posts: 20
Joined: Mon 25 Feb 2013, 23:11

#33 Post by standard7452 »

starhawk wrote:One more question before I give up on my hardware solution.

Can these people reliably tilt their chin forwards, backwards, left, and right, about a half-inch, without problems?

If this gets into pure software I'll have to bow out. I can probably learn arduino code, but even a linux shell script is likely beyond me right now (although I'll have a look if it becomes necessary).
Starhawk, I think your hardware solution sounds awesome so I don't want you to think your efforts were in vain for a second.

It's just that the guys who get the shaft on computer access only have a single motion (that isn't even always predictable). So the only way to delineate it is by steps--move mouse, click mouse, wait for A, etc.

Imagine putting a 10kg-20kg weight around your neck and you could only interact if you could put the part in the right place....and you are disabled.

Thank you so much for your help....no good deed goes unchecked.

starhawk
Posts: 4906
Joined: Mon 22 Nov 2010, 06:04
Location: Everybody knows this is nowhere...

#34 Post by starhawk »

OK :oops:

I'm sure someone will help you with a shell script, then. Not something I can really do.

User avatar
mikeslr
Posts: 3890
Joined: Mon 16 Jun 2008, 21:20
Location: 500 seconds from Sol

If the mouse offends thee, pluck it out

#35 Post by mikeslr »

Hi standard7452 and all,

Seems I was mistaken about having little more to add.

I previously indicated that I was able to obtain a functioning version of onboard in jejy69's lxpup-aptget-test by using its included synaptic to install the necessary debs. With onboard onscreen and geany opened clicking the onscreen keys sent the keys selected to geany which displayed them. I'm not certain, however, if onboard is entirely functional under lxpup-aptget. I called up its config module and turned on scanning. Thereafter, various sections of the keyboard were highlighted, but (1) selecting mouse as the keyboard device and checking "mouse only" froze the application except to allow pressing the Enter key to uncheck it and (2) after unchecking "mouse-only" clicking a key on the onscreen keyboard while in the scan mode no longer sent key-strokes to geany. Turning off that mode restored onboard’s communication with geany. Similarly, unless scanning was turned off, onboard did not communicate with either opera or firefox. Hopefully, that problem is simply because I don't really know how to configure onboard or can be overcome. Exploring onboard has overcome my concern that an on-screen keyboard would occasionally obscure other on-screen windows: its display can be dragged and resized so that it uses only space immediately above the taskbar. Although other windows may extend under that display, there's probably a workaround. Interestingly, when set to “grid
Attachments
WP51Dos.jpg
Wordperfect5.1 for dos running under dosemu
(124.23 KiB) Downloaded 659 times

User avatar
ETP
Posts: 1193
Joined: Tue 19 Oct 2010, 19:55
Location: UK

Need on-screen keyboard with mouse functions

#36 Post by ETP »

Hi Will,

I have found another free Windows webcam solution which, in parallel with this thread, you may want to check out. It looks really good, covering both mouse control and keyboard input purely with head/eye movements. Text to speech is also catered for.

http://www.cameramouse.org

http://www.staggeredspeech.org
Regards ETP
[url=http://tinyurl.com/pxzq8o9][img]https://s17.postimg.cc/tl19y14y7/You_Tube_signature80px.png[/img][/url]
[url=http://tinyurl.com/kennels2/]Kennels[/url]

User avatar
ETP
Posts: 1193
Joined: Tue 19 Oct 2010, 19:55
Location: UK

Wordperfect 5.1

#37 Post by ETP »

Hi mikeslr,

Your mention of WP 5.1 brought back a lot of memories. :D :D :D
I used it for years in conjunction with an add-on (The A-Z of WordPerfect) which made it easier to use than Word. It was in essence a collection of macros covering all features. I may still have a copy on one of my archive CDs. Let me know if you wish me to check.
Regards ETP
[url=http://tinyurl.com/pxzq8o9][img]https://s17.postimg.cc/tl19y14y7/You_Tube_signature80px.png[/img][/url]
[url=http://tinyurl.com/kennels2/]Kennels[/url]

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#38 Post by greengeek »

Hi standard7452,

forum member SFR has done some fantastic work developing a couple of scripts (attached below) that allow a Puppy to be fully controlled from a single keyboard switch. In their current form the scripts respond to the Pause/Break key that you have requested (and can initially be tested by anyone without having to disassemble the keyboard).

Unfortunately I have not used GOK or Onboard so I don't know if the script will suit your needs with those programs, but I have tested each of them with the xvkbd onscreen keyboard and the combination is very powerful.

The user only needs to run one of these scripts. Although they perform identical tasks they are visually different and it is up to the user to choose which best suits their needs/screen layout etc.

The scripts are as follows:

1) Radar6 places a small window at bottom right of the screen and contains a rotating line within a circle. The rotating line determines the direction that the mouse will travel, and the pause key determines the actions to be performed.

2) Navbar2 places a navigation bar across the top of the screen and uses 8 arrows to display the intended direction of mouse travel, and again, uses the pause key to control actions.

*** DO NOT RUN BOTH SCRIPTS AT THE SAME TIME ***

How does a single key control the actions? Both Radar and Navbar alternate between "choose direction" mode and "choose action" mode and the user simply presses the pause key when they see their preferred direction/action highlighted in the control window.

NOTE : SFR has done a lot of work to make these scripts perform well, but it is important to remember that there are many Pups, many browsers, and a variety of window managers to choose from. Your mileage may vary!! I have been testing with a small number of Pups, and will be testing more combinations, but my first recommendation is that Magoo (which is based on Precise) seems very compatible. There are screenshots below of Radar6 running on Magoo on my netbook. (obviously a bigger screen would suit most users...)

I will describe how I install Radar, but please note that I don't know a lot about the "right way" to do this. (I really don't know all the conventions about where things should go, and how they should be connected together, so I will describe what worked for me). I recommend that you try this script (or any new script for that matter...) on a fresh installation that does not contain important data. The script itself will not cause you any problems but if you are a bad driver and lose control of the mouse you have only youself to blame :-)
If you want to trial Navbar instead of Radar you simply follow the same install steps.

1) Download the attached Radar6 (or Navbar) tar and extract it somewhere. Then place a copy of the extracted folder in /root
2) Click on the radar file inside that folder. You should see the radar window appear at bottom right of the screen. (It should appear on top of all other windows, but if not you may need to shrink other windows enough for it to be always visible)
3) Click the Pause key and you should see the mouse start to move in the direction shown by the radar screen. When you get to where you want to go just hit the pause button again and the radar screen will change to show you the actions you can choose. Just cycle through this process as required
Navbar is similar, but places the control window at the top of the screen so that it doesnt overlap maximised windows below it. At this stage I find radar a little easier to drive than navbar - but both have their advantages.

xvkbd
xvkbd is not a requirement for Radar or Navbar, but it will be a valuable add-on for most users. If the user wishes to type into a document or browser etc you can have xvkbd running in the background, then navigate to the taskbar, select the xvkbd tab (which will bring xvkbd to the front) then click the "focus" key on the xvkbd keyboard, click on the chosen program window, then start typing the text on the keyboard and it will appear in the desired document window. I don't know how xvkbd compares with Onboard but it seems to work well and I found it quite easy to adapt to.

8bit mentioned this post which details the installation and use of xvkbd:
http://www.murga-linux.com/puppy/viewto ... 712#261712
I think the pet will install on most systems, but if not you can manually install the files: Basically you put the xvkbd file somewhere like /usr/bin and click on it when you want it running, or else you can put the file (or a symlink) into the /root/startup folder if you want xvkbd running as soon as Puppy is started.

Some systems also require the two extra libraries mentioned in that post to be added into /usr/X11R7/lib, and then it is necessary to add symlinks with the correct names that xvkbd is looking for. This only has to be done once.

Magoo : Magoo is not a requirement for Radar or Navbar - it just happens to be a Puppy that I like (targeted at vision-challenged or menu-challenged users) and it seems to work well with radar and navbar. It is available as an iso or a .pet
http://murga-linux.com/puppy/viewtopic. ... 472#671472

Now all that's required is to find someone to offer a service grafting 3.5mm mono connectors onto old keyboards so we can tap into the pause key with a footswitch/headswitch or whatever...
Attachments
Radar-0.6.tar.gz
(66.89 KiB) Downloaded 505 times
NavBar-0.2.tar.gz
(51.7 KiB) Downloaded 534 times
Radar6_Magoo_1_.jpg
(95.84 KiB) Downloaded 813 times
Radar6_Magoo_2_.jpg
(92.98 KiB) Downloaded 864 times

standard7452
Posts: 20
Joined: Mon 25 Feb 2013, 23:11

#39 Post by standard7452 »

Sorry I have not responded, I was out of town for a funeral...currently reading and looking at all this wonderful info!

standard7452
Posts: 20
Joined: Mon 25 Feb 2013, 23:11

#40 Post by standard7452 »

OMG!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

I am so excited this is exactly what I needed for this project.

SFR you have no idea how many people you will help with this...it is better than I could have imagined!

WOW, just WOW!

I will move on to testing with some of the people who will benefit and post results / videos.

And try to come up with a way to thank you all for your efforts!

Post Reply