Command Visio with Your Voice!

Started by Visio Guy, March 08, 2016, 01:09:16 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Visio Guy

I was messing around with Visio automation, and created a VSTO add-in that does some fancy stuff using voice commands.

Speech Commands for Visio Proof-of-concept
https://www.youtube.com/watch?v=A8RsnoPOk34

I'm sure there's a whole bunch of accessibility stuff built into Windows and Office that could do a lot, but none of that stuff would do the very customized operations that I demonstrate in the Visio.

Anyway, I've not finished the editing, nor produced a formal article to go with this yet, but I hope it's good food-for-thought and gets your cerebral juices flowing!
For articles, tips and free content, see the Visio Guy Website at http://www.visguy.com
Get my Visio Book! Using Microsoft Visio 2010

JohnGoldsmith

Very nice Chris.  I particularly like the combined centering. 

I wonder if you could implement a verb-noun grammar combination like PowerShell?  Or maybe target-action, where your targets could be things like Document, Page or Selection and actions could map to DoCmds (names rather than numbers).

The possibilities are endless :)

Best regards

John
John Goldsmith - Visio MVP
http://visualsignals.typepad.co.uk/

Visio Guy

#2
What does PowerShell have to do with this? I still don't get it. (you know I'm just whining/whinging :) :) )

Yeah, I tried to make phrases that wouldn't occur so much, hence "ok ellipse", but since my "grammar" was so small, it would assume "ellipse" was close enough to "ok ellipse" and draw it anyway.

So I think your noun-verb thing might make more sense - kind of leads you to states. As soon as it hears "OK", then it is open to process the next command. I didn't code a hierarchy. Similarly, you couldn't do any verb until some sort of "noun state" had been set up first, and that it made sense with the following verb.

Kind of fun to think about.

What scares me is what might already exist in the accessibility or Cortana stuff that I'm re-inventing. Of course the functions I am ultimately running won't exist in any product from Microsoft, because it's custom, Visio-nerd code, so in the end, I suppose that's the real value.

I honestly think that the ShapeSheet stuff would save me the most time, because I can hook up all of my techniques to various words, and none of it makes any sense unless a ShapeSheet cell is selected. And based on where that cell is, lots of stuff wouldn't make sense as well, so you shouldn't get too many crazy accidental behaviors while mumbling to the kids.
For articles, tips and free content, see the Visio Guy Website at http://www.visguy.com
Get my Visio Book! Using Microsoft Visio 2010

JohnGoldsmith

#3
I'm just refering to the PowerShell's verb-noun cmdlet naming convention.  So for example: "Drop-Master".

Does the recognition engine support some kind of "sub" grammar?  To continue the PowerShell analogy, you have a verb-noun command and then if a command is recognised, then you go on to listen for parameters.  Something like:

case "Drop-Master":
   //Now recognise 'Name-SomeMasterName'

I think the grammar for Visio would be unique enough that the buitlin support wouldn't do the job properly.

I'm looking forawrd to relieving my rsi once you're at RTM :)
John Goldsmith - Visio MVP
http://visualsignals.typepad.co.uk/

Visio Guy

Call me a heathen (HEATHEN!),

but every time I try to learn something about PowerShell, the, say, video starts off with some command line interface, where the guy spends ten minutes importing stuff...EVERYTIME HE WANTS TO RUN A COMMAND.

Import LibraryA
Import LibraryB
Import LibraryC
Screen.Print "Bob"
Bob

Wow! Productivity.

1. Show me a form
2. Click the button
3. Show me the code behind the button and quit wasting my time.

:)
For articles, tips and free content, see the Visio Guy Website at http://www.visguy.com
Get my Visio Book! Using Microsoft Visio 2010

Yacine

So pity, I've not arrived at Cortana yet. Could have played with the concept.
Yacine

VisioDeveloper

#6
This is one of those things that i know it's possible but it still amazes me seeing it in action :)

This is something i will definitely take a look at!

Edit:

Let's for a second imagine this combined with numbered shapes. In a PoC we could do something like:

"Visio, Select 1"
"Visio, Paint red"
"Visio, Drop Rectangle Right of 1" ("Right of 1" being an optional parameter)
"Visio, connect 1 to 2"


i want to see some jaws drop in a demo i need to do  :)

Visisthebest

Visio Guy this is still an incredibly cool idea, did you progress it further?
Visio 2021 Professional

wapperdude

#8
Talk about gargantuan of major projects!!!  😱😵😬

👂👀👻.  Visio is listening.   .... and in what language???*

> " Visio..."
>>> "Yes Wapperdude, how may I help you?"
> "Please deactivate voice input mode."
>>> "Sorry, command not understood.  It sounded like you want all personal data permanently deleted.  Processing..."

Oh, got ahead of myself, there's no AI included with this.  😁😂😁😉😌

*: machine language of course.

Visio 2019 Pro

Visisthebest

Is there not a good cloud solution that has good .NET libraries for use in a Visio VSTO addin? I'd prefer Cortana but can't find good uptodate info about custom business solutions in a .NET desktop app with Cortana unfortunately, seems like Microsoft dropped this:
https://docs.microsoft.com/en-us/windows/configuration/cortana-at-work/cortana-at-work-voice-commands

"This content applies to Cortana in versions 1909 and earlier, but will not be available in future releases."
Visio 2021 Professional