The Illusive Complex Visio Webpage - Discussions

Started by gunslingor, November 20, 2012, 01:42:37 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

gunslingor

I am starting this thread to discuss Visio web developement without Sharepoint Visio services, from more of an advanced developer perspective than most similar discussions online.

The Problem:
You manage a giant network with, say, 1000 devices including servers, workstations, PLCs, DCS, SCADA, HMIs, media converters, etc, etc... You make your self a giant network diagram that accounts for every device and every device connection; the diagrams are large and complex, pushing the boundries of what Visio can handle. You attach all the shapes to a database backend that houses configuration data for each device, You either do this by hyperlinks or Visio data imports or another method. Now you need to go online with the Visio diagrams in uneditable form so all the system admins can use, but not change the diagram. Alas, Microsoft falls short of providing adequate web functionality and tries to force you to buy a new service, Visio Services.

Potential solutions:
Saving as a webpage is very limited. You can't specify specifics like maximum zoom level. Hyperlinks work in VML format only when going to another page; same page hyperlinks require refresh and will not zoom to the location of the linking shape making them useless (but these are needed due to the complexity of the drawings with on-page connectors run rampant). VML is also rumored to preserve layer data, which is definitely a desired functionality so users can view certain sections only of this complex drawing... but alas, again, this presents the problem that, to use the layer data on a webpage, you'd have to throw out MS's web interface and replace it with something custom...

Here is where I am looking to go. Forget Visio's save as webpage... make my own....but how? It appears to be near impossible to interface with the emz file (from the VML output). XAML is just as difficult. Any other file type will require months of preparation and learning, and no one know's how it will turn out. Has anyone else ever embarked on this journey? Any success stories? Should I move away from the webpage and try to create my on executable instead? But again, how do I interface with the visio File from an IDE? What are my options? Am I boned?

Discuss, Discuss, Discuss...



Jumpy

Just an idea that came to mind reading your thread:

You could try to create your on small application using the Visio (ActiveX) drawing control. With that you can do nearly everything you can do with Visio itself, but embedded in your application you could restrict (to the extreme) what the user can do. So you could create effectivly a: "MySimpleNetplanViewer" :-)

gunslingor

Sounds perfect to me... but correct me if I am wrong: I'd have to get my company to purchase Visual Studio and I'd have to brush up seriously on VB.NET. Or is there another free environment that will allow me to do this? The programming language is unimportant, I've used just about every major language at this point.

Dmitry

#3
My 2 cents:
1) I think that the only one giant diagram showing 1000s devices will be just useless, it's too much for human eye. It's more convenient to have multiple pages, starting from bird's-eye view down to department or rack level. Just for example you can take a look at some complex monitoring systems like DataMiner (http://www.skyline.be/Products/AboutDataMiner.htm), iControl (http://www.miranda.com/iControl%20Web/chapter107) or MS Data Center template (http://www.microsoft.com/en-us/download/details.aspx?id=26521). Views also can be spanned across multiple monitors.
2) Maybe there is no need to save diagrams as webpages at all - you can install Visio viewer (it's ActiveX control in fact) and you can use interactive features like multiple hyperlinks per single shape, context menus, zoom in/out (but not sure about VBA and add-ins - full version may be required).
3) If you need some special functions you could create macros and/or add-in, create ribbon with your own buttons. So instead of reinventing the wheel you can focus on functions that are specific to your solution, presentation layer may be completely based on Visio.
4) You can set read-only attribute to the file to avoid accidental changes.

gunslingor

The diagrams are for overall network security, so they do need to be large in general. As I understand it, Visio viewer and Visio drawing control (activeX) are two different things. Visio Viewer is obsolete, as it does not work in IE9 only IE8 and before... and only IE9 works on windows 7. I suspect the ActiveX control will work in IE9 if programmed from visual studio. I was trying to set it up in sharpdevelop, a visual studio alternative, but this doesn't seem to be an easy thing to do. Check this out:

http://www.codeproject.com/Articles/17365/ASP-NET-2-0-Visio-Custom-Control

vojo

frankly...a single sheet with 1000s of shapes is just plain useless.....its about the same as a deer in your headlights
User probably wont be able to read it.   

Have you every seen any web page from any site any where that even comes close to 1000 "things" on it???
closest I have seen are webpages that just list of text only where text is a hyperlink to something else
(usually a product interop guide or list of free games).

If you really want do something like that....I would suggest a more hierarchical approach...using a country analogy
   - Sheet 1:   States with hyperlink to 50 other sheets
   - sheets 2-51:  each state with hyperlinks to counties
   - sheets 52:1000:   each county with hyperlinks to towns/cities/neighborhoods
   - etc

Sort of why atlases and maps either limit scope (country only....state only....city only) or are a collection of submaps for more detail.

Visio wise....unless these 1000 shapes on 1 page are incredibly trivial, you are going to have a real hard time selecting or updating a given shape
(you will have to zoom in to 1600+ in order to give visio a frame of reference relative to your cursor)

Also connection point behavior will get buggy if all the shapes have connection points (let alone if you plan to do any auto routing).

Bottom line...you are in for a world of pain/agony only to produce something nobody can really use.

On the positive side....would be a nice piece of wall art  ;-)
   

Jumpy

There are the free express Versions of Visual Studio in either VB or C# but I don't know if you can use the drawing control with them. I know, that you can not produce AddOns for Visio with them, because some essential part for that is missing (Al or Paul wrote about that sometime ago in one of my threads).

gunslingor

#7
Thanks for the input Vojo,

I do truely understand where you are coming from, however, I have been thru this process with many companies and in regards to NERC CIP compliance, the more shapes you can fit on one page the better. I'm also an engineer who is used to dealing with 30x40 prints... autocad seems to handle it fine =). One of the problems with splitting it up, say onto 50 individual pages... say one for each cabinet... is that it becomes very time consuming and error prone to verify every physical network connection due to, what would be, a real large number of off-page connectors. Think of a book with 50 pages in it, and each page contains on average 20 references to other pages in the book. Constantly flipping around all the pages, you'd waste a lot of time and the story as a whole would become convoluted. Your country/states analogy is correct, its the right way to do it in some cases like making a map book... but other cases one giant diagram showing the entire country in as much detail as possible is the way to go... like in planning national defense strategies or rail lines, or... google maps comes to mind (though this is really an ideal combination or both of our strategies. this combine approach really isn't practical because a. Visio doesn't have this detail decision making algorithums and b. a state like georgia might share a border with a state like Washington if they were network components). 

There really aren't 1000 shapes on the drawings, but the potential is there. Honestly I have 4 drawings and the most dense one has 260 shapes/devices. You are right though, max zoom is usually needed... but that's expected when viewing a 44x36 inch printed page on a 12 inch screen. And it is real easy to distinguish shapes from afar when viewing the whole page, I designed the shapes for this... though the text under the shapes is usually unreadable till you zoom in.
Here's a shape example:

And from afar:

I am not showing port numbers only relative shape/device connections. I've tried showing port numbers and that's way way to cumbersome... I'm going to show port numbers via a backend database. So the user clicks on a shape hyperlink and that opens a non visio webpage showing which ports are what... also will show change/asset/configuration management data at a later date.

Now, I absolutely do want to be able to turn a lot of shapes and connections off via layers so that I can display say a subnet or VPN route of the network and nothing else... but saveasweb doesn't support layers so this is one of the reasons I am looking for alternatives.

Modified: I just found out that our company does have sharepoint with visio services... just that no one uses it or knows how to... I'm going to experiment with this solution for a little while and come back.

aledlund

#8
Maybe this will add a little to the discussion. Since VSTO wasn't used for any of it, the express versions might apply.

http://www.youtube.com/playlist?list=PLBAFAAC68D36484CA&feature=view_all

The reason for this project was I had a 1000+ sites that had to be documented (racks, servers, network, connectivity, workstations), and with the average number of  changes (2% per month) that meant almost 25% of all of the drawings would have to be updated every year. It made more sense to automate the drawings (the target was to create a new updated drawing in less than 15 seconds which was the accepted draw time of a web page) rather than attempt to archive them. Of course if you really do want to keep static drawings in Sharepoint, you can batch process the build operation.
In the operations area these drawings are linked to the real network/server devices so that I could click on them and attach to the real world object.
al

gunslingor

Quote from: aledlund on November 21, 2012, 01:38:14 PM
Maybe this will add a little to the discussion. Since VSTO wasn't used for any of it, the express versions might apply.

http://www.youtube.com/playlist?list=PLBAFAAC68D36484CA&feature=view_all

The reason for this project was I had a 1000+ sites that had to be documented (racks, servers, network, connectivity, workstations), and with the average number of  changes (2% per month) that meant almost 25% of all of the drawings would have to be updated every year. It made more sense to automate the drawings (the target was to create a new updated drawing in less than 15 seconds which was the accepted draw time of a web page) rather than attempt to archive them. Of course if you really do want to keep static drawings in Sharepoint, you can batch process the build operation.
In the operations area these drawings are linked to the real network/server devices so that I could click on them and attach to the real world object.
al

Interesting, I have messed around with automated drawings a few years ago, but it always resulted in a complete spaghetti mesh of nonsense. How do you resolve that issue? I usually prefer to update the drawings manually based on change managment tickets and a yearly device/route discovery scan.

You also mentioned you have an active drawing that lets the user view configuration or status data when the user clicks a shape. What tool are you using for this? Currently we have solar winds and the admins raster network diagrams as pdf, then manually add hyperlink buttons on top of the pdf... are you doing it automatically and if so, I'd like to know more.

We will need to archive a copy of the diagrams every time something changes for auditing purposes.

aledlund

#10
Let's reference the playlist I posted. Remembering that these are examples.

Regarding viewing configuration - check out the visRackSysMgmt video (it has the source code over on codeplex and a vba version on this forum).

"then manually add hyperlink buttons on top of the pdf", this trivial code to add to a drawing.
http://visguy.com/vgforum/index.php?topic=14.msg14720#msg14720

Regarding spaghetti drawings, The flexibility to only add cabling to the drawing when desired (check out the wan topology discussion visRackLocations) and the link drill down discussion (visRackLinkAttr). This is also touched briefly in the discussion on of dc_rack_draw and composite views.

"and a yearly device/route discovery scan" - We did it monthly (with over 120K unique/device mac addresses that had to be tracked for security and audit). Some of this of course is easily automated and can take advantage of MAC address table captured in tools such as CiscoWorks.

hth,

al

ps assetgen has a lot of this in a commercial product

cliff50

The Problem:
You manage a giant network with, say, 1000 devices including servers, workstations, PLCs, DCS, SCADA, HMIs, media converters, etc, etc... You make your self a giant network diagram that accounts for every device and every device connection; the diagrams are large and complex, pushing the boundries of what Visio can handle. You attach all the shapes to a database backend that houses configuration data for each device, You either do this by hyperlinks or Visio data imports or another method. Now you need to go online with the Visio diagrams in uneditable form so all the system admins can use, but not change the diagram. Alas, Microsoft falls short of providing adequate web functionality and tries to force you to buy a new service, Visio Services.

Hi .. we had a similar problem... our approach was a Visio GUI /access database back end... this was located on a central server and periodically download from and updated to that central distribution point.. The database could be synchronized to minimize the amount of data transferred on the WAN.... Database size @ 50Mb VSD file size @ 20Mb....  the Network is a state wide network  with @600 major nodes spread across @400 sites.   The VSD file has @200 pages...   the VSD GUI manifests as a drill down geographical Map view into the connectivity layered pages which show the physical connections between adjacent nodes... the logical connections are rendered on a single page via automated drawing routines in most cases.  We incorporated telnet links within shapes to communicate to  real world devices where possible.   In other cases we auto download node configuration data  and compare that to its database equivalent to determine if any changes have occurred and thus ensure the accuracy of the VSD network model..   the level of information stored in the VSD\ DB system ranges from voice traffic services  to LAN\ WAM, MPLS services... transmission mediums covered are fibre and broadband microwave...GIS of cable routes.... The rate of data change at this level of detail is about 3 changes per day on average...   The distribution of  the model is conditional upon clients having to have a professional version of Visio and MS Access resident on their local PC... in our case its about 40 engineering and technical staff

gunslingor

Okay, thanks guys, I appreciate your input. I guess the speghetti mess has probably gotten better since visio 2003, and it'll probably be even better in 2013. I will play around with that at a later date. Right now my plan is to finish the drawings as I have been which shouldn't take that long, get sharepoint visio site up and running, connect it to our Access asset managment DB (under develiopment), followed by connecting it to actual device managment software (different tools are used by different groups on different devices). After that I'll look at autoupdating the drawing (hopefully it won't move all my shapes around) as well as auto config comparisons. One of the problems with the autogenerate feature, if I recall, is it doesn't always capture every connection nor every device without fail... particullarly it doesn't capture, say an RS232 connection coming off a server to a terminal server... it only captures IP devices and even then, if I recall, it can't always tell what that device is so the symbol is generic I think. I'm definitely rusty with these features, but I will play next year... thanks for all the help!

aledlund

As a final note when working with the AccessDb ensure that any calls are sql compliant (I remember there being an option in setup for that).
al