Network Diagramming with Visio

[cliff50]

Hi
I am curious to hear other peoples points of view on the subject of Network Diagramming using Visio.

Routed networks invariably have remote access functionality. The devices, switches, routers, hubs can be configured from afar. Moreover the addition and deletion  of the inter-nodal links occurs on a regular basis.

My questions are philosophical.

Linking:
If a routed Network can be diagrammed using a Visio Tool, should the tool be linked to the actual nodes, for the purpose of determining accuracy? It would seem the correct thing to do , however this would infer a routine of interrogation which may (depending on network size, frequency of interrogation, rate of change etc) cause congestion to traffic on the network itself. Do you think so ?



Manual draw or Automatic draw
 if a Visio based tool could automatically "draw" the entire network inter-connectivity, based on real time configuration data ("detect neighbors"), then why or should , the Visio Tool also require functionality to draw imaginary nodes and connections.?



I am sure others have passed this way before, and would be interested in their views.

Cheers
Cliff

[aledlund]

My first choice has always been to use configuration tools from the appropriate vendors (ciscoview, netview, hp-openview, etc.) This has more to do with "I don't want to write a system that has an intimate knowledge of the necessary configuration data", than "I can do a better job".
Current network configuration for data flow is already kept by the devices in many cases. The obvious exception is device attachments to switch ports. Port configuration is often the lost child in that only the more sophisticated clients have the insight to drive them to managing MAC addresses.

There was a study performed on NSFNet when Merit Corp. managed them that determined that almost 25% of the traffic on the net was 'management' (pings etc.). TCP/IP is probably the most inefficient protocol out there for managing. Beware of solutions that rely on a high number of keep-alives to determine what is available.

The time to rebalance a network after a router is lost can take up to several minutes, so the querying time should keep this in mind (if it takes five minutes to re-home the routing protocols, why query in less than that).

just some thoughts on a Sunday morning.

al Edlund

[cliff50]

Thanks Al

my thoughts re ciscoview netview etc  concern their ability to render the imaginary portions of a network .. that is .. yet to be built extensions or historically decommissioned devices.

There always seems to be a specification requirement to have network Design functionality combined with network Management functionality.

Said another way -> "I want a tool that can show me the status of the entire routed network today (the way it is), yesterday (the way it was) and tomorrow (the way it is yet to be)"

I am unsure if vender specific tools can draw history or render the future.  however I am open to enlightenment should it be the case.

In your experience,........ is chronologically dependent diagramming a worthwhile pursuit when chasing Network images ?

cheers

[daihashi]

I do this with my diagrams. While certainly there are products that are dedicated for this purpose; I needed a lightweight version of those features. My documents/diagrams will not automatically go out and fetch the information upon open. My concern was having an unnecessary flood of ICMP/SNMP packets on the network. Instead I left it up to the user to run a tool from the context menu. It allows them to check for reachability, and also check for accuracy of hostname/mgmt IP/ L3 and VLAN configurations/ uplink and downlink interfaces, etc... and update the diagram accordingly. I give them the option to do this on a per device, per page, or per document basis.

With this method it is up to the user to initiate the requests. I also make it so that the document is locked/frozen until it completes it's routine entirely. This helps to discourage people against running the tool against all devices in a given document since that would take quite a while to complete. Also, I have the document make a call to a centralized server that runs these polls instead of the individuals desktop. When they run the tool it automatically logs their username and machine name. This allows me to control  the rate of requests from my server, and also allows me to identify individuals who may be unknowingly abusing the tool.

[cliff50]

Thank you for your comment Daihashi

I am interested by your solution , when you say "...... and update the diagram accordingly."

Do you mean the human operator updates the diagrams information, yet the update is subject to his\her determination of the validity of the device's information ?
OR
Do you mean the "system" automatically updates the diagrams information, albeit without any scrutiny as to validity of the changes detected between device's information and the diagram ?

The latter would infer any device born information naturally holds the high ground and automatically over-rides any old information that the diagram may hold.

The former infers any network changes are subject to human approval.


My interest is the philosphical design of network diagramming tools .

From a distance, your solution may be a better option to vendor specific management solutions

thanks
Cliff.

[aledlund]

" chronologically dependent diagramming " Daihashi and I are of the same mind in that it is vitally important to understand who (and when) a tool is being manipulated. There was a survey by (I believe) Forrester/Cisco that showed 40% of network outages were self inflicted. Another issue to the device discovery process is that flooding a network with pings is also the symptom of a denial-of-service attack and justifiably should get network security involved. I had a client that turned off ICMP Echo in the entire WAN to protect against these types of floods. We ended up writing a telnet application and used the connect sequence as a 'ping' prior to allowing an expert system to log onto the cisco devices to determine their configuration. All of the data was logged to SQL for analysis (approximately 110,000 devices discovered on an average query which could take up to three days to execute).
al

[aledlund]

a final note. Our target was diagramming in support of problem determination and all documents were a point in time since they were created as needed and disposed of ASAP. The diagrams tend to have a life of their own and unless their context is understood they may lead to wrong assumptions. In our network on average there was a 2% change in configuration per month (excluding special projects), which means that at the end of the year 25% of any created documentation was probably wrong. We relied entirely upon captured data from the devices and in some cases queried from vendor captured data.
al

[daihashi]

What I meant about auto updating the document is that it will query specific OIDs on the devices, and validate lldp/cdp information. It will also retrieve VLAN configuration information from L2 devices, specific L3 information (OSPF, p2p subnets, interface IPs), and interfaces that are used for uplinks/downlinks, p2p links, aggregate IDs etc etc. It will also validate hostname against IP address, and IP address to hostname. There are many other things that it does, but I won't get into all that since I could probably write a 10-page document on my template document. Point is that the document is capable of maintaining it's own accuracy with minimal effort from the user.

It retrieves about 95% of that information via SNMP polling, and the tool polls very specific OIDs instead of trying to do large bulk gets or walks. It parses out all the junk and only grabs the portions of the value fields that are relevant for the document; in other words, it grabs the portion of the OID value that a human would understand.

However, the user of the document must initiate the report. When the report completes it will report the difference between what the diagram/document shows and what is actually configured on the device. The report has two buttons that the user can click; one is labeled "Apply Updates".... which will update the document with the findings from the report, and the other is "Cancel" so that the user can back out if necessary.

Aledlund is correct in that we both share the exact same concerns. I understand, and support, the need for an engineer to be able to do some of these traffic generating tasks. However, at the same time I must mitigate the network traffic that it generates... and more importantly I must mitigate the risk as much as possible.

Most users don't understand that there is an impact to the network with things such as this; especially when you build the tool so well that it appears to be like "magic" to them. All that they know is that the tool helps them accomplish the goal of their task. So to help discourage against abuse of the tool I've added a few minor manual steps in the process, and have also added an authentication and logging portion to it as well. The server on the back end that runs all this helps me to further control things from a centralized point.

When you work in a global company, with over 300,000 employees and countless more contract workers, you know that any given document will be opened multiple times a day. If each document did all of this automatically upon open, then this would be a flood of unnecessary traffic on the network. This increase in bandwidth may escalate to the point of needing to increase the size of an MPLS circuit, or might even require a change in the access type provided (i.e. OC3,OC12, GigE, etc).

There is a lot of forward thinking that must happen before you build and publish a network tool that will have wide spread use. Otherwise the impact it will have on the network will be far too great to be of any actual benefit.

I hope what I've stated here makes sense.

[cliff50]

Thanks guys, I really appreciate your insight.
 
What seems to be filtering through here is->  the need to know what the Network status is "NOW" .
The most current snapshot is paramount to successful network management.

Al-> "We relied entirely upon captured data from the devices and in some cases queried from vendor captured data."
I have to ask , on days when captured data was not forthcoming (for any reason)..was this a show-stopper ?
Or was there a plan B, a phone call to a colleague, or a site visit to eyeball the status quo, in order to obtain the info.
Al, I am assuming by your description, the diagrams were generated on the fly from captured data, would it be fair to say at the end of the day the diagram had to be created...and so .... On a good day the diagram looked wholesome, on a bad day the diagram looked holed.

Or could you manually tickle the diagram to fill in the gaps, where the capture routine had been remiss.

Manual adjustments to Network diagrams that have no link to captured data, suggests the ability to draw the imaginary portions of the Network and this is where I stumble over Network Design functionality while trying to appease Network Management functionality.

Example: the Design dept embarks on a project to extend the Network, to be deployed in stages.  There is a requirement to show all stages in advance, on the network diagrams. The commissioning of each device is staged and in some cases rolled back or delayed. Yet all the while clients are expecting to be made aware of "yet to be on line" devices, courtesy of yon Network diagrams.
Unfortunately there aint no captured data bubbling up from a device that is unplugged

To some eyes, the current Network snapshot includes the devices that are dormant, decommissioned or deployed as design only.
I believe that captured data can be archived and used by the diagramming tool to draw "yesterdays" Network.
However, Would your diagramming tool have a manual override to allow future machinations to be drawn as well ?

Hyperthetical.
As part of a project a new router is stitched into your network. It is turned on , it is situated at the end of a spur link, no end user circuits are attached to it . It is in essence non-traffic carrying.  its commissioning for traffic is still yet to be realised.
In your schema would you interrogate its data capture and have it appear on your network diagram ?
Moreover, if this routing device developed an alarm status, why would you invest time to rectify it ?

cheers

Cliff

[aledlund]

"I have to ask , on days when captured data was not forthcoming (for any reason)..was this a show-stopper" Show stopper no. Alternate scenarios included a.) cisco had a free tool to gather config data, viewable not printable. b.) remote probe for sniffer data that we could remote desktop too.
 
"On a good day the diagram looked wholesome". I had an audit trail of the issued command prompts with the responses. If commands aren't completed successfully, the data doesn't get into the database and corrupt the last image.

"To some eyes the current Network snapshot includes the devices that are dormant, decommissioned or deployed as design only". We used a color by value philosophy green=good, red=not available, yellow=issue or not yet implemented. We needed the not yet implemented to allow us to plan rack spaces. The tickler for a device in trouble was an IP address of '911' which told the query engine not to try the device. When the device was supposed to be live we would update the address.

al

[cliff50]

hi Al,

That sounds to me a very comprehensive solution to what must have been a massive undertaking.

I get the impression this solution would comprise of static images, an underlying SQL data repository, an interface to invoke
the device interrogation and log the event, a coded componant to report results of comparing captured data against encumbant data, and of course -> a coded componant to generate "on the fly diagrams"

hmmmmm....by the numbers:
110,000 devices with a 2% change per month
computes as 14 data conflicts to resolve per hour (based on 8 hrs / day.. 20 working days in a month)
assuming of course, a solitary anomaly detected per device.

posibly even more if 3 of the days were used by the capture routine.

interesting the choice of default address for a troubled IP target

thankyou for your description
Cliff

[aledlund]

cliff,
the 2% number was a site level specific (1000+ sites) with about twenty changes per month (swapping out defective swiths/routers, servers, etc.).
al

[cliff50]

twenty data conflicts to resolve per month, seems to me a reasonable rate.

In a philosophical sense therein lays a crucial point associated with designing Network Diagramming tools.

1.The network's size is directly proportional to data capture time.
2.The diagram's detail requirements are directly proportional to the detail in data capture.
3.The time elapsed to produce a current network diagram is directy proportional to data conflict resolution time.

If twenty a month is do-able, yet a million per month is not,
 then who among us would be brave enough to nominate the maximum rate they could process ?

So this leads me to wonder, why would you capture the changes in the real network by the interrogation method, then resolve any data conflicts prior to and in order to, produce a current Network diagram.

as opposed to:

Having a Network Diagramming tool which is not linked to any data capture, thereby removing the elapsed time associated with fetching the data and removing the elapsed time associated with processing data conflicts.

Network diagrams produced by the first method are only current if the rate of data processing time is equal to, or exceeds the rate of changes in the actual Network.

Network diagrams produced by the second method infers "The document always holds the high ground and the actual Network device's configuration is meaningless if it does not coincide with the document."

what would be your prescription "changes in device configs lead changes in diagrams ... or vica versa " ?

Cliff

[Networker]

Hey Guys, I have found new 3D stencils for 3D style network diagrams with shadow, glowing light effects for MS Visio.

https://3d-networking.com/3d-symbols/

See the sample. They are awesome.

https://3d-networking.com/showcase/
Modify message

[vojo]

nice looking...not actually vision shapes....just images...but nice looking