I realize this is an old post but I thought I'd tie the ends up a little in case the thread gets read again:
The dynamic connector is code driven and goes through a calculation cycle of its own. It has to coexist with the shape sheet which is the final arbiter of all the data results but it gets a crack at everything the shape sheet engine does. There's no getting around it really since it is baked into the product.

You can, of course, make the connector not be 'dynamic' by changing it's placement and routing cell. I suspect that's not what you'd like however.
The minimum offset (I'm working from memory here so please forgive me) is based on (a fraction of) shape to shape placement values, I believe - the offset itself was never broken out. Those used to be set in the page setup dialog but I don't know where it is now.
Take note: Changing the default shape to shape distance can have some unpleasant effects on the more 'normal' routing situations, possibly so experiment a bit before committing.
* not sure what the terminology in the shape sheet, UI or docs is at this time (e.g. 'shape to shape' or 'placement and routing cell')
** the point of the shape to shape clearance et, al. is to give Visio's placement and routing facilities (of which the dynamic connector is a consumer) a hint as to the average size and distance between the shapes meant to be interconnected. Naturally those 'hints' are violated all the time and are sometimes not even correct for the template it is based on. This might even describe your situation.