Hi Nikolay, I've pinned this problem down to the installation of Visual Studio (there may also be other software installations that cause the same problem). I now have a system here where without VS my software works fine, install VS and this error message occurs in certain places. Uninstall VS and the error messages disappear.
The message appears to occur when there is a dialog box on the screen and my software tries to open another dialog box, the most common reason for this is to show an error or information dialog box, like "registration accepted". So, this error occurs when displaying the standard message dialog box, also when showing the "open folder path dialog" but not the open file or save file dialogs. There is also an identical problem when using a tabbed dialog box and trying to open one of the other tabs (this is in my SDL for Visio application).
So, I've got around this problem within the utilities by never opening a second dialog box but always showing the current status within the main dialog.
I've tried asking about this error ("A call to an OS function failed") in the MSDN Visio Studio forum but the only person who responded to my message totally misunderstood what I was talking about and moved my post into an "off topic" area of the forum.