Introduction

This article is based on an EMail message I sent to the software development team at Vispero on August 26, 2008. I had just spent hours debugging an odd bug in JAWS®. When focus was in a particular window while JAWS was running I heard constant, rapid fire, beeping. I searched the JAWS source code for calls to the Beep and MessageBeep functions and was not able to determine the cause of the bug. I was at a loss. Then I asked the following question, is there a way I can just set breakpoints in the Beep and MessageBeep functions? This article discusses what I found during my quest for an answer to that question.

I am writing this article now because I encountered this issue again today and just spent five hours retracing my steps.

Note: After I wrote this article I learned that things are much easier in Visual Studio 2017 and Visual Studio 2019. In these versions of Visual Studio all that you need to do is open the “New Function Breakpoint” dialog box using the “Debug/New Breakpoint/Function Breakpoint…” menu item and enter the undecorated name of the function in the dialog box. If you are interested in learning how things used to be, follow me down the rabbit hole.

Falling down the rabbit hole

It turns out that it is possible to set a breakpoint in a Windows API function in Visual Studio. I do not know when this functionality was added to Visual Studio but I do know that it was available in Visual Studio 2005. The question is how.

The Set function breakpoints section of the Use breakpoints in the Visual Studio debugger page provides some relevant information; it provides very limited information on setting a breakpoint on a function with a known name. Simply use the “Debug/New Breakpoint/Function Breakpoint…” menu item to open the “New Function Breakpoint” dialog box and type a magical incantation in the “Function Name” edit.

Unfortunately, the information provided here is infuriating since it reveals that the feature is supported but it does not provide enough information to allow you to actually use the feature. In other words, it does not tell you what magical incantation to use.

Use the context operator in native C++.

{function, , [module]} [+]

Example: {MethodA, , App1.dll}+2

The article How Can I Debug Windows API Functions provides more information but it does not explain the context operator. It does reveal how to set a breakpoint in the MessageBeep function.

{,,USER32.DLL}_MessageBeep@4

This page mentions that you need to use the “decorated form of the function name” and suggests that you refer to the Viewing Decorated Names page for more information. The Viewing Decorated Names page in turn refers you to the Use a listing and Using DUMPBIN to View Decorated Names pages. Unfortunately, it turns out that neither of the techniques mentioned on the Viewing Decorated Names page will allow you to determine the decorated name for a function that is exported from a DLL. The DBH tool is supposed to be able to provide this information via the -d command-line option but in my tests using the following command it only provides undecorated names these days (bad Microsoft!).

dbh -s:srv*C:\SymbolCache*http://msdl.microsoft.com/Download/Symbols -d C:\Windows\System32\User32.dll enum *CreateWindowExW*

This command provides the following output.

Symbol Search Path: srv*C:\SymbolCache*http://msdl.microsoft.com/Download/Symbols

 index            address     name
     1            1007760 :   CreateWindowExW

Note that this is not the decorated name of the function since there is no @ character.

Unfortunately, I do not know an easy way to determine the “decorated form of the function name” other than to figure it out yourself using the information provided at Using Win32 calling conventions. Most Windows API function use the __stdcall calling convention. The general syntax for the decorated name of a __stdcall function is as follows.

_[UndecoratedFunctionName]@[SizeOfParametersInBytes]

This provides enough information for you to write the part of the magical incantation that comes after the } character, but what about the beginning of the magical incantation? Based on the example provided for the MessageBeep function one could assume that it should simply be the name of the DLL that implements the Windows API function, but is more information available?

Fortunately, the Context Operator in the Visual Studio Debugger (C++) page does provide a more detailed explanation for the first part of the magical incantation. This page reveals part of the syntax for the Context Operator.

{,,[module]} expression

Of course, this very annoyingly leaves a question unanswered. Why does this text between the { character and the } character start with two commas? What are you leaving out?

I have not been able to find official Microsoft documentation that answers that question yet. However, I do have an answer for you from Breaking on System Functions with the Context Operator. This page reveals that the full syntax of the Context Operator is as follows.

{[function],[source],[module]} expression

In most cases you will not need to worry about [function] and , you will only need to worry about the module and the expression.

Thus, the syntax for the magical incantation you will need to set the breakpoint is as follows.

{,,[module]}_[UndecoratedFunctionName]@[SizeOfParametersInBytes]

Pulling everything together.

To set a breakpoint in any Windows API function, such as the CreateWindowExW function, do the following.

1. Look up the Windows API function in the Microsoft documentation. In the Requirements section the DLL the function is implemented in is provided; in this case it is User32.dll.
2. Determine the size of the function parameters in bytes, as follows.

3. Fill in the blanks.

If you are debugging a 32-bit application the magical incantation is as follows.

{,,User32.dll}_CreateWindowExW@48

If you are debugging a 64-bit application the magical incantation is as follows.

{,,User32.dll}_CreateWindowExW@68

4. Enter the appropriate magical incantation in the “Function Name” edit of the “New Function Breakpoint” dialog box and press the OK button.

Note that in order for this to work you will need to set up Visual Studio to use the Microsoft symbol server. To do this select the “Debug/Options…” menu item to open the Debugging section of the “Options” dialog box. Then select “Debugging/Symbols” in the tree view. Next press tab to move focus to the “Symbol file (.pdb) locations” check list box and check the “Microsoft Symbol Servers” item. Then press the OK button.

References

• Breaking on System Functions with the Context Operator
• How to set breakpoints without source code in Visual Studio 2010

Down the Rabbit Hole (CC BY-SA 2.0) by Valerie Hinojosa