Debugging Applications on Linux

On Linux, Intel® Pin provides application debugging capabilities utilizing GDB server protocol. This allows to use any gdbserver-based tool (GDB comes natural) to debug the emulated application. Intel® SDE extends this capability with disassembly and emulated registers support.

Step by step instructions:

There are two options for how this works, depending on your choice of either:

  1. Using GDB directly

  2. Using your tool of choice, such as VS Code, to connect to the gdbserver session

Using GDB directly:

You need two terminal sessions to debug your application. In the first one you launch Intel® SDE to run the application and in the second one you run the debugger.

Window number 1:

% sde -skx -debug -- myapp

The “-debug” knob sets several pin knobs. This will print out a port number you will need to connect-to from within gdb in the second window.

Window number 2:

% gdb myapp

In GDB paste the target remote string that was provided when launching Intel® SDE and you can start your debugging session.

Example:

Window number 1:

% sde -skx -debug -- avg512
Application stopped until continued from debugger.
Start GDB, then issue this command at the (gdb) prompt:
target remote :38049

Window number 2:

% gdb avg512
GNU gdb (GDB) 9.1
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from obj-intel64/avg_avx512.intel64.noopt.exe...
(gdb) target remote :38049
Remote debugging using :38049
0x00002aaaaaaac1f0 in _start () from /lib64/ld-linux-x86-64.so.2
(gdb) break main
Breakpoint 1 at 0x401666: file avg_avx512.c, line 138.
(gdb) cont
Continuing.

Breakpoint 1, main (argc=1, argv=0x7fffffffd3b8) at avg_avx512.c:138
138         work_mode_t mode = ALL;
(gdb) list
135
136     int main(int argc, char *argv[])
137     {
138         work_mode_t mode = ALL;
139         unsigned int n = array_size;
140
141         if (argc >= 2) {
142             if (strcmp(argv[1], "scalar") == 0)
(gdb) break calc_avg_vector
Breakpoint 2 at 0x401285: file avg_avx512.c, line 87.
(gdb) c
Continuing.

Breakpoint 2, calc_avg_vector (in=0x256, n=4061683432, avg=0x2aaaf1b5d9a0, stddev=0x7fffffffd018)
    at avg_avx512.c:87
87      {
(gdb) next
89          unsigned int nn = (n/8)*8;
...
(gdb) x/10i $pc
=> 0x4013b8 <calc_avg_vector+307>:      vmovups -0x290(%rbp),%ymm0
   0x4013c0 <calc_avg_vector+315>:      vcvtps2pd %ymm0,%zmm0
   0x4013c6 <calc_avg_vector+321>:      vmovups %zmm0,-0x170(%rbp)
   0x4013d0 <calc_avg_vector+331>:      vmovups -0x170(%rbp),%zmm0
   0x4013da <calc_avg_vector+341>:      vmovups %zmm0,-0x130(%rbp)
   0x4013e4 <calc_avg_vector+351>:      vmovups -0x270(%rbp),%zmm0
   0x4013ee <calc_avg_vector+361>:      vmovups -0x130(%rbp),%zmm1
   0x4013f8 <calc_avg_vector+371>:      vaddpd %zmm1,%zmm0,%zmm0
   0x4013fe <calc_avg_vector+377>:      vmovupd %zmm0,-0xf0(%rbp)
   0x401408 <calc_avg_vector+387>:      vmovups -0xf0(%rbp),%zmm0
(gdb) p/x $zmm1.v16_int32
$2 = {0xaacce4a0, 0x2aaa, 0x1, 0x0, 0xaacce148, 0x2aaa, 0xaaab4ccc, 0x2aaa, 0x0, 0x0, 0xf216a600,
      0x2aaa, 0x1, 0x2aaa, 0x0, 0x0}
...

As you can see in this example, all GDB commands are available. You can set breakpoints, see the values of variables and registers, examine the code and memory, and use the debugging next, step and continue as in any debugging session.

Using VS Code to connect to gdbserver

Note

The provided instructions here are for VS Code, but should translate to any other tool which is gdbserver-based.

You need one terminal sessions opened alongside your VS Code window. In this terminal, you launch Intel® SDE to run the application and then start debugging in VS Code.

Required one-time setup in VS Code:

In your launch.json file, make sure to include below fields in below example file, along with the rest of fields you had in your launch.json file:

 1{
 2    "configurations": [
 3    {
 4        "name": "gdbserver:50001",
 5        "type": "cppdbg",
 6        "request": "launch",
 7        "program": "${workspaceFolder}/avg512", # avg512 is your app binary
 8        "MIMode": "gdb",
 9
10        # Replace machine-name here with your machine name or IP address
11        # We use 50001 for our port. You can use any available port
12        "miDebuggerServerAddress": "machine-name.domain.com:50001",
13        "args": [],
14        "stopAtEntry": false, # Use this boolean to stop the debugger upon connction
15        "cwd": "${fileDirname}",
16        "environment": [],
17        "externalConsole": false,
18        "setupCommands": [
19            {
20                "description": "Enable pretty-printing for gdb",
21                "text": "-enable-pretty-printing",
22                "ignoreFailures": true
23            },
24            {
25                "description": "Set Disassembly Flavor to Intel",
26                "text": "-gdb-set disassembly-flavor intel",
27                "ignoreFailures": true
28            }
29        ]
30    }]
31}

Window number 1 (terminal session):

% sde -skx -debug -p -appdebug_server_port 50001 -- myapp
Application stopped until continued from the debugger.
Start GDB, then issue this command at the (gdb) prompt:
  target remote: 50001

The -debug knob sets several pin knobs to activate the GDB server. The -appdebug_server_port is used to set the same port here as in above launch.json.

Window number 2 (VS Code):

To start debugging, use the F5 shortcut or click on this green triangle:

_images/vscode-debug-btn.png

All VS Code IDE features are available: You can set breakpoints, see the values of variables and registers, examine memory, and use the debugger controls and continue as in any VS Code debugging session.

To execute all GDB commands (including monitor SDE commands specified below), you need to open the DEBUG CONSOLE tab (usually Ctrl+Shift+Y in VS Code) and type -exec followed by the same command as you would have typed it in GDB.

Example:

> -exec x/3i $pc
=> 0x401311 <main+1>:   mov    rbp,rsp

> -exec monitor xdis 0x401311 4
XDIS 0000000000000401311: BASE 4889E5 mov rbp,rsp

Note

In both above flows, you can use GDB print command to dump also emulated registers. This means that you can print ZMM and mask registers even on machines that do not support Intel® AVX-512 Architecture.

Intel® SDE GBD Custom Commands

Intel® SDE extends GDB commands via the monitor GDB command. You can use the monitor help command to see these extended commands.

(gdb) monitor help
general                    - General commands.
breakpoints                - Breakpoint commands.
tracepoints                - Tracepoint commands.
registers                  - Register names.
sde                        - Commands specific to Intel(R) SDE.
type "help <category>" for help on commands in that category.

(gdb) monitor help sde
xdis ADDRESS NBYTES        - Disassemble NBYTES at ADDRESS
xdis-xml ADDRESS NBYTES    - Disassemble NBYTES at ADDRESS with XML formatting
register-information       - Description of supported registers
tile-get-config            - Dump the AMX state configurations
tile-get-regs <as type>    - Dump the TMM tiles.
                            type options: flat, int16, int16x2, int16x4,
                            int32, int8, int8x4, uint32. use "dec" for decimal base
tile-get-reg N <as type>   - Dump the TMM tile N.
                            type options: flat, int16, int16x2, int16x4,
                            int32, int8, int8x4, uint32. use "dec" for decimal base
k-set-reg N VALUE          - Set the mask register N to hex VALUE
zmm-set-reg N VALUE        - Set the ZMM register N to hex VALUE.
k-get-regs                 - Dump the mask registers
k-get-reg N                - Dump the mask register N
zmm-get-reg N              - Dump the ZMM register N in various formats
zmm-get-regs               - Dump the ZMM registers in hex

Example:

(gdb) monitor xdis 0x4013b8 40
XDIS 00000000004013b8:  AVX C5FC108570FDFFFF               vmovups ymm0, ymmword ptr [rbp-0x290]
XDIS 00000000004013c0: AVX512EVEX 62F17C485AC0             vcvtps2pd zmm0, ymm0
XDIS 00000000004013c6: AVX512EVEX 62F17C48118590FEFFFF     vmovups zmmword ptr [rbp-0x170], zmm0
XDIS 00000000004013d0: AVX512EVEX 62F17C48108590FEFFFF     vmovups zmm0, zmmword ptr [rbp-0x170]
(gdb) monitor zmm-get-reg 1
zmm01 = {
    v16_float = {-3.63963e-13, 1.5305e-41, 1.4013e-45, 0, -3.6394e-13, 1.5305e-41, -3.0429e-13,
                  1.5305e-41, 0, 0, -2.9839e+30, 1.5305e-41, 1.4013e-45, 1.5305e-41, 0, 0},
    v8_double = {2.31779e-310, 4.94066e-324, 2.31779e-310, 2.31779e-310, 0, 2.31784e-310, 2.31764e-310, 0},
    v64_int8 = {-96, -28, -52, -86, -86, 42, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 72, -31, -52, -86, -86, 42,
                 0, 0, -52, 76, -85, -86, -86, 42, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -90, 22, -14, -86,
                 42, 0, 0, 1, 0, 0, 0, -86, 42, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
    v32_int16 = {-7008, -21812, 10922, 0, 1, 0, 0, 0, -7864, -21812, 10922, 0, 19660, -21845, 10922, 0, 0,
                 0, 0, 0, -23040, -3562, 10922, 0, 1, 0, 10922, 0, 0, 0, 0, 0},
    v16_int32 = {-1429412704, 10922, 1, 0, -1429413560, 10922, -1431614260, 10922, 0, 0, -233396736, 10922,
                 1, 10922, 0, 0},
    v8_int64 = {46912498361504, 1, 46912498360648, 46912496159948, 0, 46913694377472, 46909632806913, 0},
    v64_uint8 = {160, 228, 204, 170, 170, 42, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 72, 225, 204, 170, 170, 42, 0,
                 0, 204, 76, 171, 170, 170, 42, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 166, 22, 242, 170, 42, 0
                 0, 1, 0, 0, 0, 170, 42, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
    v32_uint16 = {58528, 43724, 10922, 0, 1, 0, 0, 0, 57672, 43724, 10922, 0, 19660, 43691, 10922, 0, 0, 0,
                  0, 0, 42496, 61974, 10922, 0, 1, 0, 10922, 0, 0, 0, 0, 0},
    v16_uint32 = {2865554592, 10922, 1, 0, 2865553736, 10922, 2863353036, 10922, 0, 0, 4061570560, 10922,
                  1, 10922, 0, 0},
    v8_uint64 = {46912498361504, 1, 46912498360648, 46912496159948, 0, 46913694377472, 46909632806913, 0}
}

Intel® SDE application debugging provides an advanced breakpoints options that use binary instrumentation to detect the triggering event. More information is available in the monitor help breakpoints command.