On Windows, I use Process Monitor, Process Explorer, and Dependency Walker.
On Linux, the primary troubleshooting tool is strace. strace is a tool that traces all the system calls that a running process makes. When a program is experiencing problems associated with I/O, strace can often help in quickly isolating the problem, since (almost) all I/O happens through system calls.
Here’s the basic syntax:
strace -o /tmp/out -f XSI
-o writes the output to a log file
-f tells strace to follow forks, and trace all child processes
-p can be used to attach to an already running process
As well as system calls, strace also tracks signals that were thrown in the process. If a process is crashing due to a segmentation fault, it is worth investigating what was happening before the fault occurred; you can find where the fault occurred by searching for — SIGSEGV in the strace output.
strace output has been added as an XSI history log client under Linux. If you search the output of strace for the string write(-1, you should see all the scripting commands as they are executed. This is useful for associating system calls with the commands that caused them to execute.
There’s also ltrace ltrace is a tool that traces all the calls to shared library functions. It can be extremely useful with XSI because almost all of the XSI code lies in shared libraries. Note that calls from a given library to itself are generally not traced because the call is address-relative. Some useful options to ltrace are:
-C: remove leading underscores from C-linkage symbols and demangle C++-linkage symbols
-o and -f: same as for strace
eX-SI 