python tracing a segmentation fault

I’m developing C extensions from python and I obtain some segfaults (inevitable during the development…).

I’m searching for a way to display at which line of code the segfault happens (an idea is like tracing every single line of code), how can I do that?

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

If you are on linux, run python under gdb

gdb python
(gdb) run /path/to/script.py
## wait for segfault ##
(gdb) backtrace
## stack trace of the c code

Method 2

Here’s a way to output the filename and line number of every line of Python your code runs:

import sys

def trace(frame, event, arg):
    print("%s, %s:%d" % (event, frame.f_code.co_filename, frame.f_lineno))
    return trace

def test():
    print("Line 8")
    print("Line 9")

sys.settrace(trace)
test()

Output:

call, test.py:7
line, test.py:8
Line 8
line, test.py:9
Line 9
return, test.py:9

(You’d probably want to write the trace output to a file, of course.)

Method 3

Segfaults from C extensions are very frequently a result of not incrementing a reference count when you create a new reference to an object. That makes them very hard to track down as the segfault occurs only after the last reference is removed from the object, and even then often only when some other object is being allocated.

You don’t say how much C extension code you have written so far, but if you’re just starting out consider whether you can use either ctypes or Cython. Ctypes may not be flexible enough for your needs, but you should be able to link to just about any C library with Cython and have all the reference counts maintained for you automatically.

That isn’t always sufficient: if your Python objects and any underlying C objects have different lifetimes you can still get problems, but it does simplify things considerably.

Method 4

I came here looking for a solution to the same problem, and none of the other answers helped me. What did help was faulthandler, and you can install it in Python 2.7 just using pip install.

faulthandler was introduced to Python only in version 3.3, that was released in September 2012, which was after most other answers here were written.

Method 5

There are somewhat undocumented python extensions for gdb.

From the Python source grab Tools/gdb/libpython.py (it is not included in a normal install).

Put this in sys.path

Then:

# gdb /gps/python2.7_x64/bin/python coredump
...
Core was generated by `/usr/bin/python script.py'.
Program terminated with signal 11, Segmentation fault.
#0  call_function (oparg=<optimized out>, pp_stack=0x7f9084d15dc0) at Python/ceval.c:4037
...
(gdb) python
>import libpython
>
>end
(gdb) bt
#0  call_function (oparg=<optimized out>, pp_stack=0x7f9084d15dc0) at Python/ceval.c:4037
#1  PyEval_EvalFrameEx (<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="680e550e280d061c1a11">[email protected]</a>=
    Frame 0x7f9084d20ad0, 
    for file /usr/lib/python2.7/site-packages/librabbitmq/__init__.py, line 220, 
    in drain_events (self=<Connection(channels={1: <Channel(channel_id=1, connection=<...>, is_open=True, connect_timeout=4, _default_channel=<....(truncated), <a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="e3978b918c94858f8284de978b918c94858f8284a3868d97919a">[email protected]</a>=0) at Python/ceval.c:2681
...
(gdb) py-list
 218            else:
 219                timeout = float(timeout)
>220            self._basic_recv(timeout)
 221
 222        def channel(self, channel_id=None):

As you can see we now have visibility into the Python stack corresponding with the CPython call chain.

Some caveats:

• Your version of gdb needs to be greater than 7 and it needs to have been compiled with --with-python
• gdb embeds python (by linking to libpython), it doesn’t run it in a subshell. This means that It may not necessarily match the version of python that is on $PATH.
• You need to download libpython.py from whatever version of the Python source that matches whatever gdb is linked to.
• You may have to run gdb as root – if so you may need to set up sys.path to match that of the code that you are debugging.

If you cannot copy libpython.py into sys.path then you can add it’s location to sys.path like this:

(gdb) python
>import sys
>sys.path.append('/path/to/containing/dir/')
>import libpython
>
>end

This is somewhat poorly documented in the python dev docs, the fedora wiki and the python wiki

If you have an older gdb or just can’t get this working there is also a gdbinit in the Python source that you can copy to ~/.gdbinit which add some similar functionality

Method 6

Here are 3 more alternatives:

1: Executing a script with faulthandler enabled:

python3 -X faulthandler your_script.py

2: Executing a script in debug mode (pdb)

python3 -m pdb your_script.py

and execute the script with the continue command.

The gdb tool provided the most information, but none of them print the last executed line number in my script.

3: I ended up using pytest. For this to work, I wrapped my code in a function prefixed with test_ and execute the script like this:

pytest your_script.py

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0