Skip to content

Segfault in _Unpickler_MemoPut (_pickle.c) from Unpickler.memo assignment with free-threading #150505

Open
Open
Segfault in _Unpickler_MemoPut (_pickle.c) from Unpickler.memo assignment with free-threading#150505
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-free-threadingtype-bugAn unexpected behavior, bug, or error
@Naserume

Description

@Naserume
Naserume
opened on May 27, 2026

Bug report

Bug description:

Unpickler_set_memo modifies self->memo and self->memo_size without any critical section.

When two threads set .memo on the same Unpickler instance concurrently, one can set self->memo = NULL while another is still indexing into it via _Unpickler_MemoPut, leading to a NULL-pointer dereference.

Reproducer:

import io
import pickle
from threading import Thread

shared_unpickler = pickle.Unpickler(io.BytesIO(b''))

def assign():
    for _ in range(10000):
        shared_unpickler.memo = {j: j for j in range(100)}

if __name__ == "__main__":
    threads = [Thread(target=assign) for _ in range(8)]
    for t in threads: t.start()
    for t in threads: t.join()

TSAN report:

WARNING: ThreadSanitizer: data race (pid=214122)
  Read of size 8 at 0x7dca529b9130 by thread T2:
    #0 _Unpickler_MemoPut /home/cpython/./Modules/_pickle.c:1631:22 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x2122f) 
    #1 Unpickler_set_memo /home/cpython/./Modules/_pickle.c:7804:17 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x26b77) 
    #2 getset_set /home/cpython/Objects/descrobject.c:250:16 (python3.16t+0x22d8fb) 
    #3 _PyObject_GenericSetAttrWithDict /home/cpython/Objects/object.c:2049:19 (python3.16t+0x2d844b) 
    #4 PyObject_GenericSetAttr /home/cpython/Objects/object.c:2120:12 (python3.16t+0x2d8d4b) 
    #5 Unpickler_setattr /home/cpython/./Modules/_pickle.c:7857:12 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x24eca) 
    #6 PyObject_SetAttr /home/cpython/Objects/object.c:1533:15 (python3.16t+0x2d480f) 
    #7 _PyEval_EvalFrameDefault /home/cpython/Python/generated_cases.c.h:11536:27 (python3.16t+0x45231a) 
    #8 _PyEval_EvalFrame /home/cpython/./Include/internal/pycore_ceval.h:122:16 (python3.16t+0x436463) 
    #9 _PyEval_Vector /home/cpython/Python/ceval.c:2134:12 (python3.16t+0x436463)
    #10 _PyFunction_Vectorcall /home/cpython/Objects/call.c (python3.16t+0x215013) 
    #11 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x216b11) 
    #12 _PyObject_VectorcallPrepend /home/cpython/Objects/call.c:855:20 (python3.16t+0x216b11)
    #13 method_vectorcall /home/cpython/Objects/classobject.c:55:12 (python3.16t+0x21a132) 
    #14 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x48ec15) 
    #15 context_run /home/cpython/Python/context.c:728:29 (python3.16t+0x48ec15)
    #16 method_vectorcall_FASTCALL_KEYWORDS /home/cpython/Objects/descrobject.c:421:24 (python3.16t+0x22e84c) 
    #17 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x2149a8) 
    #18 PyObject_Vectorcall /home/cpython/Objects/call.c:327:12 (python3.16t+0x2149a8)
    #19 _Py_VectorCallInstrumentation_StackRefSteal /home/cpython/Python/ceval.c:766:11 (python3.16t+0x4370c8) 
    #20 _PyEval_EvalFrameDefault /home/cpython/Python/generated_cases.c.h:1846:35 (python3.16t+0x43d022) 
    #21 _PyEval_EvalFrame /home/cpython/./Include/internal/pycore_ceval.h:122:16 (python3.16t+0x436463) 
    #22 _PyEval_Vector /home/cpython/Python/ceval.c:2134:12 (python3.16t+0x436463)
    #23 _PyFunction_Vectorcall /home/cpython/Objects/call.c (python3.16t+0x215013) 
    #24 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x216b11) 
    #25 _PyObject_VectorcallPrepend /home/cpython/Objects/call.c:855:20 (python3.16t+0x216b11)
    #26 method_vectorcall /home/cpython/Objects/classobject.c:55:12 (python3.16t+0x21a132) 
    #27 _PyVectorcall_Call /home/cpython/Objects/call.c:273:16 (python3.16t+0x214ca2) 
    #28 _PyObject_Call /home/cpython/Objects/call.c:348:16 (python3.16t+0x214ca2)
    #29 PyObject_Call /home/cpython/Objects/call.c:373:12 (python3.16t+0x214d07) 
    #30 thread_run /home/cpython/./Modules/_threadmodule.c:388:21 (python3.16t+0x61a708) 
    #31 pythread_wrapper /home/cpython/Python/thread_pthread.h:234:5 (python3.16t+0x5473bb) 

  Previous write of size 8 at 0x7dca529b9130 by thread T1:
    #0 Unpickler_set_memo /home/cpython/./Modules/_pickle.c:7816:21 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x26cc4) 
    #1 getset_set /home/cpython/Objects/descrobject.c:250:16 (python3.16t+0x22d8fb) 
    #2 _PyObject_GenericSetAttrWithDict /home/cpython/Objects/object.c:2049:19 (python3.16t+0x2d844b) 
    #3 PyObject_GenericSetAttr /home/cpython/Objects/object.c:2120:12 (python3.16t+0x2d8d4b) 
    #4 Unpickler_setattr /home/cpython/./Modules/_pickle.c:7857:12 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x24eca) 
    #5 PyObject_SetAttr /home/cpython/Objects/object.c:1533:15 (python3.16t+0x2d480f) 
    #6 _PyEval_EvalFrameDefault /home/cpython/Python/generated_cases.c.h:11536:27 (python3.16t+0x45231a) 
    #7 _PyEval_EvalFrame /home/cpython/./Include/internal/pycore_ceval.h:122:16 (python3.16t+0x436463) 
    #8 _PyEval_Vector /home/cpython/Python/ceval.c:2134:12 (python3.16t+0x436463)
    #9 _PyFunction_Vectorcall /home/cpython/Objects/call.c (python3.16t+0x215013) 
    #10 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x216b11) 
    #11 _PyObject_VectorcallPrepend /home/cpython/Objects/call.c:855:20 (python3.16t+0x216b11)
    #12 method_vectorcall /home/cpython/Objects/classobject.c:55:12 (python3.16t+0x21a132) 
    #13 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x48ec15) 
    #14 context_run /home/cpython/Python/context.c:728:29 (python3.16t+0x48ec15)
    #15 method_vectorcall_FASTCALL_KEYWORDS /home/cpython/Objects/descrobject.c:421:24 (python3.16t+0x22e84c) 
    #16 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x2149a8) 
    #17 PyObject_Vectorcall /home/cpython/Objects/call.c:327:12 (python3.16t+0x2149a8)
    #18 _Py_VectorCallInstrumentation_StackRefSteal /home/cpython/Python/ceval.c:766:11 (python3.16t+0x4370c8) 
    #19 _PyEval_EvalFrameDefault /home/cpython/Python/generated_cases.c.h:1846:35 (python3.16t+0x43d022) 
    #20 _PyEval_EvalFrame /home/cpython/./Include/internal/pycore_ceval.h:122:16 (python3.16t+0x436463) 
    #21 _PyEval_Vector /home/cpython/Python/ceval.c:2134:12 (python3.16t+0x436463)
    #22 _PyFunction_Vectorcall /home/cpython/Objects/call.c (python3.16t+0x215013) 
    #23 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x216b11) 
    #24 _PyObject_VectorcallPrepend /home/cpython/Objects/call.c:855:20 (python3.16t+0x216b11)
    #25 method_vectorcall /home/cpython/Objects/classobject.c:55:12 (python3.16t+0x21a132) 
    #26 _PyVectorcall_Call /home/cpython/Objects/call.c:273:16 (python3.16t+0x214ca2) 
    #27 _PyObject_Call /home/cpython/Objects/call.c:348:16 (python3.16t+0x214ca2)
    #28 PyObject_Call /home/cpython/Objects/call.c:373:12 (python3.16t+0x214d07) 
    #29 thread_run /home/cpython/./Modules/_threadmodule.c:388:21 (python3.16t+0x61a708) 
    #30 pythread_wrapper /home/cpython/Python/thread_pthread.h:234:5 (python3.16t+0x5473bb) 

SUMMARY: ThreadSanitizer: data race /home/cpython/./Modules/_pickle.c:1631:22 in _Unpickler_MemoPut
==================

==214455==ERROR: ThreadSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7afb5f96f25a bp 0x7afb187fd6f0 sp 0x7afb187fd690 T214460)
==214455==The signal is caused by a READ memory access.
==214455==Hint: address points to the zero page.
    #0 _Unpickler_MemoPut /home/cpython/./Modules/_pickle.c:1657:16 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x2125a) 
    #1 Unpickler_set_memo /home/cpython/./Modules/_pickle.c:7804:17 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x26b77) 
    #2 getset_set /home/cpython/Objects/descrobject.c:250:16 (python3.16t+0x22d8fb) 
    #3 _PyObject_GenericSetAttrWithDict /home/cpython/Objects/object.c:2049:19 (python3.16t+0x2d844b) 
    #4 PyObject_GenericSetAttr /home/cpython/Objects/object.c:2120:12 (python3.16t+0x2d8d4b) 
    #5 Unpickler_setattr /home/cpython/./Modules/_pickle.c:7857:12 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x24eca) 
    #6 PyObject_SetAttr /home/cpython/Objects/object.c:1533:15 (python3.16t+0x2d480f) 
    #7 _PyEval_EvalFrameDefault /home/cpython/Python/generated_cases.c.h:11536:27 (python3.16t+0x45231a) 
    #8 _PyEval_EvalFrame /home/cpython/./Include/internal/pycore_ceval.h:122:16 (python3.16t+0x436463) 
    #9 _PyEval_Vector /home/cpython/Python/ceval.c:2134:12 (python3.16t+0x436463)
    #10 _PyFunction_Vectorcall /home/cpython/Objects/call.c (python3.16t+0x215013) 
    #11 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x216b11) 
    #12 _PyObject_VectorcallPrepend /home/cpython/Objects/call.c:855:20 (python3.16t+0x216b11)
    #13 method_vectorcall /home/cpython/Objects/classobject.c:55:12 (python3.16t+0x21a132) 
    #14 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x48ec15) 
    #15 context_run /home/cpython/Python/context.c:728:29 (python3.16t+0x48ec15)
    #16 method_vectorcall_FASTCALL_KEYWORDS /home/cpython/Objects/descrobject.c:421:24 (python3.16t+0x22e84c) 
    #17 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x2149a8) 
    #18 PyObject_Vectorcall /home/cpython/Objects/call.c:327:12 (python3.16t+0x2149a8)
    #19 _Py_VectorCallInstrumentation_StackRefSteal /home/cpython/Python/ceval.c:766:11 (python3.16t+0x4370c8) 
    #20 _PyEval_EvalFrameDefault /home/cpython/Python/generated_cases.c.h:1846:35 (python3.16t+0x43d022) 
    #21 _PyEval_EvalFrame /home/cpython/./Include/internal/pycore_ceval.h:122:16 (python3.16t+0x436463) 
    #22 _PyEval_Vector /home/cpython/Python/ceval.c:2134:12 (python3.16t+0x436463)
    #23 _PyFunction_Vectorcall /home/cpython/Objects/call.c (python3.16t+0x215013) 
    #24 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x216b11) 
    #25 _PyObject_VectorcallPrepend /home/cpython/Objects/call.c:855:20 (python3.16t+0x216b11)
    #26 method_vectorcall /home/cpython/Objects/classobject.c:55:12 (python3.16t+0x21a132) 
    #27 _PyVectorcall_Call /home/cpython/Objects/call.c:273:16 (python3.16t+0x214ca2) 
    #28 _PyObject_Call /home/cpython/Objects/call.c:348:16 (python3.16t+0x214ca2)
    #29 PyObject_Call /home/cpython/Objects/call.c:373:12 (python3.16t+0x214d07) 
    #30 thread_run /home/cpython/./Modules/_threadmodule.c:388:21 (python3.16t+0x61a708) 
    #31 pythread_wrapper /home/cpython/Python/thread_pthread.h:234:5 (python3.16t+0x5473bb) 
    #32 __tsan_thread_start_func <null> (python3.16t+0xf9182) 
    #33 start_thread nptl/pthread_create.c:447:8 (libc.so.6+0x9caa3) (BuildId: 8e9fd827446c24067541ac5390e6f527fb5947bb)
    #34 clone3 misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:78 (libc.so.6+0x129c6b) (BuildId: 8e9fd827446c24067541ac5390e6f527fb5947bb)

ThreadSanitizer can not provide additional info.
SUMMARY: ThreadSanitizer: SEGV /home/cpython/./Modules/_pickle.c:1657:16 in _Unpickler_MemoPut
==214455==ABORTING

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-free-threadingtype-bugAn unexpected behavior, bug, or error

    Fields

    No fields configured for issues without a type.

    Projects

    Pickle and copy issues 🥒

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions