Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PyList_GetItemRef and use it in list.get_item #4410

Merged
merged 6 commits into from
Aug 10, 2024

Conversation

ngoldbaum
Copy link
Contributor

Refs #4265 #4355

See the C API docs.

Both the new shim and the bindings to the CPython C API on 3.13 should be covered by existing tests in PyO3.

Copy link
Member

@davidhewitt davidhewitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Just a few small observations to round this one off...

pyo3-ffi/src/compat.rs Outdated Show resolved Hide resolved
pyo3-ffi/src/compat.rs Show resolved Hide resolved
newsfragments/4410.added.md Outdated Show resolved Hide resolved
src/types/list.rs Outdated Show resolved Hide resolved
ngoldbaum and others added 2 commits August 7, 2024 14:07
Co-authored-by: David Hewitt <mail@davidhewitt.dev>
Copy link
Member

@davidhewitt davidhewitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks very much! 👍

@davidhewitt davidhewitt added this pull request to the merge queue Aug 7, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 7, 2024
@davidhewitt davidhewitt added this pull request to the merge queue Aug 8, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 8, 2024
@davidhewitt
Copy link
Member

davidhewitt commented Aug 8, 2024

Hmm the debug builds failing on 3.12. If I had to guess, it's an issue in the definition of Py_XINCREF just due to the way we set up the debug interpreter. I'll try to investigate tomorrow.

@ngoldbaum
Copy link
Contributor Author

ngoldbaum commented Aug 8, 2024

The debug build is hitting an assert inside CPython, running test_gevent.py.

Separately, it looks like there have also been some upstream changes for subinterpreters in 3.13. that aren't captured in this file (the module is still private and named _interpreters now). I'll issue a separate PR for that.

@ngoldbaum
Copy link
Contributor Author

(this is on main btw)

frame #7: 0x00000001011a6b68 libpython3.12d.dylib`gc_decref(g=0x00000001045397b0) at gcmodule.c:113:5
   110 	static inline void
   111 	gc_decref(PyGC_Head *g)
   112 	{
-> 113 	    _PyObject_ASSERT_WITH_MSG(FROM_GC(g),
   114 	                              gc_get_refs(g) > 0,
   115 	                              "refcount is too small");
   116 	    g->_gc_prev -= 1 << _PyGC_PREV_SHIFT;
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
    frame #0: 0x000000018425aa60 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x0000000184292c20 libsystem_pthread.dylib`pthread_kill + 288
    frame #2: 0x000000018419fa30 libsystem_c.dylib`abort + 180
    frame #3: 0x000000010114713c libpython3.12d.dylib`fatal_error_exit(status=-1) at pylifecycle.c:2735:9
    frame #4: 0x0000000101147000 libpython3.12d.dylib`fatal_error(fd=2, header=1, prefix="_PyObject_AssertFailed", msg="_PyObject_AssertFailed", status=-1) at pylifecycle.c:2916:5
    frame #5: 0x00000001011468dc libpython3.12d.dylib`_Py_FatalErrorFunc(func="_PyObject_AssertFailed", msg="_PyObject_AssertFailed") at pylifecycle.c:2932:5
    frame #6: 0x0000000100f6e714 libpython3.12d.dylib`_PyObject_AssertFailed(obj=[], expr="gc_get_refs(g) > 0", msg="refcount is too small", file="Modules/gcmodule.c", line=115, function="gc_decref") at object.c:2603:5
  * frame #7: 0x00000001011a6b68 libpython3.12d.dylib`gc_decref(g=0x00000001045397b0) at gcmodule.c:113:5
    frame #8: 0x00000001011a6ad0 libpython3.12d.dylib`visit_decref(op=[], parent=0x000000010474d850) at gcmodule.c:472:13
    frame #9: 0x0000000100f4e850 libpython3.12d.dylib`dict_traverse(op=0x000000010474d850, visit=(libpython3.12d.dylib`visit_decref at gcmodule.c:462), arg=0x000000010474d850) at dictobject.c:3545:17
    frame #10: 0x00000001011a6768 libpython3.12d.dylib`subtract_refs(containers=0x00000001014fa4a8) at gcmodule.c:491:16
    frame #11: 0x00000001011a5860 libpython3.12d.dylib`deduce_unreachable(base=0x00000001014fa4a8, unreachable=0x000000016fdcdda8) at gcmodule.c:1116:5
    frame #12: 0x00000001011a1760 libpython3.12d.dylib`gc_collect_main(tstate=0x0000000101557d80, generation=0, n_collected=0x000000016fdcde20, n_uncollectable=0x000000016fdcde18, nofail=0) at gcmodule.c:1242:5
    frame #13: 0x00000001011a13c4 libpython3.12d.dylib`gc_collect_with_callback(tstate=0x0000000101557d80, generation=0) at gcmodule.c:1426:14
    frame #14: 0x00000001011a27f8 libpython3.12d.dylib`gc_collect_generations(tstate=0x0000000101557d80) at gcmodule.c:1481:17
    frame #15: 0x00000001011a270c libpython3.12d.dylib`_Py_RunGC(tstate=0x0000000101557d80) at gcmodule.c:2295:5
    frame #16: 0x000000010110a644 libpython3.12d.dylib`_Py_HandlePending(tstate=0x0000000101557d80) at ceval_gil.c:1045:9
    frame #17: 0x00000001010908cc libpython3.12d.dylib`_PyEval_EvalFrameDefault(tstate=0x0000000101557d80, frame=0x000000010010f0e0, throwflag=0) at ceval.c:834:9

@ngoldbaum
Copy link
Contributor Author

Odd, if I go back to a version of PyO3 from last month the issue is still there on the debug python build, so it's not a new problem. I have no idea why it's suddenly showing up in CI.

@davidhewitt
Copy link
Member

davidhewitt commented Aug 9, 2024

The bug with test_gevent is zopefoundation/zope.interface#316 (comment) - I can fix locally if I install zope.interface<7. Will push a PR to fix.

@davidhewitt davidhewitt added this pull request to the merge queue Aug 9, 2024
Merged via the queue into PyO3:main with commit be8ab5f Aug 10, 2024
42 checks passed
davidhewitt added a commit that referenced this pull request Sep 3, 2024
* Add PyList_GetItemRef bindings and compat shim

* Use PyList_GetItemRef in list.get_item()

* add release notes

* apply code review comments

* Update newsfragments/4410.added.md

Co-authored-by: David Hewitt <mail@davidhewitt.dev>

* apply `all()` doc cfg hints

---------

Co-authored-by: David Hewitt <mail@davidhewitt.dev>
davidhewitt added a commit that referenced this pull request Sep 3, 2024
* Add PyList_GetItemRef bindings and compat shim

* Use PyList_GetItemRef in list.get_item()

* add release notes

* apply code review comments

* Update newsfragments/4410.added.md

Co-authored-by: David Hewitt <mail@davidhewitt.dev>

* apply `all()` doc cfg hints

---------

Co-authored-by: David Hewitt <mail@davidhewitt.dev>
davidhewitt added a commit that referenced this pull request Sep 15, 2024
* Add PyList_GetItemRef bindings and compat shim

* Use PyList_GetItemRef in list.get_item()

* add release notes

* apply code review comments

* Update newsfragments/4410.added.md

Co-authored-by: David Hewitt <mail@davidhewitt.dev>

* apply `all()` doc cfg hints

---------

Co-authored-by: David Hewitt <mail@davidhewitt.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants