Bug report
I found a race condition happening in the rich comparison function for list where calling a rich comparison on items, which may release the GIL, may result in the list being mutated and the wrong items being compared later on in list's rich comparison.
The current code iterates through the list's ob_item to find the index of the first non-equal objects. It then calls PyObject_RichCompareBool on the two objects (which may call user-defined Python code, which may release the GIL), then checks whether the rich comparison said the items were different.
At this point, what the items exactly were is forgotten (vitem and witem go out of scope).
Later in the rich comparison function, if the desired comparison operation is not EQ/NEQ, a new call to PyObject_RichCompare is made to know which is larger/smaller. To do this, vitem and witem are re-read from the ob_item, but ob_item may actually have changed in the meantime, meaning the return value may be incorrect.
This is in a way similar to my other issue #148259 -- the root cause is mostly the same (re-reading a previously-read value that may have changed in the meantime).
I have a branch with a proposed fix, although there may be a better way as this one requires to add a lot of refcount changes which makes it all a bit hard to read / do bookkeeping for.
Note I don't currently have a reproducer, but I may try to make one if needed.
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Bug report
I found a race condition happening in the rich comparison function for
listwhere calling a rich comparison on items, which may release the GIL, may result in the list being mutated and the wrong items being compared later on inlist's rich comparison.The current code iterates through the
list'sob_itemto find the index of the first non-equal objects. It then callsPyObject_RichCompareBoolon the two objects (which may call user-defined Python code, which may release the GIL), then checks whether the rich comparison said the items were different.At this point, what the items exactly were is forgotten (
vitemandwitemgo out of scope).Later in the rich comparison function, if the desired comparison operation is not EQ/NEQ, a new call to
PyObject_RichCompareis made to know which is larger/smaller. To do this,vitemandwitemare re-read from theob_item, butob_itemmay actually have changed in the meantime, meaning the return value may be incorrect.This is in a way similar to my other issue #148259 -- the root cause is mostly the same (re-reading a previously-read value that may have changed in the meantime).
I have a branch with a proposed fix, although there may be a better way as this one requires to add a lot of refcount changes which makes it all a bit hard to read / do bookkeeping for.
Note I don't currently have a reproducer, but I may try to make one if needed.
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS