Crash with PYTHON_LLTRACE=4 due to presence of PyDictKeysObject on stack

[markshannon]

Bug report

Bug description:

The dump_stack() function, used when tracing micro-op execution, crashes if a PyDictKeysObject* pointer is on the stack.
Despite the name, PyDictKeysObject is not a PyObject.

Introduced in f978fb4

I think the best fix would be arrange the fields (at least in the debug build) of PyDictKeysObject such that the dk_kind field is placed in the least significant byte of the ob_type field of PyObject and change DictKeysKind so that none of its values have the low 2 bits set to 0.
Then dump_stack can check the low bits of the ob_type to see whether the "object" is a PyObject or a PyDictKeysObject.

Looking forward, we expect to have an unused value for the low bits in PyStackRef, so we could assign them the meaning "not a Python object".
This would add overhead when pushing and popping PyDictKeysObjects, but would make introspection a lot more robust.

@mpage

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

[mpage]

Alternatively, since this only affects tracing of micro-ops, we could special case the few affected micro-ops in dump_stack rather than modifying PyDictKeysObject.

Also, I think the same issue should affect tracing of _PUSH_FRAME, which consumes a _PyInterpreterFrame, which is also not a PyObject, from the stack:

cpython/Python/bytecodes.c

Lines 3784 to 3788 in 99ed302

	op(_PUSH_FRAME, (new_frame: _PyInterpreterFrame* -- )) {
	// Write it out explicitly because it's subtly different.
	// Eventually this should be the only occurrence of this code.
	assert(tstate->interp->eval_frame == NULL);

Are you also seeing a crash when tracing those opcodes?

[markshannon]

Alternatively, since this only affects tracing of micro-ops, we could special case the few affected micro-ops ...

Which is fine when everything is working correctly. But if everything is working then you don't need low-level tracing.
Plus we change micro-ops all the time, so this is bound to get out of sync.

So this needs to work regardless of which micro-op we are about to execute or what is on the stack.

I think the same issue should affect tracing of _PUSH_FRAME, which consumes a _PyInterpreterFrame ...

Presumably the layout of _PyInterpreterFrame is such that nothing crashes. I think it ends up printing the bytes of the previous frame's globals as a C string. Mostly non-crashing.

[markshannon]

We do have at least one spare tag in _PyStackRef so we could tag non-objects. dump_stack could then print "<non-object>" or similar. Not super informative, but a lot better than crashing, and a lot less finicky that rearranging the layout of PyDictKeyObjects and _PyInterpreterFrame.

I think we would want to tag the pointers only in the debug build, for performance.