Intermittent test failure on iOS during test_interpreters stress test: RuntimeError: Failed to import encodings module

[freakboy3742]

Bug report

Bug description:

We've seen intermittent failures on the iOS buildbot during the execution of the test_interpreters test:

...
test_create_many_sequential (test.test_interpreters.test_stress.StressTests.test_create_many_sequential) ... ok
test_create_many_threaded (test.test_interpreters.test_stress.StressTests.test_create_many_threaded) ... ok
test_many_threads_running_interp_in_other_interp (test.test_interpreters.test_stress.StressTests.test_many_threads_running_interp_in_other_interp) ... Warning -- Uncaught thread exception: InterpreterError
Exception in thread Thread-1571 (run):
RuntimeError: Failed to import encodings module

During handling of the above exception, another exception occurred:

interpreters.InterpreterError: sub-interpreter creation failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/buildbot/Library/Developer/XCTestDevices/4F6EA510-4EA0-4706-8B1F-020FE958C08F/data/Containers/Bundle/Application/DBFDCCFF-85B7-4B66-9B7D-D1A7666166D0/iOSTestbed.app/python/lib/python3.14/threading.py", line 1054, in _bootstrap_inner
    self.run()    
    ~~~~~~~~^^  
  File "/Users/buildbot/Library/Developer/XCTestDevices/4F6EA510-4EA0-4706-8B1F-020FE958C08F/data/Containers/Bundle/Application/DBFDCCFF-85B7-4B66-9B7D-D1A7666166D0/iOSTestbed.app/python/lib/python3.14/threading.py", line 996, in run
    self._target(*self._args, **self._kwargs)    
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
  File "/Users/buildbot/Library/Developer/XCTestDevices/4F6EA510-4EA0-4706-8B1F-020FE958C08F/data/Containers/Bundle/Application/DBFDCCFF-85B7-4B66-9B7D-D1A7666166D0/iOSTestbed.app/python/lib/python3.14/test/test_interpreters/test_stress.py", line 47, in run
    interp = interpreters.create()  
  File "/Users/buildbot/Library/Developer/XCTestDevices/4F6EA510-4EA0-4706-8B1F-020FE958C08F/data/Containers/Bundle/Application/DBFDCCFF-85B7-4B66-9B7D-D1A7666166D0/iOSTestbed.app/python/lib/python3.14/test/support/interpreters/__init__.py", line 76, in create    
    id = _interpreters.create(reqrefs=True)
interpreters.InterpreterError: interpreter creation failed
ok

----------------------------------------------------------------------
Ran 152 tests in 7.483s
OK (skipped=14)
0:05:31 load avg: 4.70 [248/486/1] test_interpreters failed (env changed)

The test itself passes seems to pass; the failure that is reported is a change in environment. The test passes on a re-run.

There have been 6 examples of this in the last 2 months:

• https://buildbot.python.org/#/builders/1380/builds/2955
• https://buildbot.python.org/#/builders/1380/builds/2860
• https://buildbot.python.org/#/builders/1380/builds/2813
• https://buildbot.python.org/#/builders/1380/builds/2528
• https://buildbot.python.org/#/builders/1380/builds/2481
• https://buildbot.python.org/#/builders/1380/builds/2425

It has also been observed in 3.13

• https://buildbot.python.org/#/builders/1386/builds/988

It's not clear to me if this is an iOS issue (falling apart under the stress), or an issue with the underlying subinterpreter setup that iOS is able to (intermittently) expose. The "Failed to import encodings module" is the bit that gives me pause - There's no diagnostics on why the import failed, but that sort of error would be consistent with a path not being fully configured. It clearly is configured (in the normal case at least), but I'm wondering if having multiple threads start simultaneously could be exposing a leak where one thread thinks something has been configured but another thread is still initialising.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Other

[vstinner]

I'm able to reproduce the issue on Linux with the command ./python -m test test_interpreters.test_stress -u all -v -F -j2 --fail-env-changed:

0:00:49 load avg: 7.92 [ 16/1] test_interpreters.test_stress worker non-zero exit code (Exit code -6 (SIGABRT))
test_create_many_sequential (test.test_interpreters.test_stress.StressTests.test_create_many_sequential) ... ok
test_create_many_threaded (test.test_interpreters.test_stress.StressTests.test_create_many_threaded) ... Warning -- Uncaught thread exception: InterpreterError
Exception in thread Thread-134 (task):
RuntimeError: Failed to import encodings module

During handling of the above exception, another exception occurred:

interpreters.InterpreterError: sub-interpreter creation failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/threading.py", line 1054, in _bootstrap_inner
    self.run()
    ~~~~~~~~^^
  File "/home/vstinner/python/main/Lib/threading.py", line 996, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/test/test_interpreters/test_stress.py", line 30, in task
    interp = interpreters.create()
  File "/home/vstinner/python/main/Lib/test/support/interpreters/__init__.py", line 76, in create
    id = _interpreters.create(reqrefs=True)
interpreters.InterpreterError: interpreter creation failed
ok
test_many_threads_running_interp_in_other_interp (test.test_interpreters.test_stress.StressTests.test_many_threads_running_interp_in_other_interp) ... ok

----------------------------------------------------------------------
Ran 3 tests in 6.056s

OK
python: Objects/typeobject.c:301: managed_static_type_state_clear: Assertion `!_PyRuntime.types.managed_static.types[full_index].interp_count' failed.

[freakboy3742]

Thanks @vstinner - that seems to confirm it's not an iOS error, just something iOS is good at exposing for some reason.

[ZeroIntensity]

Yeah, this is a known issue. See also #127108 and #129824. Static types are a bit racy right now; I've been working on a PR to fix it over the past week or two, but due to some burnout and lack of time, it's not going to be ready for at least a few more days.

[freakboy3742]

In which case, I'll close this in favor of #127108, and copy over the useful details. Thanks for letting me know about the other tickets.