can't step through _frozen_importlib/importlib._bootstrap using pdb

[exarkun]

BPO	15911
Nosy	@brettcannon, @ericsnowcurrently

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2012-09-10.19:30:36.307>
labels = ['interpreter-core', 'type-bug', 'library', '3.9']
title = "can't step through _frozen_importlib/importlib._bootstrap using pdb"
updated_at = <Date 2020-01-29.00:42:31.587>
user = 'https://bugs.python.org/exarkun'

bugs.python.org fields:

activity = <Date 2020-01-29.00:42:31.587>
actor = 'brett.cannon'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core', 'Library (Lib)']
creation = <Date 2012-09-10.19:30:36.307>
creator = 'exarkun'
dependencies = []
files = []
hgrepos = []
issue_num = 15911
keywords = []
message_count = 3.0
messages = ['170221', '170224', '170301']
nosy_count = 4.0
nosy_names = ['brett.cannon', 'exarkun', 'Arfrever', 'eric.snow']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'test needed'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue15911'
versions = ['Python 3.9']

Linked PRs

• gh-60115: Support frozen modules for linecache.getline() #131638

[exarkun]

Debugging problems involving the frozen importlib._bootstrap is difficult, because the source for importlib._bootstrap is not available to pdb. The bootstrap code can be stepped through, but with only function names and line numbers available, not source lines.

The value of having importlib written in Python would be greatly enhanced if it were more like a regular Python module that, eg, pdb could display source lines from.

[ericsnowcurrently]

This comes back to bpo-14657, which addressed the frozen vs. non-frozen copies of importlib. It sounds like one useful solution for your situation would be for _frozen_importlib to be used only long enough for bootstrap purposes. This was discussed in that other issue.

[brettcannon]

I'm going to guess this is a shortcoming of pdb when it comes to frozen modules as I can get to the source using inspect (which pdb leans on)::

>>> len(inspect.findsource(_frozen_importlib)[0])
1761
>>> len(inspect.findsource(importlib._bootstrap)[0])
1761

Which are accurate line counts::

$ wc Lib/importlib/_bootstrap.py 
 1761  6236 62517 Lib/importlib/_bootstrap.py

So why gdb can't output the source line when it has the line number of the file I don't know when it can already get access to the source without issue.

And yes, debugging imports are hard. =) Still, it's better than before as you can easily toss in a print statement or two and then just regenerate the frozen object. But I do agree it would be nice to get gdb to play along with the whole situation (and thus the title change for this bug).

[ericsnowcurrently]

This should mostly be resolved in 3.10. See issue gh-65935 (and PR gh-28656).

[gsnedders]

For those like me who stumble across this, on 3.13:

% python3.13 -IS -m pdb foo.py 
> /private/var/folders/s_/xqf0y_kx0g5bflff0vsd3xvc0000gn/T/tmp.sjn6ukQdY2/foo.py(1)<module>()
-> import urllib
(Pdb) step
--Call--
> <frozen importlib._bootstrap>(1349)_find_and_load()
(Pdb) list
1344 	
1345 	
1346 	_NEEDS_LOADING = object()
1347 	
1348 	
1349 ->	def _find_and_load(name, import_):
1350 	    """Find and load the module."""
1351 	
1352 	    # Optimization: we avoid unneeded module locking if the module
1353 	    # already exists in sys.modules and is fully initialized.
1354 	    module = sys.modules.get(name, _NEEDS_LOADING)
(Pdb) import inspect
(Pdb) pp __name__
'importlib._bootstrap'
(Pdb) pp sys.modules[__name__]
<module '_frozen_importlib' (frozen)>
(Pdb) pp sys.modules[__name__].__file__
'/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/importlib/_bootstrap.py'
(Pdb) pp sys.modules[__name__].__spec__
ModuleSpec(name='_frozen_importlib', loader=<class '_frozen_importlib.FrozenImporter'>, origin='frozen')
(Pdb) pp sys.modules[__name__].__spec__.has_location
False
(Pdb) pp sys.modules[__name__].__spec__.origin
'frozen'
(Pdb) pp inspect.getfile(sys.modules[__name__])
'/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/importlib/_bootstrap.py'
(Pdb) pp inspect.getsourcefile(sys.modules[__name__])
'/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/importlib/_bootstrap.py'

So while it isn't actually printing the current line during execution, list does manage to get the source.