[Windows] New REPL doesn't allow to input non-ASCII Unicode characters

[sergey-miryanov]

Bug report

Bug description:

I got following error when I tried to type Cyrillic characters (tried ч in this example) in the new repl:

.\python.bat
Running Release|x64 interpreter...
Python 3.14.0a5+ (heads/main-dirty:d0eb01c9de9, Mar  3 2025, 23:39:38) [MSC v.1942 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> x
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    x
NameError: name 'x' is not defined
>>> Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\__main__.py", line 6, in <module>
    __pyrepl_interactive_console()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\main.py", line 59, in interactive_console
    run_multiline_interactive_console(console)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\simple_interact.py", line 143, in run_multiline_interactive_console
    statement = multiline_input(more_lines, ps1, ps2)
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\readline.py", line 389, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\reader.py", line 802, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\reader.py", line 758, in handle1
    event = self.console.get_event(block=False)
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\windows_console.py", line 472, in get_event
    self.event_queue.push(rec.Event.KeyEvent.uChar.UnicodeChar)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\base_eventqueue.py", line 77, in push
    char = bytes(bytearray((ord_char,)))
                 ~~~~~~~~~^^^^^^^^^^^^^
ValueError: byte must be in range(0, 256)

CPython versions tested on:

CPython main branch

Operating systems tested on:

Windows

Linked PRs

• gh-130804: Fix support of typing unicode chars in pyrepl #130805

[picnixz]

Did it previously worked in 3.12 or with the old repl?

[sergey-miryanov]

Did it previously worked in 3.12 or with the old repl?

Will check on 3.12, it definitely worked on 3.8 repl.

Python 3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> ч
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'ч' is not defined

[sergey-miryanov]

It works on 3.12 and 3.13

Running Release|x64 interpreter...
Python 3.12.9+ (heads/3.12-dirty:3614649714a, Mar  4 2025, 00:02:21) [MSC v.1942 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined
>>> ч
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'ч' is not defined

Python 3.13.1 (tags/v3.13.1:0671451, Dec  3 2024, 19:06:28) [MSC v.1942 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> x
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    x
NameError: name 'x' is not defined
>>> ч
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    ч
NameError: name 'ч' is not defined

[StanFromIreland]

It doesn't occur for me?

Python 3.14.0a5+ (heads/add-heapq-max-dirty:ddc27f9c385, Mar  1 2025, 14:42:30) [GCC 14.2.1 20250110 (Red Hat 14.2.1-7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> ч
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    ч
NameError: name 'ч' is not defined
>>> exit

[sergey-miryanov]

@StanFromIreland it seems a windows related.

[gvanrossum]

That means it could also be code-page or locale related.

[zooba]

We've bypassed the code page for the default REPL since 3.6, so it's almost certainly due to something in the new REPL. Possibly it's gone back to using codepages instead of Unicode?

[sergey-miryanov]

I believe it is from @y5c4l3 a65366e#diff-77cdde1442e2d3a5fe5ee852304d1af6d0a2033e718dc588552d61596cb72100R472