Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to build cpython in macOS because it was expecting "python.exe" #131563

Closed
wyatt-wong opened this issue Mar 21, 2025 · 14 comments
Closed

Failed to build cpython in macOS because it was expecting "python.exe" #131563

wyatt-wong opened this issue Mar 21, 2025 · 14 comments

Comments

@wyatt-wong
Copy link

wyatt-wong commented Mar 21, 2025

I clone the GitHub repository of cpython with git clone https://github.com/python/cpython/ in macOS Sequoia 15.3.2, then I executed ./configure --enable-optimizations --with-lto to create a Makefile. After that, I run make but I got the following error which showed that python.exe was expected in macOS environment ?

I have included config.log and Makefile for reference.

config.zip

Makefile.zip

Bug report

Bug description:

Rebuilding with profile guided optimizations:
rm -f profile-clean-stamp
make build_all CFLAGS_NODIST=" -fprofile-instr-use="/Users/wyattwong/GitHub/cpython/code.profclangd"" LDFLAGS_NODIST=""
make[1]: Entering directory '/Users/wyattwong/GitHub/cpython'
Checked 112 modules (34 built-in, 78 shared, 0 n/a on macosx-15.3-x86_64, 0 disabled, 0 missing, 0 failed on import)
./python.exe -E ./Tools/build/generate-build-details.py `cat pybuilddir.txt`/build-details.json
make[1]: Leaving directory '/Users/wyattwong/GitHub/cpython'

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

@wyatt-wong wyatt-wong added the type-bug An unexpected behavior, bug, or error label Mar 21, 2025
@StanFromIreland
Copy link
Contributor

I'm not on macOS and I don't have a clue as to how it works but have you tried following the steps in the devguide?

Maybe there is something you missed.

@ambv
Copy link
Contributor

ambv commented Mar 21, 2025

You need to build on a file system that's case-insensitive, as is the default on AppleFS and HFS+.

@wyatt-wong
Copy link
Author

I'm not on macOS and I don't have a clue as to how it works but have you tried following the steps in the devguide?

Maybe there is something you missed.

Maybe let someone else who have ever build it under macOS to provide me with some insights.

@wyatt-wong
Copy link
Author

You need to build on a file system that's case-insensitive, as is the default on AppleFS and HFS+.

I am using macOS Sequoia and already using case-insensitive APFS. But how would running make in macOS will come up with Python.exe which will only exists in Windows environment ?

I guessed your reply was in relevant to my question.

@wyatt-wong
Copy link
Author

wyatt-wong commented Mar 22, 2025

I wish ONLY those who have ever build cpython from GitHub source in macOS environment to review my question and check what's wrong ?

I followed the Build Instructions in the README.md file with additional parameters in running ./configure --enable-optimizations --with-lto but failed after I run make command.

./configure
make
make test
sudo make install

@zware
Copy link
Member

zware commented Mar 22, 2025

I'm afraid you're going to have to be clearer about what error you saw, because your initial post does not include one. python.exe is the expected executable name on macOS to avoid collision with the Python/ directory on the case-insensitive file systems used by macOS.

@ned-deily
Copy link
Member

ned-deily commented Mar 22, 2025

If you are building on a case-insensitive macOS file system (the macOS default and as it appears you are from the config log), the Python binary in the build directory will have a file name of python.exe to avoid collision with the Python directory name as noted above. However when you do a make install, the installed Python binary will have a name like python, not python.exe. Since it appears you didn't override the default install location (for example, by using the PREFIX variable), the installed python binary should be in /usr/local/bin/python3.13. That said, it is still not clear from your descriptions exactly what error you are seeing.

@wyatt-wong
Copy link
Author

I'm afraid you're going to have to be clearer about what error you saw, because your initial post does not include one. python.exe is the expected executable name on macOS to avoid collision with the Python/ directory on the case-insensitive file systems used by macOS.

The error you saw on my initial post is the SAME error that I saw on screen.

I ran with the following commands:

git clone https://github.com/python/cpython
./configure --enable-optimizations --with-lto
make

@ned-deily
Copy link
Member

The error you saw on my initial post is the SAME error that I saw on screen.

I’m sorry but I really don’t see an error in your initial post. Please show exactly what you mean.

@wyatt-wong
Copy link
Author

If you are building on a case-insensitive macOS file system (the macOS default and as it appears you are from the config log), the Python binary in the build directory will have a file name of python.exe to avoid collision with the Python directory name as noted above. However when you do a make install, the installed Python binary will have a name like python, not python.exe. Since it appears you didn't override the default install location (for example, by using the PREFIX variable), the installed python binary should be in /usr/local/bin/python3.13. That said, it is still not clear from your descriptions exactly what error you are seeing.

I checked the generated Makefile and then I realized what you are taking about.

The Makefile contained these lines:

# Executable suffix (.exe on Windows and Mac OS X)
EXE=		
BUILDEXE=	.exe
PYTHON=		python$(EXE)
BUILDPYTHON=	python$(BUILDEXE)

So BUILDPYTHON contains the value of python.exe but it is macOS binary and NOT a Windows executable despite the name is called python.exe

Now I knew what's the problem. When I run make command to build the cpython GitHub source in macOS, the macOS binary python.exe was NOT yet created. However the make process have reached to the point that it requires using python.exe to proceed and so it displayed the error of file not found when it tries to run ./python.exe xxxx

In short, that is a make dependency issue and someone should review the configure Shell script which generate the Makefile. The python.exe macOS binary should be created FIRST before make execute ./python.exe xxxx

Today I found the cpython GitHub repository tree was updated. Furthermore I saw the Mac/README.rst file which mentioned additional parameters specific for macOS. So I have removed my cpython GitHub folder and re-download a CLEAN cpython GitHub repository, then re-run ./configure with additional parameters and run make to see the compilation process.

I will provide any update after finish running make again on the Makefile which contains additional parameters from Mac/README.rst

@ned-deily
Copy link
Member

I'm still confused as to what error you might be seeing since I see no file not found message in your post.

But here's a sample build of the current head of cpython (which is currently in the pre-release 3.14 alpha phase) on a vanilla macOS 15.3 system using the current Apple Command Line Tools. If you have a current Xcode installed, it should give the same results.

% sw_vers
ProductName:		macOS
ProductVersion:		15.3
BuildVersion:		24D60
% cc --version
Apple clang version 16.0.0 (clang-1600.0.26.6)
Target: arm64-apple-darwin24.3.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
% xcode-select --install
xcode-select: note: Command line tools are already installed. Use "Software Update" in System Settings or the softwareupdate command line interface to install updates
% git clone https://github.com/python/cpython
Cloning into 'cpython'...
[...]
% cd cpython
% ./configure --enable-optimizations --with-lto
checking for git... found
checking build system type... aarch64-apple-darwin24.3.0
checking host system type... aarch64-apple-darwin24.3.0
[...]
configure: creating Modules/Setup.local
configure: creating Makefile
configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.
% make
Running code to generate profile data (this can take a while):
# First, we need to create a clean build with profile generation
# enabled.
/Library/Developer/CommandLineTools/usr/bin/make profile-gen-stamp
[...]
The necessary bits to build these optional modules were not found:
_gdbm                     _hashlib                  _lzma
_ssl                      _tkinter
To find the necessary bits, look in configure.ac and config.log.

Could not build the ssl module!
Python requires a OpenSSL 1.1.1 or newer

Checked 112 modules (34 built-in, 73 shared, 0 n/a on macosx-15.3-arm64, 0 disabled, 5 missing, 0 failed on import)
./python.exe -E ./Tools/build/generate-build-details.py `cat pybuilddir.txt`/build-details.json

% ./python.exe
Python 3.14.0a6+ (heads/main:8b7d20d3a9d, Mar 22 2025, 02:06:10) [Clang 16.0.0 (clang-1600.0.26.6)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

Note that the inability to build those optional modules is to be expected here
since I didn't provide any of the needed third-party libraries.
The Devguide gives some suggestions on how to easily do that,
by using third-party distributors like Homebrew or MacPorts.
On the other hand, those distributors also provide full distributions
of various versions of Python and many other third-party packages,
potentially eliminating the need to build Python yourself.

If the above doesn't work for you, I suggest simplifying by removing the extra configure options at first, i.e. just:

./configure

@ned-deily
Copy link
Member

ned-deily commented Mar 22, 2025

And for completeness, if you choose to install the Python built above, the next steps are:

% sudo make install
Password:
Creating directory /usr/local/bin
Creating directory /usr/local/lib
if test "no-framework" = "no-framework" ; then \
[...]
WARNING: Disabling truststore since ssl support is missing
Looking in links: /tmp/tmpda7fn65b
Processing /tmp/tmpda7fn65b/pip-25.0.1-py3-none-any.whl
Installing collected packages: pip
Successfully installed pip-25.0.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.

% /usr/local/bin/python3.14
Python 3.14.0a6+ (heads/main:8b7d20d3a9d, Mar 22 2025, 02:06:10) [Clang 16.0.0 (clang-1600.0.26.6)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

Since I didn't specify a --prefix= to ./configure, the Python is installed to the default location, /usr/local/bin, which will usually require using sudo and may interfere with some other third-party Python already installed. You can use --prefix= to specify another installation location, pretty much anywhere.

@wyatt-wong
Copy link
Author

Sorry I re-read the text and found there was NO errors at all. I was intimidated by the ./python.exe and thought it is an error.

Rebuilding with profile guided optimizations:
rm -f profile-clean-stamp
make build_all CFLAGS_NODIST=" -fprofile-instr-use="/Users/wyattwong/GitHub/cpython/code.profclangd"" LDFLAGS_NODIST=""
make[1]: Entering directory '/Users/wyattwong/GitHub/cpython'
Checked 112 modules (34 built-in, 78 shared, 0 n/a on macosx-15.3-x86_64, 0 disabled, 0 missing, 0 failed on import)
./python.exe -E ./Tools/build/generate-build-details.py `cat pybuilddir.txt`/build-details.json
make[1]: Leaving directory '/Users/wyattwong/GitHub/cpython'

@zware zware closed this as not planned Won't fix, can't repro, duplicate, stale Mar 22, 2025
@zware zware removed the type-bug An unexpected behavior, bug, or error label Mar 22, 2025
@wyatt-wong
Copy link
Author

And for completeness, if you choose to install the Python built above, the next steps are:

% sudo make install
Password:
Creating directory /usr/local/bin
Creating directory /usr/local/lib
if test "no-framework" = "no-framework" ; then \
[...]
WARNING: Disabling truststore since ssl support is missing
Looking in links: /tmp/tmpda7fn65b
Processing /tmp/tmpda7fn65b/pip-25.0.1-py3-none-any.whl
Installing collected packages: pip
Successfully installed pip-25.0.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.

% /usr/local/bin/python3.14
Python 3.14.0a6+ (heads/main:8b7d20d3a9d, Mar 22 2025, 02:06:10) [Clang 16.0.0 (clang-1600.0.26.6)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

Since I didn't specify a --prefix= to ./configure, the Python is installed to the default location, /usr/local/bin, which will usually require using sudo and may interfere with some other third-party Python already installed. You can use --prefix= to specify another installation location, pretty much anywhere.

I don't want to mess up my Python 3.13 with an alpha version of 3.14 unless I do it in a test environment. Furthermore it may not required sudo in macOS as opposed to Linux. In general it is discouraged to use sudo to execute make install in macOS. However as I said so I am not going to try it in my production macOS so I couldn't confirm if there is any errors or missing files running make install.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants