Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iOS binaries crash with latest nightly #138212

Closed
tdomhan opened this issue Mar 8, 2025 · 41 comments · Fixed by #138695
Closed

iOS binaries crash with latest nightly #138212

tdomhan opened this issue Mar 8, 2025 · 41 comments · Fixed by #138695
Labels
A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. C-bug Category: This is a bug. C-external-bug Category: issue that is caused by bugs in software beyond our control llvm-fixed-upstream Issue expected to be fixed by the next major LLVM upgrade, or backported fixes O-apple Operating system: Apple (macOS, iOS, tvOS, visionOS, watchOS) O-ios Operating system: iOS P-high High priority regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@tdomhan
Copy link

tdomhan commented Mar 8, 2025

iOS builds (aarch64-apple-ios) of Dioxus apps crash on nightly-2025-02-18 (ce36a96). The latest working version is nightly-2025-02-17 (5bc6231).
This problem only occurs when running on a real device, not when running the simulator.

To reproduce the error do the following:

# Create a new Dioxus app
 dx new test_app
# Create application bundle
rustup override set nightly-2025-02-18 && rustup target add aarch64-apple-ios && dx bundle --platform ios  --trace --device true

Next sign the application, i.e. via iOS App Signer and install on a real device.

The log console shows the following errors:

Snapshot generation request for bundleID: com.appidentifier rejected due to the app being denylisted.
...
Process exited: <FBApplicationProcess: 0x913f14000; app<com.appidentifier>:<invalid>> ->
Application process state changed for com.appidentifier: (null)
...

NSUnderlyingError = <NSError: 0x302466070; domain: NSPOSIXErrorDomain; code: 88>
...
SpringBoard	[app<com.appidentifier>:-1] Now flagged as pending exit for reason: Bootstrap failed

The problem must have been introduced somewhere between commits ce36a96 and 5bc6231. I'd be happy to also test any other rustc binaries or try provide any other information.

@tdomhan tdomhan added the C-bug Category: This is a bug. label Mar 8, 2025
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Mar 8, 2025
@tdomhan
Copy link
Author

tdomhan commented Mar 8, 2025

Probably unrelated, but looking for "Snapshot generation request for bundleID: com.appidentifier rejected due to the app being denylisted." one finds fyne-io/fyne#2270 and golang/go#47952 where there was a similar crash in golang. Could be unrelated as the error message is of course very unspecific.

@jieyouxu jieyouxu added O-ios Operating system: iOS E-needs-bisection Call for participation: This issue needs bisection: https://github.com/rust-lang/cargo-bisect-rustc T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. regression-untriaged Untriaged performance or correctness regression. labels Mar 8, 2025
@rustbot rustbot added the I-prioritize Issue: Indicates that prioritization has been requested for this issue. label Mar 8, 2025
@jieyouxu jieyouxu added regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. and removed regression-untriaged Untriaged performance or correctness regression. labels Mar 8, 2025
@jieyouxu
Copy link
Member

jieyouxu commented Mar 8, 2025

Is it possible for you to bisect between ce36a96 and 5bc6231? There shouldn't be too many commits in between, and the steps you described is a bit specific (and requires a real iOS device).

Or maybe someone with a real iOS device can try to bisect this...

@jieyouxu jieyouxu added regression-untriaged Untriaged performance or correctness regression. and removed regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. labels Mar 8, 2025
@tdomhan
Copy link
Author

tdomhan commented Mar 8, 2025

Yeah, indeed it is non-trivial to reproduce in terms of setup. I can try but I have never used a custom rustc version and I'm not sure how easy it is to set up for iOS builds. If you have any pointers let me know.

@tdomhan
Copy link
Author

tdomhan commented Mar 8, 2025

Looks like I can use https://github.com/kennytm/rustup-toolchain-install-master to install specific commits.
Update: never mind. Artifacts don't exist for all commits.

@Noratrieb
Copy link
Member

You may be able to use cargo-bisect-rustc which has all the necessary logic

@tdomhan
Copy link
Author

tdomhan commented Mar 8, 2025

Unfortunately I won't have the disk space/setup to build custom rustc versions, but would be happy to test if they can be built elsewhere. I have no experience with rustc, but from the commit list the LLVM 20 update stands out.

@tdomhan
Copy link
Author

tdomhan commented Mar 8, 2025

Thanks for the pointer to cargo-bisect-rustc. I'll give that a go.

@Noratrieb Noratrieb added regression-from-stable-to-beta Performance or correctness regression from stable to beta. and removed regression-untriaged Untriaged performance or correctness regression. needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Mar 8, 2025
@tdomhan
Copy link
Author

tdomhan commented Mar 8, 2025

Actually I was able to install and test 2162e9d which works. This only leaves 1873bd3 and ce36a96 (LLVM20).

@jieyouxu jieyouxu added the A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. label Mar 8, 2025
@jieyouxu
Copy link
Member

jieyouxu commented Mar 8, 2025

Tagging with LLVM but it might not be the direct cause

@Noratrieb
Copy link
Member

@rustbot ping apple

@rustbot rustbot added the O-apple Operating system: Apple (macOS, iOS, tvOS, visionOS, watchOS) label Mar 8, 2025
@rustbot
Copy link
Collaborator

rustbot commented Mar 8, 2025

Hey Apple notification group! This issue or PR could use some Apple-specific
guidance. Could one of you weigh in? Thanks <3

(In case it's useful, here are some instructions for tackling these sorts of
issues).

cc @BlackHoleFox @hkratz @inflation @madsmtm @nvzqz @shepmaster @thomcc

@madsmtm
Copy link
Contributor

madsmtm commented Mar 8, 2025

To be clear, does the error come before the application even starts? As-in, it occurs as part of the installation process/before the process is launched? Or does it happen before fn main is called (i.e. as part of the dyld startup routines)? Or does the crash happen as part of UIApplicationMain?

In any case, I think this could be caused by one of:

  • compiler-builtins started using some unsupported symbols.
  • LLVM 20 doing invalid codegen that is caught by a consistency check somewhere.

I don't have a real iOS device to test with at the moment (would take me a while to test this with a borrowed device), could I get you try the following to help narrow down the problem:

  1. Build with IPHONEOS_DEPLOYMENT_TARGET=18.0
  2. Build with dx bundle, but with a simple fn main() {} binary instead.
  3. Use something else to run the binary on the device, like cargo dinghy (or the underlying ios-deploy)

@madsmtm
Copy link
Contributor

madsmtm commented Mar 8, 2025

A quick look at the difference in mach-o headers between a binary generated by nightly-2025-02-17 and nightly-2025-02-18 didn't reveal any insight other than nightly-2025-02-18 load commands having a large fileoffset for some reason? But that could easily be a red herring.

@TimNN
Copy link
Contributor

TimNN commented Mar 8, 2025

I can reproduce this:

  • with a simple fn main() { println!("Hello, ICR 18!"); }, without dixoxus in the deps, but built with dx bundle
  • with export IPHONEOS_DEPLOYMENT_TARGET=18.0 active in the shell invoking dx bundle
  • by launching the binary using xcrun devicectl, as described by Long-term solution to replacing ios-deploy flutter/flutter#133465 (comment), though the --start--stopped flag doesn't do anything, since the binary technically isn't crashing, it fails to launch

The output I get from xcrun is:

ERROR: The application failed to launch. (com.apple.dt.CoreDeviceError error 10002 (0x2712))
       BundleIdentifier = com.example.IcrRepro
--------------------------------------------------------------------------------
ERROR:     The request to open "com.example.IcrRepro" failed. (FBSOpenApplicationServiceErrorDomain error 1 (0x01))
           FBSOpenApplicationRequestID = 0x3e71
           BSErrorCodeDescription = RequestDenied
           NSLocalizedFailureReason = The request was denied by service delegate (SBMainWorkspace).
--------------------------------------------------------------------------------
ERROR:         The operation couldn’t be completed. The process failed to launch. (FBProcessExit error 64 (0x40))
               BSErrorCodeDescription = launch-failed
               NSLocalizedFailureReason = The process failed to launch.
--------------------------------------------------------------------------------
ERROR:             The operation couldn’t be completed. Launch failed. (RBSRequestErrorDomain error 5 (0x05))
                   NSLocalizedFailureReason = Launch failed.
--------------------------------------------------------------------------------
ERROR:                 Launchd job spawn failed (NSPOSIXErrorDomain error 88 (0x58))

@jieyouxu jieyouxu removed the E-needs-bisection Call for participation: This issue needs bisection: https://github.com/rust-lang/cargo-bisect-rustc label Mar 9, 2025
@TimNN
Copy link
Contributor

TimNN commented Mar 9, 2025


Current steps to reproduce:

echo "fn main() {}" >> unstripped.rs

# Rust version doesn't matter for this
rustc +stable --target=aarch64-apple-ios unstripped.rs

$(rustc +nightly --print=sysroot)/lib/rustlib/aarch64-apple-darwin/bin/rust-objcopy unstripped stripped

# This is fine
dyld_info unstripped

# This fails
dyld_info stripped

@jieyouxu
Copy link
Member

jieyouxu commented Mar 9, 2025

Right. The LLVM fix I meant as being "possibly related" not that it fixes this (it obviously is broken 😆).

@TimNN
Copy link
Contributor

TimNN commented Mar 9, 2025

This regressed in llvm/llvm-project@1a830aa, I'll open an upstream bug

@TimNN
Copy link
Contributor

TimNN commented Mar 9, 2025

Upstream: llvm/llvm-project#130472

@jieyouxu jieyouxu added the C-external-bug Category: issue that is caused by bugs in software beyond our control label Mar 9, 2025
@tdomhan
Copy link
Author

tdomhan commented Mar 9, 2025

@tdomhan or someone else with access to macOS -> iOS cross env can you try the toolchain 06d1077d5ba6da9a0b3b6b84e6f919ec600dc1b6 with rustup-toolchain-install-master (from #138250 (comment)) to see if that can produce an iOS binary that doesn't crash?

@jieyouxu : yeah absolutely. Unfortunately, I run into:
error: missing component `rustc` on toolchain `06d1077d5ba6da9a0b3b6b84e6f919ec600dc1b6` on channel `nightly` for target `aarch64-apple-darwin

@TimNN
Copy link
Contributor

TimNN commented Mar 9, 2025

@tdomhan: Could you confirm whether setting

[profile.release]
strip = false

in your Cargo.toml works around the issue?

@tdomhan
Copy link
Author

tdomhan commented Mar 9, 2025

Yes, setting strip = false does address the issue.

@Noratrieb Noratrieb added P-high High priority and removed I-prioritize Issue: Indicates that prioritization has been requested for this issue. labels Mar 9, 2025
jieyouxu added a commit to jieyouxu/rust that referenced this issue Mar 9, 2025
For <rust-lang#138212>.

Looks like LLVM 20's initial `llvm-objcopy` version that we
ship as `rust-objcopy` may be producing stripped iOS binaries with
invalid offsets that fail iOS platform consistency checks.

For the time being, use macOS system strip at `/usr/bin/strip` which
should be available (unlike Linux -> macOS cross where this is not
guaranteed to be available, partially why we are shipping `llvm-objcopy`
in the first place) that shouldn't have this offset problem.
jieyouxu added a commit to jieyouxu/rust that referenced this issue Mar 9, 2025
For <rust-lang#138212>.

Looks like LLVM 20's initial `llvm-objcopy` version that we
ship as `rust-objcopy` may be producing stripped iOS binaries with
invalid offsets that fail iOS platform consistency checks.

For the time being, use macOS system strip at `/usr/bin/strip` which
should be available (unlike Linux -> macOS cross where this is not
guaranteed to be available, partially why we are shipping `llvm-objcopy`
in the first place) that shouldn't have this offset problem.
@jieyouxu
Copy link
Member

jieyouxu commented Mar 9, 2025

@tdomhan or someone else with access to macOS -> iOS cross env can you try the toolchain 06d1077d5ba6da9a0b3b6b84e6f919ec600dc1b6 with rustup-toolchain-install-master (from #138250 (comment)) to see if that can produce an iOS binary that doesn't crash?

@jieyouxu : yeah absolutely. Unfortunately, I run into: error: missing component `rustc` on toolchain `06d1077d5ba6da9a0b3b6b84e6f919ec600dc1b6` on channel `nightly` for target `aarch64-apple-darwin

Ah... I'm not sure what toolchain the try-job actually produced 🤔

EDIT: I think it's because I forgot to specify a dist-aarch64-apple try-job...

bors added a commit to rust-lang-ci/rust that referenced this issue Mar 9, 2025
Use `/usr/bin/strip` on macOS -> iOS cross as a temporary workaround

Crude workaround for <rust-lang#138212> to unblock nightly macOS-to-iOS cross builds. This reintroduces the "hard-coded strip" workaround (but only for iOS) that was initially introduced in rust-lang#130781.

Looks like LLVM 20's initial `llvm-objcopy` version that we ship as `rust-objcopy` may be producing stripped iOS binaries with invalid offsets that fail iOS platform consistency checks.

For the time being, use macOS system strip when cross-compiling from macOS -> iOS at `/usr/bin/strip` which should be available (unlike Linux -> macOS cross where this is not guaranteed to be available, partially why we are shipping `llvm-objcopy` in the first place[^linux-darwin-cross]) that shouldn't have this offset problem.

### Alternatives

- We *could* also try to use a `strip` from PATH. *However*, that can regress iOS users if they have broken homebrew binutils strip in their PATH that take precedence over a "good" strip.
- We could also wait for upstream `llvm-objcopy` fixes.

### Review and testing advice

I don't have access to either macOS or iOS platforms, and I can't really write a test for this. This will need to be tested manually.

r? `@davidtwco` (or reroll)
cc *-apple-ios target maintainers `@badboy` `@deg4uss3r` `@madsmtm`

try-job: dist-apple-various
try-job: dist-x86_64-apple
try-job: dist-aarch64-apple

[^linux-darwin-cross]: rust-lang#131206
@jieyouxu
Copy link
Member

jieyouxu commented Mar 9, 2025

@tdomhan or someone else with access to macOS -> iOS cross env can you try the toolchain 06d1077d5ba6da9a0b3b6b84e6f919ec600dc1b6 with rustup-toolchain-install-master (from #138250 (comment)) to see if that can produce an iOS binary that doesn't crash?

@jieyouxu : yeah absolutely. Unfortunately, I run into: error: missing component `rustc` on toolchain `06d1077d5ba6da9a0b3b6b84e6f919ec600dc1b6` on channel `nightly` for target `aarch64-apple-darwin

@tdomhan does this work for you? I ran another try-job but this time with actually aarch64-apple-darwin dist...

rustup-toolchain-install-master d1f7fb3751190d6b1c8642ea402d066147d01e28 --host=aarch64-apple-darwin

@TimNN
Copy link
Contributor

TimNN commented Mar 10, 2025

A fix has been proposed upstream (not merged yet): llvm/llvm-project#130517

@tdomhan
Copy link
Author

tdomhan commented Mar 10, 2025

@tdomhan or someone else with access to macOS -> iOS cross env can you try the toolchain 06d1077d5ba6da9a0b3b6b84e6f919ec600dc1b6 with rustup-toolchain-install-master (from #138250 (comment)) to see if that can produce an iOS binary that doesn't crash?

@jieyouxu : yeah absolutely. Unfortunately, I run into: error: missing component `rustc` on toolchain `06d1077d5ba6da9a0b3b6b84e6f919ec600dc1b6` on channel `nightly` for target `aarch64-apple-darwin

@tdomhan does this work for you? I ran another try-job but this time with actually aarch64-apple-darwin dist...

rustup-toolchain-install-master d1f7fb3751190d6b1c8642ea402d066147d01e28 --host=aarch64-apple-darwin

does it also include aarch64-apple-ios? I'm able to run rustup-toolchain-install-master d1f7fb3751190d6b1c8642ea402d066147d01e28 --host=aarch64-apple-darwin but get error can't find crate for core and rustup target add aarch64-apple-ios does not work (error: toolchain 'd1f7fb3751190d6b1c8642ea402d066147d01e28' does not support components). Let me know if I'm doing something wrong 🙈

@jieyouxu
Copy link
Member

Ugh, maybe that dist toolchain doesn't include iOS. I'll wait for the apple target maintainers to double-check, or we can just wait for upstream fix.

@madsmtm
Copy link
Contributor

madsmtm commented Mar 10, 2025

Yeah, you probably need dist-apple-various for that (though unsure if that then also includes host tooling or not).

@TimNN
Copy link
Contributor

TimNN commented Mar 18, 2025

The fix for this has been backported into LLVM's release/20.x branch in llvm/llvm-project@cb50aaf

edit: According to https://discourse.llvm.org/t/llvm-20-1-0-released/85122, LLVM 20.1.1 will be releases in ~a week.

@TimNN
Copy link
Contributor

TimNN commented Mar 18, 2025

Is the regression-from-stable-to-beta label accurate?

AFAICT, LLVM 20 isn't included in the current beta.

(So we have a bit more time to wait for the LLVM patch to make it into Rust's LLVM version).

@dianqk dianqk added the llvm-fixed-upstream Issue expected to be fixed by the next major LLVM upgrade, or backported fixes label Mar 19, 2025
@dianqk dianqk added regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. and removed regression-from-stable-to-beta Performance or correctness regression from stable to beta. labels Mar 19, 2025
@bors bors closed this as completed in 87e60a7 Mar 20, 2025
github-actions bot pushed a commit to rust-lang/rustc-dev-guide that referenced this issue Mar 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. C-bug Category: This is a bug. C-external-bug Category: issue that is caused by bugs in software beyond our control llvm-fixed-upstream Issue expected to be fixed by the next major LLVM upgrade, or backported fixes O-apple Operating system: Apple (macOS, iOS, tvOS, visionOS, watchOS) O-ios Operating system: iOS P-high High priority regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants