Avoiding calling queries when collecting active queries

[Zoxc]

This PR changes active query collection to no longer call queries. Instead the fields needing queries have their computation delayed to when an cycle error is emitted or when printing the query backtrace in a panic.

This is done by splitting the fields in QueryStackFrame needing queries into a new QueryStackFrameExtra type. When collecting queries QueryStackFrame will contain a closure that can create QueryStackFrameExtra, which does make use of queries. Calling lift on a QueryStackFrame or CycleError will convert it to a variant containing QueryStackFrameExtra using those closures.

This also only calls queries needed to collect information on a cycle errors, instead of information on all active queries.

Calling queries when collecting active queries is a bit odd. Calling queries should not be done in the deadlock handler at all.

This avoids the out of memory scenario in #124901.

This is based on #138581.

[rustbot]

r? @fmease

rustbot has assigned @fmease.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[bors]

☔ The latest upstream changes (presumably #122156) made this pull request unmergeable. Please resolve the merge conflicts.

[fmease]

r? oli-obk or fee1-dead?

[rust-log-analyzer]

The job mingw-check-tidy failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

info: removing rustup binaries
info: rustup is uninstalled
##[group]Image checksum input
mingw-check-tidy
# We use the ghcr base image because ghcr doesn't have a rate limit
# and the mingw-check-tidy job doesn't cache docker images in CI.
FROM ghcr.io/rust-lang/ubuntu:22.04

ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
  g++ \
  make \
---

COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
COPY host-x86_64/mingw-check/validate-error-codes.sh /scripts/

# NOTE: intentionally uses python2 for x.py so we can test it still works.
# validate-toolstate only runs in our CI, so it's ok for it to only support python3.
ENV SCRIPT TIDY_PRINT_DIFF=1 python2.7 ../x.py test \
           --stage 0 src/tools/tidy tidyselftest --extra-checks=py,cpp
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --allow-unsafe --generate-hashes reuse-requirements.in
---
#12 2.892 Building wheels for collected packages: reuse
#12 2.894   Building wheel for reuse (pyproject.toml): started
#12 3.104   Building wheel for reuse (pyproject.toml): finished with status 'done'
#12 3.105   Created wheel for reuse: filename=reuse-4.0.3-cp310-cp310-manylinux_2_35_x86_64.whl size=132720 sha256=0c2fd2aaf7b0bf8d6e131220aff14712a774c2ca462f3204d25460cbcf610b63
#12 3.105   Stored in directory: /tmp/pip-ephem-wheel-cache-cx0qaqjc/wheels/3d/8d/0a/e0fc6aba4494b28a967ab5eaf951c121d9c677958714e34532
#12 3.107 Successfully built reuse
#12 3.108 Installing collected packages: boolean-py, binaryornot, tomlkit, reuse, python-debian, markupsafe, license-expression, jinja2, chardet, attrs
#12 3.503 Successfully installed attrs-23.2.0 binaryornot-0.4.4 boolean-py-4.0 chardet-5.2.0 jinja2-3.1.4 license-expression-30.3.0 markupsafe-2.1.5 python-debian-0.1.49 reuse-4.0.3 tomlkit-0.13.0
#12 3.503 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#12 4.027 Collecting virtualenv
#12 4.064   Downloading virtualenv-20.29.3-py3-none-any.whl (4.3 MB)
#12 4.147      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.3/4.3 MB 52.9 MB/s eta 0:00:00
#12 4.201 Collecting platformdirs<5,>=3.9.1
#12 4.203   Downloading platformdirs-4.3.7-py3-none-any.whl (18 kB)
#12 4.221 Collecting distlib<1,>=0.3.7
#12 4.224   Downloading distlib-0.3.9-py2.py3-none-any.whl (468 kB)
#12 4.231      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 469.0/469.0 KB 87.7 MB/s eta 0:00:00
#12 4.265 Collecting filelock<4,>=3.12.2
#12 4.268   Downloading filelock-3.18.0-py3-none-any.whl (16 kB)
#12 4.350 Installing collected packages: distlib, platformdirs, filelock, virtualenv
#12 4.530 Successfully installed distlib-0.3.9 filelock-3.18.0 platformdirs-4.3.7 virtualenv-20.29.3
#12 4.531 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#12 DONE 4.6s

#13 [7/8] COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
#13 DONE 0.0s
---
DirectMap4k:      131008 kB
DirectMap2M:     8257536 kB
DirectMap1G:    10485760 kB
##[endgroup]
Executing TIDY_PRINT_DIFF=1 python2.7 ../x.py test            --stage 0 src/tools/tidy tidyselftest --extra-checks=py,cpp
+ TIDY_PRINT_DIFF=1 python2.7 ../x.py test --stage 0 src/tools/tidy tidyselftest --extra-checks=py,cpp
##[group]Building bootstrap
    Finished `dev` profile [unoptimized] target(s) in 0.05s
##[endgroup]
WARN: currently no CI rustc builds have rustc debug assertions enabled. Please either set `rust.debug-assertions` to `false` if you want to use download CI rustc or set `rust.download-rustc` to `false`.
[TIMING] core::build_steps::tool::LibcxxVersionTool { target: x86_64-unknown-linux-gnu } -- 0.213
---
##[endgroup]
[TIMING] core::build_steps::tool::ToolBuild { compiler: Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, target: x86_64-unknown-linux-gnu, tool: "tidy", path: "src/tools/tidy", mode: ToolBootstrap, source_type: InTree, extra_features: [], allow_features: "", cargo_args: [], artifact_kind: Binary } -- 36.583
[TIMING] core::build_steps::tool::Tidy { compiler: Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, target: x86_64-unknown-linux-gnu } -- 0.000
fmt check
Diff in /checkout/compiler/rustc_query_impl/src/plumbing.rs:86:
     /// Returns a query map representing active query jobs.
     /// It returns an incomplete map as an error if it fails
     /// to take locks.
-    fn collect_active_jobs(self) -> Result<QueryMap<QueryStackDeferred<'tcx>>, QueryMap<QueryStackDeferred<'tcx>>> {
+    fn collect_active_jobs(
+        self,
+    ) -> Result<QueryMap<QueryStackDeferred<'tcx>>, QueryMap<QueryStackDeferred<'tcx>>> {
         let mut jobs = QueryMap::default();
         let mut complete = true;
 
Diff in /checkout/compiler/rustc_query_impl/src/plumbing.rs:93:
         for collect in super::TRY_COLLECT_ACTIVE_JOBS.iter() {
             if collect(self.tcx, &mut jobs).is_none() {
                 complete = false;
-        }
+            }
         }
 
         if complete { Ok(jobs) } else { Err(jobs) }
fmt: checked 5924 files
Build completed unsuccessfully in 0:01:18
  local time: Sat Mar 22 20:06:16 UTC 2025
  network time: Sat, 22 Mar 2025 20:06:16 GMT
##[error]Process completed with exit code 1.