Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor HIR item-like traversal #95004

Open
2 of 7 tasks
cjgillot opened this issue Mar 16, 2022 · 4 comments
Open
2 of 7 tasks

Refactor HIR item-like traversal #95004

cjgillot opened this issue Mar 16, 2022 · 4 comments
Assignees
Labels
A-HIR Area: The high-level intermediate representation (HIR) E-help-wanted Call for participation: Help is requested to fix this issue. E-medium Call for participation: Medium difficulty. Experience needed to fix: Intermediate. E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion.

Comments

@cjgillot
Copy link
Contributor

cjgillot commented Mar 16, 2022

HIR is globally traversed using the tcx.hir().{,par_}visit_all_item_likes{,_in_module} family of functions. These functions create a lot of dependency edges (they call the hir_crate query, or call hir_owner for each item-like). These dependencies are not always useful: many use sites only require items, and don't care for impl items, trait items or foreign items.

This can be avoided by replacing those functions by a scheme based on ItemIds. Actual user will be responsible for calling tcx.hir().item to get the HIR. The amount of access to HIR will be further reduced by pre-filtering using tcx.def_kind query.

Steps:

  • create a hir_crate_items query which traverses tcx.hir_crate(()).owners to collect all the ids and return a rustc_middle::hir::ModuleItems;
  • use tcx.hir_crate_items in tcx.hir().items to return an iterator of hir::ItemId;
  • use tcx.hir_crate_items to introduce a tcx.hir().par_items(impl Fn(hir::ItemId)) to traverse all items in parallel;
  • gradually replace uses of tcx.hir().{,par_}visit_all_item_likes with tcx.hir().par_items or tcx.hir().items;
    when possible, create a fast path using tcx.def_kind(item_id.def_id) before calling tcx.hir().item(item_id);
  • introduce tcx.hir().items_in_module and tcx.hir().par_items_in_module which do the same thing with tcx.hir_module_items instead of tcx.hir_crate_items;
  • gradually replace uses of tcx.hir().visit_all_item_likes_in_module with tcx.hir().par_items_in_module or tcx.hir().items_in_module;
  • retire hir::ItemLikeVisitor.

The steps starting with "gradually" can be split into several PRs.
I am available on zulip for any question.

@cjgillot cjgillot added E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. A-HIR Area: The high-level intermediate representation (HIR) E-help-wanted Call for participation: Help is requested to fix this issue. E-medium Call for participation: Medium difficulty. Experience needed to fix: Intermediate. labels Mar 16, 2022
@kckeiks
Copy link
Contributor

kckeiks commented Mar 16, 2022

@rustbot claim

@kckeiks
Copy link
Contributor

kckeiks commented Mar 16, 2022

Not sure why rustbot is ignoring me but I'd like to pick this up. I'll have some bandwidth in the next couple of weeks to work on this.

bors added a commit to rust-lang-ci/rust that referenced this issue Apr 17, 2022
… r=cjgillot

Refactor HIR item-like traversal (part 1)

Issue  rust-lang#95004

- Create hir_crate_items query which traverses tcx.hir_crate(()).owners to return a hir::ModuleItems
- use tcx.hir_crate_items in tcx.hir().items() to return an iterator of hir::ItemId
- use tcx.hir_crate_items to introduce a tcx.hir().par_items(impl Fn(hir::ItemId)) to traverse all items in parallel;

Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>

cc `@cjgillot`
flip1995 pushed a commit to flip1995/rust that referenced this issue Apr 21, 2022
… r=cjgillot

Refactor HIR item-like traversal (part 1)

Issue  rust-lang#95004

- Create hir_crate_items query which traverses tcx.hir_crate(()).owners to return a hir::ModuleItems
- use tcx.hir_crate_items in tcx.hir().items() to return an iterator of hir::ItemId
- use tcx.hir_crate_items to introduce a tcx.hir().par_items(impl Fn(hir::ItemId)) to traverse all items in parallel;

Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>

cc `@cjgillot`
bors added a commit to rust-lang-ci/rust that referenced this issue May 7, 2022
…-rustc-typeck, r=cjgillot

Remove ItemLikeVisitor impls from rustc_typeck

Issue rust-lang#95004
cc `@cjgillot`
bors added a commit to rust-lang-ci/rust that referenced this issue May 17, 2022
…t, r=cjgillot

 Retire `ItemLikeVisitor` trait

Issue rust-lang#95004
cc `@cjgillot`
xFrednet pushed a commit to xFrednet/rust that referenced this issue May 21, 2022
…-rustc-typeck, r=cjgillot

Remove ItemLikeVisitor impls from rustc_typeck

Issue rust-lang#95004
cc `@cjgillot`
xFrednet pushed a commit to xFrednet/rust that referenced this issue May 21, 2022
…t, r=cjgillot

 Retire `ItemLikeVisitor` trait

Issue rust-lang#95004
cc `@cjgillot`
@kckeiks
Copy link
Contributor

kckeiks commented Jun 11, 2022

@cjgillot I think #96825, #96531, and #95655 have address the tasks in this issue. We have created hir_crate_items, replaced visit_all_item_likes where appropriate, and retired ItemLikeVisitor. Please let me know if we can close this issue or if there is anything else we should address.

If the parallelization of the compiler still in early stages, maybe it would be better to address the parallelization-related tasks in this issue in a future issue along with other parallelization-related endeavors? If so, I'll leave it up to you to review and approve or close #97998.

@oli-obk oli-obk reopened this Mar 20, 2025
@oli-obk
Copy link
Contributor

oli-obk commented Mar 20, 2025

I'm doing some more work here. My current goal is to remove almost all direct callers to the hir_crate query and instead rely on other queries

bors added a commit to rust-lang-ci/rust that referenced this issue Mar 20, 2025
[perf] Decouple directly accessing a HIR owner from ast lowering

r? `@ghost`

cc rust-lang#95004
bors added a commit to rust-lang-ci/rust that referenced this issue Mar 21, 2025
…try>

Avoid directly accessing the hir_crate query from crate_hash

based on rust-lang#138772

cc rust-lang#95004

Move the crate hashing to the crate_hash query and make it work off hir_crate_items instead of hir_crate.

r? `@ghost`
bors added a commit to rust-lang-ci/rust that referenced this issue Mar 21, 2025
…try>

Avoid directly accessing the hir_crate query from crate_hash

based on rust-lang#138772

cc rust-lang#95004

Move the crate hashing to the crate_hash query and make it work off hir_crate_items instead of hir_crate.

r? `@ghost`
Kobzol added a commit to Kobzol/rust that referenced this issue Mar 21, 2025
…e1-dead

Make `crate_hash` not iterate over `hir_crate` owners anymore

cc rust-lang#95004

One more direct usage of hir::Crate removed
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Mar 21, 2025
…e1-dead

Make `crate_hash` not iterate over `hir_crate` owners anymore

cc rust-lang#95004

One more direct usage of hir::Crate removed
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Mar 21, 2025
…e1-dead

Make `crate_hash` not iterate over `hir_crate` owners anymore

cc rust-lang#95004

One more direct usage of hir::Crate removed
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Mar 22, 2025
…e1-dead

Make `crate_hash` not iterate over `hir_crate` owners anymore

cc rust-lang#95004

One more direct usage of hir::Crate removed
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Mar 22, 2025
Rollup merge of rust-lang#138750 - oli-obk:decouple-hir-queries, r=fee1-dead

Make `crate_hash` not iterate over `hir_crate` owners anymore

cc rust-lang#95004

One more direct usage of hir::Crate removed
bors added a commit to rust-lang-ci/rust that referenced this issue Mar 26, 2025
[perf experiment] Split the resolver tables into per-owner tables

r? `@ghost`

just doing some experiments to see if splitting `hir_crate` is feasible by checking if splitting the resolver's output into per-owner queries is feasible (rust-lang#95004)

Basically necessary for rust-lang#138705 as that can't be landed perf-wise while the `hir_crate` query is still a thing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-HIR Area: The high-level intermediate representation (HIR) E-help-wanted Call for participation: Help is requested to fix this issue. E-medium Call for participation: Medium difficulty. Experience needed to fix: Intermediate. E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion.
Projects
None yet
Development

No branches or pull requests

3 participants