test rustc-dev dist component on CI

[Kobzol]

Recently, the rustc-dev component was broken several times in a row (#138304, #138647).

It would be nice if we had some CI check that would at least run cargo metadata on the extracted rustc-src component.

CC @RalfJung - Would running cargo metadata be enough as a smoke test?

[jieyouxu]

(Related to #136822)

[jieyouxu]

I also don't think this is really compiletest, it probably needs to be its own test suite post-dist doing some basic checks.

[RalfJung]

Would running cargo metadata be enough as a smoke test?

Yes, at least for these two recent issues that would have been enough.
I was about to open an issue asking @rust-lang/infra for what is the best way to test this. :)

[RalfJung]

(Related to #136822)

Maybe let's make this issue here specific for rustc-src, and use the other one to track more potential tests.

[jieyouxu]

I wonder if it makes sense to have something like citool, but post-dist build, which actually exercises the final dist artifacts (literally unpacking from a dist archive, for instance) that the users would actually receive.

(Testing rustc-src dist component may be one of its jobs)

[Kobzol]

I mean, in the ideal world we should do all (or almost all) of our testing on the dist artifacts, but that woud be too costly CI wise at the moment (apart from other problems).

The rustc-src component should IMO have its own separate job, rather than being a part of the x64 dist builder. Then we could add specific smoke tests to it.

[jieyouxu]

Wait, wouldn't we still have to produce the rustc-src dist component in the first place? Or as in a fast custom dist component (say at stage 1 or sth)?

[Kobzol]

Yeah I meant to create a new CI job that would create the dist rustc-src component. It doesn't require almost anything and should be very fast to build.

[Kobzol]

Just a note that this problem occurred in the rustc-dev component, not rustc-src.

[RalfJung]

Ah, the folder is called rustc-src but the component rustc-dev.