CI failing due to jemalloc symbol changes

[Mark-Simulacrum]

It looks like rust-lang/rust#134690 may have changed the jemalloc symbols (or perhaps a previous PR, haven't bisected yet):

$ nm /home/mark/Build/rustc-perf/cache/7a0cde96f83c6d38237bb8062df6300ecf4c2687/bin/rustc  | grep jemalloc
0000000000074870 t jemalloc_constructor
0000000000074880 t jemalloc_postfork_child
00000000000704d0 t jemalloc_postfork_parent
0000000000070aa0 t jemalloc_prefork
$ nm ~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc | grep jemalloc
0000000000020440 t _rjem_je_jemalloc_postfork_child
0000000000020360 t _rjem_je_jemalloc_postfork_parent
00000000000201e0 t _rjem_je_jemalloc_prefork
00000000000201d0 t jemalloc_constructor

I suspect that also affected valgrind's detection of hitting jemalloc code and as a result we no longer see it in profiles, though it's also in theory possible LTO just optimized the code better...

[Kobzol]

Yes, it was indeed that PR. I will take a look at how to fix rustc-perf's CI (I didn't understand lqd's comment first, now I get it).

[Mark-Simulacrum]

Yeah, just also bisected/confirmed it was that PR.

[Kobzol]

After that PR, jemalloc functions are used directly as malloc/free/etc. function symbols, if I understand it correctly, so this special detection no longer works. I suppose that we can just remove that check from the check-profiling.sh script?

[Mark-Simulacrum]

Hm, I thought that was already the case? Or are you saying we removed a layer of indirection where previously there was some kind of malloc stub function between the rustc alloc and the jemalloc alloc, that got removed with LTO?

I'd sort of expect that the detection should still work though since presumably not all of jemalloc's code got inlined and we'd expect some of it to be present? Maybe we turned off debuginfo at some point and so that's why we're no longer able to determine that it's jemalloc vs. not?

[Mark-Simulacrum]

I'm generally OK dropping the check, it seems likely that we still get attribution to at least malloc() hopefully? That might be good to check -- otherwise it can be hard to interpret diffs if it's mostly due to allocator traffic increasing.

[Kobzol]

Hm, I thought that was already the case? Or are you saying we removed a layer of indirection where previously there was some kind of malloc stub function between the rustc alloc and the jemalloc alloc, that got removed with LTO?

Yes, but I'm actually surprised that LTO was what caused it to happen. Because we should have been doing that even before due to usage of unprefixed_malloc_on_supported_platforms, but for some reason it probably wasn't applied?

I'd sort of expect that the detection should still work though since presumably not all of jemalloc's code got inlined and we'd expect some of it to be present? Maybe we turned off debuginfo at some point and so that's why we're no longer able to determine that it's jemalloc vs. not?

Yes, together with the LTO change, we also started stripping debuginfo from the rustc binary, because the debuginfo somehow got quite larger with LTO enabled (but even before, the debuginfo was extraneous, I think).

I'm generally OK dropping the check, it seems likely that we still get attribution to at least malloc() hopefully? That might be good to check -- otherwise it can be hard to interpret diffs if it's mostly due to allocator traffic increasing.

I will try to add some extra allocations to the compiler and see what happens in the profiles.

[Kobzol]

      94,325    malloc_default

The symbol names are different than before, and they are no longer unified under the "all-jemalloc-functions" group. But entries like malloc/realloc/... (and other malloc abstraction functions) are still visible in the profile.