Update benchmarks

[ZuseZ4]

Splitting this out of the rust-benchmarks to make it easier to upstream rust stuff on it's own.

closes #1414

[ZuseZ4]

@wsmoses

CMake Error at /usr/lib/llvm-16/lib/cmake/llvm/AddLLVM.cmake:965 (add_executable):
  Target "enzyme-tblgen" links to target "zstd::libzstd_shared" but the
  target was not found.  Perhaps a find_package() call is missing for an
  IMPORTED target, or an ALIAS target is missing?
Call Stack (most recent call first):
  /usr/lib/llvm-16/lib/cmake/llvm/TableGen.cmake:146 (add_llvm_executable)
  tools/enzyme-tblgen/CMakeLists.txt:9 (add_tablegen)

and

➜  Enzyme git:(update-benchmarks) rg find_package . 
./enzyme/CMakeLists.txt
69:find_package(LLVM REQUIRED CONFIG)
188:    find_package(Clang REQUIRED CONFIG PATHS ${Clang_DIR} NO_DEFAULT_PATH)
200:    find_package(MLIR REQUIRED CONFIG PATHS ${MLIR_DIR} NO_DEFAULT_PATH)

./enzyme/test/test_find_package/CMakeLists.txt
5:find_package(Enzyme REQUIRED CONFIG NO_DEFAULT_PATH)

./diff.txt
13:@@ -71,14 +72,14 @@ find_package(LLVM REQUIRED CONFIG)

./enzyme/test/CMakeLists.txt
16:add_custom_target(test-cmake COMMAND rm -rf ${CMAKE_CURRENT_BINARY_DIR}/test_cmake && mkdir ${CMAKE_CURRENT_BINARY_DIR}/test_cmake && cd ${CMAKE_CURRENT_BINARY_DIR}/test_cmake &&  ${CMAKE_COMMAND} -S ${CMAKE_CURRENT_BINARY_DIR}/test_cmake ${CMAKE_CURRENT_SOURCE_DIR}/test_find_package -DEnzyme_DIR=${CMAKE_BINARY_DIR} -DCMAKE_C_COMPILER=$<TARGET_FILE:clang> && ${CMAKE_COMMAND} --build ${CMAKE_CURRENT_BINARY_DIR}/test_cmake DEPENDS ClangEnzymeFlags LLDEnzymeFlags)
18:add_custom_target(test-cmake COMMAND rm -rf ${CMAKE_CURRENT_BINARY_DIR}/test_cmake && mkdir ${CMAKE_CURRENT_BINARY_DIR}/test_cmake && cd ${CMAKE_CURRENT_BINARY_DIR}/test_cmake &&  ${CMAKE_COMMAND} -S ${CMAKE_CURRENT_BINARY_DIR}/test_cmake ${CMAKE_CURRENT_SOURCE_DIR}/test_find_package -DEnzyme_DIR=${CMAKE_BINARY_DIR} -DCMAKE_C_COMPILER=$<TARGET_FILE:clang> && ${CMAKE_COMMAND} --build ${CMAKE_CURRENT_BINARY_DIR}/test_cmake DEPENDS ClangEnzymeFlags)
➜  Enzyme git:(update-benchmarks) rg zstd
enzyme/WORKSPACE
39:    name = "llvm_zstd",
40:    build_file = "@llvm-raw//utils/bazel/third_party_build:zstd.BUILD",
42:    strip_prefix = "zstd-1.5.2",
44:        "https://github.com/facebook/zstd/releases/download/v1.5.2/zstd-1.5.2.tar.gz"

I won't look further into such cmake issues, but if you're ok with merging it despite those (or fixing them yourself), I can fix the unsupported flag combinations. As you can see it's already broken in the other PR where I just enable the benchmarks they are already broken, so fixing at least some things is IMHO a strict improvement.

[wsmoses]

Looks like an issue in setup for the script, can you copy whatever config from other CI ensures zlib is installed?

[ZuseZ4]

The current status is that no benchmark CI is working: https://github.com/EnzymeAD/Enzyme/actions/runs/10607142537/job/29399074787?pr=2056
With this PR at least 4 Benchmarks are run under llvm-16, so I'd recommend to merge it.
Happy to disable the still broken benchmarks, so someone else can pick them up once they have time.

[ZuseZ4]

lmk how to disable the benchmarks I haven't touched and which are thus still broken and this is good to merge and should even pass CI for 15 and 16.

[wsmoses]

Why is this commented out?

[ZuseZ4]

Some of the test files for some benchmarks were commented out before due to runtime. I just added a few more. I can drop this change here, but I'd suggest that we cut the number of tests that we run in CI
to keep CI time reasonable, and once we fixed all tests make it mandatory, to prevent this from regressing again.

[wsmoses]

So as is this removes the functionality for running the enzyme ablation hypothesis (by using the unified clang command to simplify rust linking).

can you add say another target for the reference pipeline which runs no optimization before enzyme AD?

[ZuseZ4]

It .. doesn't, can you check again, or point out where exactly it uses ClangEnzyme? :D
This should be cosmetic and just makes sure that these tests here at least build again.
It still uses LLVM-Enzyme for opt.
I sometimes define LOADCLANG="%loadClangEnzyme", because the RustBench will use this for optimal C++ perf, but it's not used in this PR. I can remove it from this PR and reintroduce it in the next PR that includes Rust benches if you want?

I assume that merging Rustbenches will require more discussions than I have time/energy before the deadline, so this is just a fix-mvp to reduce the diff between what I use for benching and what enzyme has.

[wsmoses]

Ah it looks like it was just in a second makefile (I thought it was a sed script which modified this in place):

Enzyme/enzyme/benchmarks/ReverseMode/ba/Makefile.makeafter

Line 13 in 87ea5eb

opt $^ $(LOAD) -enzyme -o $@ -S

Yeah this one should be modified too or ideally merged into the same makefile