MacrobenchmarkRule


public final class MacrobenchmarkRule implements TestRule


JUnit rule for benchmarking large app operations like startup, scrolling, or animations.

    @get:Rule
val benchmarkRule = MacrobenchmarkRule()

@Test
fun startup() = benchmarkRule.measureRepeated(
packageName = "com.example.my.application.id"
metrics = listOf(StartupTimingMetric()),
iterations = 5,
startupMode = StartupMode.COLD,
setupBlock = {
pressHome()
}
) { // this = MacrobenchmarkScope
val intent = Intent()
intent.setPackage("mypackage.myapp")
intent.setAction("mypackage.myapp.myaction")
startActivityAndWait(intent)
}

See the Macrobenchmark Guide for more information on macrobenchmarks.

Summary

Public constructors

Public methods

@NonNull Statement
apply(@NonNull Statement base, @NonNull Description description)
final void
measureRepeated(
    @NonNull String packageName,
    @NonNull List<@NonNull Metric> metrics,
    @NonNull CompilationMode compilationMode,
    StartupMode startupMode,
    @IntRange(from = 1) int iterations,
    @ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> setupBlock,
    @ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> measureBlock
)

Measure behavior of the specified packageName given a set of metrics.

final void
@ExperimentalBenchmarkConfigApi
measureRepeated(
    @NonNull String packageName,
    @NonNull List<@NonNull Metric> metrics,
    @IntRange(from = 1) int iterations,
    @NonNull ExperimentalConfig experimentalConfig,
    @NonNull CompilationMode compilationMode,
    StartupMode startupMode,
    @ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> setupBlock,
    @ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> measureBlock
)

Measure behavior of the specified packageName given a set of metrics, with a custom PerfettoConfig.

final void
@ExperimentalBenchmarkConfigApi
measureRepeated(
    @NonNull String packageName,
    @NonNull List<@NonNull Metric> metrics,
    @IntRange(from = 1) int iterations,
    @NonNull PerfettoConfig perfettoConfig,
    @NonNull CompilationMode compilationMode,
    StartupMode startupMode,
    @ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> setupBlock,
    @ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> measureBlock
)

This method is deprecated. Deprecated in favour of a variant that accepts experimental config

Public constructors

MacrobenchmarkRule

Added in 1.1.0
public MacrobenchmarkRule()

Public methods

apply

Added in 1.1.0
public @NonNull Statement apply(@NonNull Statement base, @NonNull Description description)

measureRepeated

public final void measureRepeated(
    @NonNull String packageName,
    @NonNull List<@NonNull Metric> metrics,
    @NonNull CompilationMode compilationMode,
    StartupMode startupMode,
    @IntRange(from = 1) int iterations,
    @ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> setupBlock,
    @ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> measureBlock
)

Measure behavior of the specified packageName given a set of metrics.

This performs a macrobenchmark with the below control flow:

    resetAppCompilation()
compile(compilationMode)
repeat(iterations) {
setupBlock()
captureTraceAndMetrics {
measureBlock()
}
}
Parameters
@NonNull String packageName

ApplicationId / Application manifest package name of the app for which profiles are generated.

@NonNull List<@NonNull Metric> metrics

List of metrics to measure.

@NonNull CompilationMode compilationMode

Mode of compilation used before capturing measurement, such as CompilationMode.Partial, defaults to CompilationMode.DEFAULT.

StartupMode startupMode

Optional mode to force app launches performed with MacrobenchmarkScope.startActivityAndWait (and similar variants) to be of the assigned type. For example, COLD launches kill the process before the measureBlock, to ensure startups will go through full process creation. Generally, leave as null for non-startup benchmarks.

@IntRange(from = 1) int iterations

Number of times the measureBlock will be run during measurement. Note that total iteration count may not match, due to warmup iterations needed for the compilationMode.

@ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> setupBlock

The block performing app actions each iteration, prior to the measureBlock. For example, navigating to a UI where scrolling will be measured.

@ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> measureBlock

The block performing app actions to benchmark each iteration.

measureRepeated

@ExperimentalBenchmarkConfigApi
public final void measureRepeated(
    @NonNull String packageName,
    @NonNull List<@NonNull Metric> metrics,
    @IntRange(from = 1) int iterations,
    @NonNull ExperimentalConfig experimentalConfig,
    @NonNull CompilationMode compilationMode,
    StartupMode startupMode,
    @ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> setupBlock,
    @ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> measureBlock
)

Measure behavior of the specified packageName given a set of metrics, with a custom PerfettoConfig.

This performs a macrobenchmark with the below control flow:

    resetAppCompilation()
compile(compilationMode)
repeat(iterations) {
setupBlock()
captureTraceAndMetrics {
measureBlock()
}
}

Note that a custom PerfettoConfigs may result in built-in Metrics not working.

You can see the PerfettoConfig used by a trace (as a text proto) by opening the trace in ui.perfetto.dev, and selecting Info and Stats view on the left panel. You can also generate a custom text proto config by selecting Record new trace on the same panel, selecting recording options, and then clicking Recording command to access the generated text proto.

Parameters
@NonNull String packageName

ApplicationId / Application manifest package name of the app for which profiles are generated.

@NonNull List<@NonNull Metric> metrics

List of metrics to measure.

@IntRange(from = 1) int iterations

Number of times the measureBlock will be run during measurement. Note that total iteration count may not match, due to warmup iterations needed for the compilationMode.

@NonNull ExperimentalConfig experimentalConfig

Configuration for experimental features.

@NonNull CompilationMode compilationMode

Mode of compilation used before capturing measurement, such as CompilationMode.Partial, defaults to CompilationMode.DEFAULT.

StartupMode startupMode

Optional mode to force app launches performed with MacrobenchmarkScope.startActivityAndWait (and similar variants) to be of the assigned type. For example, COLD launches kill the process before the measureBlock, to ensure startups will go through full process creation. Generally, leave as null for non-startup benchmarks.

@ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> setupBlock

The block performing app actions each iteration, prior to the measureBlock. For example, navigating to a UI where scrolling will be measured.

@ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> measureBlock

The block performing app actions to benchmark each iteration.

measureRepeated

@ExperimentalBenchmarkConfigApi
public final void measureRepeated(
    @NonNull String packageName,
    @NonNull List<@NonNull Metric> metrics,
    @IntRange(from = 1) int iterations,
    @NonNull PerfettoConfig perfettoConfig,
    @NonNull CompilationMode compilationMode,
    StartupMode startupMode,
    @ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> setupBlock,
    @ExtensionFunctionType @NonNull Function1<@NonNull MacrobenchmarkScopeUnit> measureBlock
)
Parameters
@NonNull PerfettoConfig perfettoConfig

Configuration for Perfetto trace capture during each iteration. Note that insufficient or invalid configs may result in built-in Metrics not working.