Startup Performance Analysis (Android)

General concepts

Emerge's performance analysis focuses on catching performance differences between app builds. To do so, Emerge controls many sources of noise and variance on the real, physical device we test on to ensure any performance difference between the two comparison builds stems from the app builds alone.

Cold startup testing

No developer setup is needed for Emerge's startup performance analysis.

Emerge supports startup performance analysis for AABs and APKs, but highly recommends using AABs as APKs can have more variable performance compared to AABs.

Startup test lifecycle

Measurements

Emerge records startup times by default from the start of your app's process to the end of the first frame rendering. This is called "time to initial display" in the official Android documentation and is Emerge's default startup time metric.

We can enable startup performance testing on any upload to Emerge, get in touch with our team to enable it for your uploads!

Marking custom startup end times

While the first frame (time to initial display) is the standard metric for default cold start time, Emerge can also record and catch regressions in time to full display, to a custom end time using reportFullyDrawn.

Place a call to this anywhere in your startup path once your application has finished fully loading. This will usually be called from your main activity, once all layout & rendering logic has been completed.

Let the Emerge team know if you'd like to use reportFullyDrawn as your primary measure.

Running startup comparisons

Contact the Emerge team through Slack or email us to enable startup comparisons for your organization.

If your team has enabled startup comparisons, you can run any comparison between two builds using the compare page.

Contact the Emerge team to enable startup comparisons in CI. As part of running in CI or using the compare page, Emerge is also able to login to test accounts or setup the app as needed.

Custom startup scenarios

📘

2.X+ Required

Custom startup scenarios were introduced in version 2.0.0 of the Emerge perf testing SDK. See the emerge-android repository for the lastest version info.

Emerge supports custom startup scenarios for apps that require more complex startup scenarios, like launching from deep links or other entry points. For custom startup scenarios, use the @EmergeStartupTest annotation with a custom startup test method:

class MyCustomStartupPerformanceTest {

  @EmergeInit  
  fun login() {  
    // OPTIONAL - Perform one-time setup, like logging in  
  }

  @EmergeStartupTest  
  fun customStartupTest() {  
    // Ex: Launch app from deeplink or other entrypoint 
  }
}

Emerge will run the @EmergeInit method once after the app is installed, then run the @EmergeStartupTest method multiple times on an isolated device in order to detect any potential performance regression.

Note: For custom startup tests, any @EmergeSetup annotated methods will not be run as their purpose is not relevant for startup scenarios.