nicolaspln/android-performance-profiler

An attempt to have Lighthouse for production Android apps. Measure Android performance even in production on CLI or Flipper plugin

Measure the performance of any Android app 🚀

• 🙅 No installation required, supports even production app
• ✨ Generates beautiful web report (like this Flatlist/Flashlist comparison)
• 💻 Via E2E test, Flipper plugin or CLI

• Getting started with the automated profiler
  • Requirements
  • Main usage
  • Example using Maestro
  • Customizing web report
  • Comparing measures
  • Advanced usage using TypeScript
    • Examples folder
  • Running in CI
    • AWS Device Farm
    • Reusing APK
    • Advanced usage
• Flipper Plugin
  • Install
  • Usage
• CLI
  • Via Custom script
• Contributing
  • web-reporter

Getting started with the automated profiler

Requirements

• Node
• An Android phone plugged in 🔌 (or an emulator started)

Main usage

1. If you have an e2e test script, you can run:

npx @perf-profiler/e2e measure --bundleId <your app bundle id> \
  --testCommand <your e2e test command> \
  --duration 10000 \
  --resultsFilePath results.json

• This will run your e2e test 10 times (by default), measure performance during 10s for each iteration and write measures to results.json
• Use npx @perf-profiler/profiler getCurrentApp to display the bundle id of the app opened on your phone
• ⚠️ Your e2e test command should start the app

2. You can then open the web report for those measures:

npx @perf-profiler/web-reporter results.json

Example using Maestro

For instance, if you're using Maestro, you can measure the startup performance of the Twitter app:

1. Create a twitter.yaml file:

appId: com.twitter.android
---
- launchApp
- tapOn: Search and Explore

2. Measure performance 🚀

npx @perf-profiler/e2e measure --bundleId com.twitter.android \
  --testCommand "maestro test twitter.yaml" \
  --duration 10000 \
  --resultsFilePath results.json

3. Open the report

npx @perf-profiler/web-reporter results.json

Customizing web report

You can change the title displayed in the web report by passing --resultsTitle:

npx @perf-profiler/e2e measure --bundleId com.twitter.android \
 --testCommand "maestro test twitter.yaml` \
 --resultsTitle "Twitter - App start"

Comparing measures

If you have several JSON files of measures, you can open the comparison view with:

npx @perf-profiler/web-reporter results1.json results2.json results3.json

Setting number of iterations to be run

By default, 10 iterations of your test will be run, and measures will be averaged. You can change this by passing --iterationCount:

npx @perf-profiler/e2e measure --bundleId com.twitter.android \
 --testCommand "maestro test twitter.yaml` \
 --iterationCount 5

Advanced usage using TypeScript

You can also run measures and exploit results programmatically via TypeScript. See here

Examples folder

Check out the examples folder for more advanced usage:

• Running programmatically with Jest

Running in CI

To run in CI, you'll need the CI to be connected to an Android device. An emulator running on the CI will likely be too slow, so it's best to be connected to a device farm cloud. The profiler needs full adb access, so only few device cloud are compatible:

Our choice is AWS Device Farm but some other options should work as well (though they haven't been tested):

• Saucelabs with Entreprise plan and Virtual USB
• Genymotion Cloud (using emulators will not accurately reproduce the performance of a real device)

AWS Device Farm

We've added a neat tool to seamlessly run your tests on AWS Device Farm and get the measures back:

export AWS_ACCESS_KEY_ID="ADD YOUR AWS KEY ID HERE" AWS_SECRET_ACCESS_KEY="ADD YOUR AWS SECRET HERE"

# If you have a simple TS script to run
npx @perf-profiler/aws-device-farm runTest --apkPath app-release.apk --testFile script.ts

Reusing APK

You might also want to upload your APK only once and reuse it:

# This will log the "ARN" associated with your APK
npx @perf-profiler/aws-device-farm uploadApk --apkPath app-release.apk
# ...

export APK_UPLOAD_ARN="arn:aws:devicefarm:..."
npx @perf-profiler/aws-device-farm runTest --testFile script.ts

Advanced usage

For more complex cases, run from your root folder, containing node_modules:

npx @perf-profiler/aws-device-farm runTest \
 --apkPath app-release.apk \
 --deviceName "A10s" \
 --testFolder . \
 --testCommand "yarn jest appium"

Flipper Plugin

flipper-film-full.mp4

Install

Simply search for android-performance-profiler in the Flipper marketplace. No further installation required! 🥳

Usage

• Start your app
• Click "AUTO-DETECT"
• Then start measuring! 🚀

CLI

You can profile directly in CLI with:

npx @perf-profiler/profiler profile --fps --ram --threadNames "(mqt_js)" "UI Thread"

You can also use a custom script:

Via Custom script

For instance:

import {
  detectCurrentAppBundleId,
  getAverageCpuUsage,
  getPidId,
  Measure,
  pollPerformanceMeasures,
} from "@perf-profiler/profiler";

const { bundleId } = detectCurrentAppBundleId();
const pid = getPidId(bundleId);

const measures: Measure[] = [];

const polling = pollPerformanceMeasures(pid, (measure) => {
  measures.push(measure);
  console.log(`JS Thread CPU Usage: ${measure.perName["(mqt_js)"]}%`);
});

setTimeout(() => {
  polling.stop();
  const averageCpuUsage = getAverageCpuUsage(measures);
  console.log(`Average CPU Usage: ${averageCpuUsage}%`);
}, 10000);

Contributing

Start by building the whole project:

At the root of the repo:

yarn
yarn tsc --build --w

Keep this open in one terminal.

web-reporter

and run in another terminal:

yarn workspace @perf-profiler/web-reporter start

Then in packages/web-reporter/src/App.tsx, uncomment the lines to add your own measures:

// Uncomment with when locally testing
// eslint-disable-next-line @typescript-eslint/no-var-requires
testCaseResults = [require("../measures.json")];

You should now be able to open the local server

Run yarn jest Plugin -u after modifications.

Flipper plugin

• Add the path to the packages folder in ~/.flipper/config.json.

For instance, my config.json is currently {"pluginPaths":["/Users/almouro/dev/projects/android-performance-profiler/packages"],"disabledPlugins":[],"darkMode":"system","updaterEnabled":true,"launcherEnabled":true,"lastWindowPosition":{"x":-195,"y":-1415,"width":1280,"height":1415}}

• in the packages/flipper-plugin-android-performance-profiler, run yarn watch.

You should now see your local plugin in Flipper (ensure you have uninstalled the one from the marketplace), in the disabled plugin section if you're installing for the first time.

⚠️ when modifying files outside of the packages/flipper-plugin-android-performance-profiler, live reload sometimes doesn't work and you need to re-run yarn watch for changes to take effect 😕