Chaos Engineering for Mobile Apps

Backend chaos engineering taught us to kill pods randomly and verify that circuit breakers trip. That mental model fails on mobile within the first five minutes of testing. When Kubernetes terminates

Mobile Chaos Isn't Just Latency Injection with a Fancy Name

Backend chaos engineering taught us to kill pods randomly and verify that circuit breakers trip. That mental model fails on mobile within the first five minutes of testing. When Kubernetes terminates a container, the request either retries or fails fast. When Android 14's ActivityManager kills your process to reclaim 180MB for a camera operation, you don't get a graceful shutdown. You get a onTrimMemory() callback you ignored, a savedInstanceState bundle that may or may not persist, and a user returning to a checkout screen that believes the cart is empty because your in-memory repository evaporated.

Mobile chaos is stateful, hardware-constrained, and governed by power managers that treat your app as a battery vampire first and a business critical workflow second. The surface area isn't just network latency—it's thermal throttling on an iPhone 15 Pro reducing your animation frame rate to 30fps, Doze mode deferring your WorkManager job for six hours, and a Bluetooth permission dialog appearing mid-gesture because the OS decided now was the time to ask. If your resilience strategy starts and ends with Charles Proxy throttling, you're testing the exception handling of a networking library, not the survival of your user experience.

The Stateful Nature of Mobile Failure

Serverless functions don't have memory leaks that accumulate over seventeen sessions. Mobile apps do. The lifecycle complexity starts with the fundamental misunderstanding of Activity recreation in Android. When you rotate a device from portrait to landscape on Android 14, isChangingConfigurations() returns true, onSaveInstanceState() fires, but isFinishing() remains false. When the system kills your process under memory pressure and the user returns via the recents menu, isChangingConfigurations() is false, onSaveInstanceState() fired (maybe), but your ViewModel is gone unless you explicitly used SavedStateHandle.

This distinction destroys naive chaos implementations. Teams often test "backgrounding" by pressing the home button, which triggers onPause() → onStop(), but keeps the process alive. The real failure mode is onTrimMemory(ComponentCallbacks2.TRIM_MEMORY_COMPLETE) followed by SIGKILL without onDestroy(). Your local cache, your RxJava disposables, your un-flushed analytics queue—all gone.

iOS introduces parallel complexity through UISceneDelegate. In iOS 17, backgrounding an app moves it to the suspended state, but the system may purge it immediately if the device is in Low Power Mode and thermal state is .critical. The sceneDidEnterBackground(_:) delegate fires, but applicationWillTerminate(_:) never does. If you persisted state in applicationWillTerminate assuming you had time, you've already lost data. The state restoration APIs (NSUserActivity, UIStateRestoring) only work if you implemented them before the chaos started.

The Four Horsemen of Mobile Instability

Effective mobile chaos engineering requires systematic injection of four specific failure domains: network degradation, memory pressure, power constraint, and OS interruption. Each requires different tooling and produces distinct failure signatures.

Network: Beyond "Slow Mode"

Network chaos on mobile isn't just latency. It's the transition from WiFi to cellular mid-upload, the captive portal that returns 200 OK with an HTML login page, and the VPN that drops silently leaving sockets half-open. Android 14 introduced explicit ConnectivityManager requirements for foreground services; if your chaos test doesn't verify that NetworkCallback triggers during a file upload when the user toggles airplane mode, you're missing production crashes.

Tools matter here. Charles Proxy 4.6 and Proxyman 4.15 allow throttling and blacklisting, but they require proxy configuration that alters SSL trust chains. For instrumentation-free chaos, Facebook's Augmented Traffic Control (ATC) on a Raspberry Pi 4 creates realistic RF conditions. On device, the iOS Network Link Conditioner (part of Additional Tools for Xcode 15) provides preset profiles—3G at 330ms latency, Edge at 2,400ms—but requires a developer image mounted. For automated CI, com.apple.network.connection simulation via simctl remains inconsistent; most teams fallback to pfctl rules on macOS runners to throttle the simulator's network interface directly.

Memory: The Silent Killer

Android's Low Memory Killer Daemon (lmkd) uses pressure stall information (PSI) thresholds to select victim processes. On a Pixel 8 with 8GB RAM, your app can consume 450MB before becoming a target, but that threshold drops to 280MB when the camera app launches. iOS uses a more opaque jetsam priority system, but the result is identical: termination without warning.

Testing this requires explicit pressure injection. Android provides adb shell am send-trim-memory , supporting levels from RUNNING_MODERATE to RUNNING_CRITICAL. A robust chaos experiment sends TRIM_MEMORY_RUNNING_CRITICAL while the user is mid-transaction, then verifies that ViewModel state survives via SavedStateHandle or that the local database—not memory—holds the source of truth.

iOS lacks direct memory pressure APIs, but you can simulate the effect. Allocating 80% of available system memory via vm_allocate in a test helper app forces jetsam to target your target app when it backgrounds. Alternatively, running memory-intensive ML models (CoreML with computeUnits: .all) raises memory pressure organically.

Power: Doze, Standby, and Thermal Throttling

Android 12+ Doze mode imposes restrictions that break naive polling architectures. After 30 minutes of inactivity (or immediately if the device is stationary and unplugged), the system defers AlarmManager setExact operations, batching network access into maintenance windows. If your chaos testing doesn't include adb shell dumpsys deviceidle force-idle followed by verification that WorkManager with setExpedited(OutOfQuotaPolicy.RUN_AS_NON_EXPEDITED_WORK_REQUEST) still completes, you're not testing production behavior.

Thermal throttling is harder to simulate. Android 10+ exposes PowerManager.getCurrentThermalStatus(), returning states from THERMAL_STATUS_NONE to THERMAL_STATUS_SHUTDOWN. While you can't easily raise hardware temperature in CI, you can mock these states via reflection in debug builds or use thermal test chambers for hardware-in-the-loop testing. iOS 17 provides NSProcessInfo.thermalState, and notably, the iPhone 15 Pro will throttle CPU performance and reduce display brightness when in .critical state. If your app relies on Metal compute shaders for image processing, thermal chaos reveals frame drops that unit tests miss.

Interruption: System Dialogs and Lifecycle Edge Cases

The most under-tested chaos vector is OS-level interruption. On Android 14, the predictive back gesture (enabled via android:enableOnBackInvokedCallback="true") can trigger onBackPressed() during a permission request dialog. If your Activity finishes while a coroutine is suspended waiting for ActivityResultLauncher output, you get an IllegalStateException. iOS 17's Live Activities and StandBy mode introduce new interruption surfaces—incoming calls now present as banner notifications that don't pause the app unless the user answers, but Siri activation does pause audio sessions.

Testing requires UI automation that interacts with system UI. Espresso 3.5.1 fails here—it can't click system permission dialogs. UIAutomator 2.2.0 can, via UiDevice.getInstance().findObject(new UiSelector().text("Allow")). On iOS, XCTest's addUIInterruptionMonitor catches permission dialogs, but requires the app to be foregrounded during the interruption. For deeper chaos, private APIs (for internal testing only) can trigger SBSpringBoard notifications that simulate low battery alerts.

The Tooling Reality Check

There's no equivalent of Gremlin or Chaos Mesh for mobile—platform sandboxing prevents external processes from killing apps or injecting network faults. The landscape is fragmented between OS-level utilities, custom test frameworks, and autonomous validation platforms.

SUSA complements manual chaos injection. After you adb shell am kill your app during a checkout flow, uploading the APK to SUSA initiates a 10-persona autonomous exploration that verifies the cart persists, the "Pay Now" button isn't dead, and accessibility announcements still fire for screen reader users. This catches the secondary failures—like NullPointerException in onResume() because the chaos experiment cleared a reference that the manual test assumed was stable.

Phase 0: Instrumentation That Survives the Blast

Before injecting chaos, you need telemetry that distinguishes between "app crashed" and "app was terminated by OS." Android's ApplicationExitInfo (API 30+) provides REASON_LOW_MEMORY and REASON_SIGNALED (SIGKILL). Log these to Firebase Crashlytics with custom keys:

val am = getSystemService(Context.ACTIVITY_SERVICE) as ActivityManager
am.getHistoricalProcessExitReasons(null, 0, 5).forEach { exitInfo ->
    if (exitInfo.reason == ApplicationExitInfo.REASON_LOW_MEMORY) {
        FirebaseCrashlytics.getInstance().setCustomKey("last_chaos_memory_kb", exitInfo.rss)
    }
}

iOS requires MetricKit (iOS 14+). Implement MXMetricManagerSubscriber to receive didReceive(_:) callbacks containing applicationExitMetrics. Check foregroundExitData for cumulativeAbnormalExitCount spikes after chaos experiments.

Establish baseline SLIs: crash-free session rate (target >99.9%), ANR rate (Android: <0.1% of sessions), and time-to-interactive (TTI) after process restart. Without these, you can't prove that chaos engineering improves resilience rather than just breaking things.

Phase 1: Controlled Network Degradation

Start with network chaos—it's the easiest to control and the most common production failure. Don't use the emulator's built-in throttling; it doesn't support HTTP/3 or QUIC degradation realistically.

For Android, create a custom Interceptor for OkHttp 4.12.0 that simulates specific failure modes:

class ChaosInterceptor : Interceptor {
    override fun intercept(chain: Interceptor.Chain): Response {
        if (Random.nextFloat() < 0.1f) {
            // Simulate captive portal: 200 OK with HTML body
            return Response.Builder()
                .request(chain.request())
                .protocol(Protocol.HTTP_2)
                .code(200)
                .message("OK")
                .body("<html>Login Required</html>".toResponseBody("text/html".toMediaType()))
                .build()
        }
        if (Random.nextFloat() < 0.05f) {
            throw IOException("Airplane mode simulation")
        }
        return chain.proceed(chain.request())
    }
}

For iOS, use URLProtocol subclassing to intercept URLSession requests. Register it in your test target only:

class ChaosURLProtocol: URLProtocol {
    override class func canInit(with request: URLRequest) -> Bool {
        return true
    }
    
    override func startLoading() {
        if #available(iOS 17.0, *), Int.random(in: 0...100) < 10 {
            self.client?.urlProtocol(self, didFailWithError: URLError(.notConnectedToInternet))
            return
        }
        // Pass through to real client
    }
}

Validate retry logic. If your app uses Retrofit with RetryAndFollowUpInterceptor, verify that exponential backoff doesn't amplify the thundering herd when 1,000 devices simultaneously reconnect after a simulated network partition. Check that WorkManager constraints (.setRequiredNetworkType(NetworkType.CONNECTED)) actually defer work, rather than throwing immediate failures.

Phase 2: Process Death and State Loss

This is where most mobile apps fail chaos testing. The scenario: user adds items to cart, backgrounds app, system kills process, user returns via recents.

Android testing requires adb shell am kill while the app is backgrounded, not am force-stop (which clears the task stack). Then launch via recents and verify state restoration:

# Background the app
adb shell input keyevent KEYCODE_HOME
# Wait for process to be eligible for killing
sleep 5
# Kill it
adb shell am kill com.example.app
# Launch from recents (task ID varies)
adb shell am start -a android.intent.action.MAIN -c android.intent.category.LAUNCHER -f 0x10200000 com.example.app

Your app must restore state without relying on onDestroy() having fired. Verify that SavedStateHandle in your ViewModel contains the cart items, or that Room database is the single source of truth. If you use Jetpack Navigation 2.7.5, check that the back stack survived—NavController saves state via onSaveInstanceState, but custom Navigator implementations may not.

iOS testing uses XCTest:

func testProcessDeath() {
    let app = XCUIApplication()
    app.launch()
    
    // Perform action that creates state
    app.buttons["Add to Cart"].tap()
    
    // Background and terminate
    XCUIDevice.shared.press(.home)
    app.terminate()
    
    // Relaunch
    app.launch()
    
    // Verify state
    XCTAssertTrue(app.staticTexts["1 item in cart"].exists)
}

Crucially, test with UIApplicationExitsOnSuspend set to false (the default) and true (legacy behavior). If your app uses SceneDelegate, verify that stateRestorationActivity was set in sceneWillResignActive(_:).

Phase 3: Resource Exhaustion and Thermal Pressure

Once process death is handled, introduce resource pressure. Android's adb shell am send-trim-memory provides granular control:

# Simulate moderate pressure (app visible but background app needs memory)
adb shell am send-trim-memory com.example.app RUNNING_MODERATE

# Critical pressure - app should release non-essential caches
adb shell am send-trim-memory com.example.app RUNNING_CRITICAL

Your app should respond by clearing in-memory image caches (Glide 4.16.0, Coil 2.5.0) and unregistering location listeners. Verify via profiling that heap size drops after the callback.

For thermal testing on Android, mock the thermal status if hardware chambers aren't available:

// Debug build only - requires hidden API access
val powerManager = getSystemService(Context.POWER_SERVICE) as PowerManager
val method = powerManager.javaClass.getDeclaredMethod("setThermalStatus", Int::class.java)
method.isAccessible = true
method.invoke(powerManager, 4) // THERMAL_STATUS_SEVERE

iOS thermal testing requires physical devices. Use ProcessInfo.processInfo.thermalState and observe behavior when running GPU-intensive Metal compute shaders. The iPhone 15 Pro will throttle CPU clocks to 50% in .critical state; verify that your ML model inference still completes within 16ms frame budget or degrades gracefully with lower quality settings.

Battery chaos is simpler. On iOS Simulator:

xcrun simctl status <device> battery level 5 state discharging

On Android Emulator, use extended controls to set battery to 5% and verify that PowerManager.isPowerSaveMode() triggers your degraded animation path (reducing frame rate to 30fps or disabling background sync).

CI Integration: Automating the Chaos

Manual chaos testing finds bugs once. Automated chaos prevents regression. Integrate into GitHub Actions or similar using matrix builds across OS versions.

Example workflow for Android:

name: Chaos Engineering
on: [push]
jobs:
  chaos:
    runs-on: macos-14
    strategy:
      matrix:
        api-level: [30, 34]
        profile: [network-loss, memory-pressure, process-death]
    steps:
      - uses: actions/checkout@v4
      
      - name: AVD Setup
        uses: reactivecircus/android-emulator-runner@v2
        with:
          api-level: ${{ matrix.api-level }}
          script: |
            ./gradlew installDebug
            # Run chaos script
            python3 chaos/${{ matrix.profile }}.py
            # Validate with autonomous QA
            susa-cli upload-apk app/build/outputs/apk/debug/app-debug.apk \
              --test-type chaos-validation \
              --junit-output chaos-results.xml

The network-loss.py script would use adb shell cmd connectivity airplane-mode enable during a UI Automator test, then verify recovery.

For iOS, use xcodebuild test with a dedicated chaos scheme that includes the ChaosURLProtocol and memory pressure injection via malloc stress tests in the test runner.

SUSA integration fits here as a validation layer. After your chaos script kills the process or throttles the network, SUSA's autonomous personas explore the app for 15 minutes, generating Appium scripts for any crashes found. This catches the "dead button" scenario where the UI renders but the presenter was cleared during the chaos event, leaving buttons that log clicks but trigger no actions.

Measuring Mobile Resilience: SLIs Beyond Uptime

Backend chaos uses availability (99.99%) and latency (p99 < 200ms). Mobile requires different SLIs:

Track these per chaos experiment. If process death chaos increases TTI from 800ms to 4s because you're reloading data from network instead of cache, you've found a resilience gap.

The Adoption Playbook: From Zero to Chaos

Don't attempt full infrastructure chaos on day one. Mobile chaos adoption follows a strict progression to avoid alert fatigue and developer burnout.

Week 1-2: Baseline Instrumentation

• Implement ApplicationExitInfo logging (Android) and MetricKit (iOS)
• Add chaos experiment IDs to all crash logs
• Establish current SLIs without any chaos injection

Week 3-4: Manual Process Death

• Daily adb shell am kill during lunch breaks
• Document 3 critical user flows (checkout, upload, login)
• Fix state loss bugs found during manual testing

Month 2: Automated Network Chaos

• Integrate OkHttp ChaosInterceptor in debug builds
• CI job runs 10% of UI tests with network faults
• Validate retry logic and offline queues

Month 3: Resource Pressure

• Add send-trim-memory calls to UI Automator tests
• Verify image cache eviction and database consistency
• Test thermal throttling on physical device farms (AWS Device Farm, Firebase Test Lab)

Quarter 2: Full CI Integration

• Matrix testing across API 30-34 and iOS 16-17
• SUSA autonomous validation post-chaos for regression detection
• OWASP Mobile Top 10 validation (M2: Insecure Data Storage) during process death chaos to ensure sensitive data isn't written to savedInstanceState

Start with Process Death Tomorrow

Don't budget for chaos engineering tools next quarter. Don't redesign your architecture for eventual consistency yet. Tomorrow morning, take a physical test device running Android 14 or iOS 17, navigate to the most critical screen in your app—payment confirmation, medical data entry, content upload—and press the home button. Wait ten seconds. Run adb shell am kill com.yourapp.package or swipe the app away on iOS. Reopen it from the recents menu.

If the screen is blank, if the form is empty, if the "Confirm" button does nothing because the presenter died, you've just found your highest impact resilience bug without spending a dollar. Fix that first. Build the telemetry to detect when it happens in production. Then, and only then, start automating the network latency and thermal throttling.

Autonomous platforms like SUSA can discover the edge cases you forgot to manually test—the dead buttons that render but don't respond, the accessibility announcements that stop after process restart, the security violations when state restoration leaks tokens. But they can't replace the discipline of manually killing your app during the moments that matter. Mobile chaos engineering starts with accepting that your process is disposable. Everything else is optimization.