How to record & replay http traffic in Android and iOS apps
Starting 2022 I joined a small but ambitious startup (mobile.dev), with the goal to create much needed infrastructure for mobile development.
We analyse apps and run performance benchmarks to detect issues pre-production and in some cases we require instrumentation tests with network mocking.
My first task? Figure out if we can mock the network ourselves in order to make it easier to use our product. Creating and maintaining instrumentation tests with mocking is never easy — so we wanted to handle that responsibility on our side.
Today i’ll share a small part of my journey. In particular, how to:
- Create a realistic mock server, just by browsing an app.
- Make the app communicate with the mock server.
- Edit api responses to whatever we like
This can be useful to create ideal testing environments , with little to no manual work.
Tools for the job
SDK ❌
There were a few considerations along the way. Notably, we thought of creating an SDK that would capture the app’s traffic, store it locally and later serve it back — all done via an http interceptor.
Pros
- Everything happens inside the app. Responses will be super fast.
- No additional infrastructure required.
- Easy to scale.
Cons
- Multiple SDKs to maintain, per platform and framework.
- Each app would require SDK integration.
- Harder to debug and fix issues since a lot of the work is done client side.
This approach, while doable, would require a lot of resources to create and maintain. So, we quickly abandoned the idea for a more agile solution.
Proxy ✅
Proxies are the best way to monitor and modify traffic and our search led to MITMProxy. An open source tool to intercept and modify http traffic.
Big shout out to the Mitmproxy maintainers, they have built a solid tool over the years. If you’re interested more on how it works, you can start here.
It’s open source, platform independent, can record http traffic into a file and can be used as a replay server. Jackpot!
Setup
Proxy
Installation is pretty easy on macOS and there are instructions for every platform.
brew install mitmproxy
Mitmproxy comes with 3 apps: mitmproxy, mitmweb and mitmdump. We’ll be using mitmweb for demonstration purposes.
Now let’s start mitmproxy using a terminal:
mitmweb
This will:
- Launch a proxy server
127.0.0.1:8080
- Launch a web interface in your browser
127.0.0.1:8081
Android
For Android i’ll use standard tools that come with Android Studio:
- Android Virtual Device (AVD)
- Android 10.0 (Google APIs)
Once the AVD is setup, you’ll have to install the CA certificate that will allow us to intercept https traffic. Thankfully, there’s already a detailed guide 😊
Lastly, set the device proxy. Using the AVD settings panel this is easy:
iOS
For iOS you have 2 options.
- You can use the iOS simulator that comes with Xcode
- You can use your iPhone
The process of certificate installation, proxy setup and accessing an app was a bit easier on an iPhone — so I went that route.
Note: The Appstore does not load when the proxy is enabled, so make sure to first download the app (see below).
The app
Last but not least, i’ll use an app we’re all familiar with, Wikipedia!
- Android (open source) (Playstore)
- iOS (open source) (Appstore)
For Android you can just grab the apk and install it in the AVD.
For iOS, if you’re using a simulator you’ll have to build from source code. If you’re using an iphone, just download the app from the Appstore.
Now the fun part begins. 🎉
Record
During setup, we launched our proxy and connected our device to it. Now, all we have to do is launch the app.
If everything was setup correctly, we should be seeing all the requests and responses the app is making.
We can also edit responses. Just select which call you would like to edit and press the pencil icon.
I’m going to edit the displayed title for a wikipedia article, so we can tell that replay is working: Singer Building becomes Singer Building (Hello World)
Lastly, i’m going to save the replay as app.replay
Replay
Now that we have our replay file we can stop the running instance of mitmweb ctl+C
and start in replay mode.
mitmweb --server-replay app_edit.replay
This will launch mitmweb in replay mode using app_edit.replay
The way it works is, every time the app makes an http request — mitmproxy will try to match the request with one stored in the replay file.
If there’s a match, the proxy will respond with the response on file.
And that’s it, we did it! We were able to record the app’s traffic, edit it and play it back.
Final thoughts
This was just an introduction of course and there are much deeper topics to explore, for example:
- Full offline replays
- Custom replay logic
- Handling authentication, expiring sessions, timestamps etc..
- Full automation of the process, to create realistic mock servers on the fly.
Let me know in the comments if you’d like me to continue this series.