In this guide we’ll be going through the process of configuring an intercepting set-up using mitmproxy and a wireless network, to inspect, modify and monitor encrypted HTTPS traffic. This will allow for a simple way to analyse traffic on mobile handsets and IoT devices, with the only requirements is Wi-Fi support and the ability to install custom certificates.
mitmproxy is your swiss-army knife for debugging, testing, privacy measurements, and penetration testing. It can be used to intercept, inspect, modify and replay web traffic such as HTTP/1, HTTP/2, WebSockets, or any other SSL/TLS-protected protocols. read more
A simple illustration of mitmproxy running in transparent mode under Debian Buster:
The communication process:
iPhone <==> Wi-Fi TAP <==> mitmproxy <== (IP Forwarding) <==> Internet
There are several software packages required for this set-up. All of which can be downloaded through
apt-get on most Linux based systems. As mentioned previously I’ll be using Debian Stretch, but the guide can be applied to other systems.
List of required packages:
mitmpoxy– a HTTP traffic interception tool.
dnsmasq- a lightweight DNS forwarder and DHCP service.
hostapd– a utility used for creating the wireless access point.
tcpdump– an all-round great packet capture utility.
To install all of the above:
sudo apt update
sudo apt install mitmproxy hostapd dnsmasq tcpdump
And for hardware, the only requirement is having two network interface cards, one for providing Internet connectivity and the other for broadcasting the wireless access point. I’ll be using one I bought from Amazon a (TP-Link Archer T2U Nano AC600) which is cheap and supports 2.4GHz and 5GHz natively on Linux systems, and my MacBook Pro’s built-in wireless card.
The TP-Link Archer T2U Nano AC600 WiFi adapter requires additional effort while installing on Linux. The only driver that worked was
realtek-rtl88xxau-dkms version 18.104.22.168~20190617 available on here.
git clone https://gitlab.com/kalilinux/packages/realtek-rtl88xxau-dkms
To check if the driver installed properly, plug in your USB adapter and ensure pass-through is enabled within the virtualization software. In my case, VMware Fusion (Menu > Virtual Machine > USB & Bluetooth).
Run the following inside the virtual machine
There should be two network interface cards, one called
ens33 which will provide access to the Internet, and the other interface we just created called
wlxd037XXXXXXXX will be configured as our wireless access point to use with mitmproxy.
Ensure the newly created network interface has a static IP address and a gateway address. This will point to the Dnsmasq service in the step next. Use the following network configuration and replace network interfaces to suite your own. A reboot may be required.
Network configuration file:
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
iface lo inet loopback
# The primary network interface
iface ens33 inet dhcp
# The secondary network interface
iface wlxd037XXXXXXXX inet static
IP Forwarding and IPtables
Start by enabling IP forwarding and add masquerade rules with IPtables to route and redirect specific network traffic to mitmproxy. This will be persistent.
Edit the main system config:
/etc/sysctl.conf and uncomment this line:
Apply the following rules to IPTables (replace to suit your wireless interface):
sudo iptables -t nat -A PREROUTING -i wlxd037XXXXXXXX -p tcp --dport 80 -j REDIRECT --to-port 8080
sudo iptables -t nat -A PREROUTING -i wlxd037XXXXXXXX -p tcp --dport 443 -j REDIRECT --to-port 8080
To make devices have their own IP addresses assigned automatically, a DHCP server is required. Dnsmasq is a lightweight DNS forwarder and DHCP server suitable for small networks. This makes sense otherwise we would need to manually assign addresses.
Create and modify the configuration file:
/etc/dnsmasq.conf and replace it with:
# Wireless Interception Interface
# DHCP server and range for assigning IP addresses
# Broadcast gateway and DNS server information
And finally, create the wireless network with Hostapd. This wireless network will operate under the 2.4GHz frequency (channel 7) and will be able to forward traffic onto the primary interface.
Create a new configuration file:
To use the 5GHz band with TP-Link adapter the Hostapd configuration file needs to be changed and driver needs to be loaded with:
modprobe -r 88XXau && modprobe 88XXau rtw_vht_enable=2
Saving TLS master keys to a file while running mitmproxy. With this file Wireshark is able to decrypt TLS traffic. See the Wireshark wiki for more information. Modify your
.bashrc file and export the environment variable:
When you want to capture all network traffic from an interface and you do not want to intercept or modify the flow. You need to flush the previous IPTables rules and apply a new rule.
To flush all IPTable rules within the nat table:
sudo iptables -t nat -F
And add a new rule to masquerade traffic to the Internet interface:
sudo iptables -t nat -A POSTROUTING -o ens33 -j MASQUERADE
After that you can run a tcpdump packet capture:
sudo tcpdump -i <interface> -s 65535 -w <file>
First, start the DNSmasq server:
sudo systemctl start dnsmasq
Next, start the Hostapd service:
sudo systemctl start hostapd
Finally, start the mitmproxy for example in transparent mode:
mitmproxy --mode transparent --showhost
Here is an example of monitoring traffic using an iPhone device. First, connect to the wireless network created by Hostapd as you would to a regular Wi-Fi network.
Then, open a browser and navigate to
http://mitm.it (make sure mitmproxy is running). Download the certificate for your device type and hit allow.
Next, go to Settings > General > Profile > mitmproxy > Install Profile. When installed, go back to the main Settings again General > About > Certificate Trust Settings and enable full trust for root certificate.
There appears to be an issue with mitmproxy packages on Debian (Stretch and Buster). When visiting the
http://mitm.it/ to download certificates, the page does not display the links properly. Leaving the user to manually browse site on devices.
http://mitm.it/cert/pem– Android & iOS
Or find and edit the file
/usr/lib/python3/dist-packages/mitmproxy/addons/onboardingapp/templates/layouts.html and replace:
<link href="/static/bootstrap.min.css" rel="stylesheet">
<link href="/static/font-awesome.min.css" rel="stylesheet">
<link href="//stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" rel="stylesheet">
<link href="//stackpath.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet">
A list of useful resources: