We run some of the most performance critical services in the cloud. These services offer 100s of Gbps bandwidth running via our software components. In the past, I have also built core networking virtualization that runs on every Azure machine. For building these network intensive services and components, it is critical to have tools for network measurements that can help optimize performance, isolate issues and prevent regressions. We use variety of tools to do various types of measurements and tune our software components.
In this post, I talk about an open source tool called Ethr, built as a side project, to provide many such capabilities in a single, easy to use binary. The tool is recently released as v1.0 and supports various types of network measurements such as throughput, packets/s, connections/s, network latency, ICMP & TCP ping, traceroute and MyTraceRoute. This makes it convenient to do various types of measurements without having to download and maintain a multitude of tools.
Overview
Ethr is a cross platform network performance measurement tool. It provides comprehensive network measurements ranging from bandwidth, connections/s, packets/s to latency, ping and TraceRoute.
It supports 3 different modes:
- Server – Runs as a server and communicates with Ethr clients
- Client – Runs as a client and communicates with Ethr servers
- External – Runs as a client and communicates with non-Ethr servers
These 3 modes enable Ethr to support variety of network measurements in a single binary.
Features
The overall set of features supported by Ethr is shown below:
| Protocol | Bandwidth | Connections/s | Packets/s | Latency | Ping | TraceRoute | MyTraceRoute |
|---|---|---|---|---|---|---|---|
| TCP | Yes | Yes | NA | Yes | Yes | Yes | Yes |
| UDP | Yes | NA | Yes | No | NA | No | No |
| ICMP | No | NA | NA | NA | Yes | Yes | Yes |
Usage
The download and usage instructions for Ethr are available at: https://github.com/microsoft/ethr/blob/master/README.md
Few examples are listed below:
// Start server
ethr -s
// Start client for default (bandwidth) test measurement using 1 thread
ethr -c localhost
// Start bandwidth test using 8 threads
ethr -c localhost -n 8
// Start connections/s test using 64 threads to server 10.1.0.11
ethr -c 10.1.0.11 -t c -n 64
// Run Ethr server on port 9999
./ethr -s -port 9999
// Measure TCP connection setup latency to ethr server on port 9999
// Assuming Ethr server is running on server with IP address: 10.1.1.100
./ethr -c 10.1.1.100 -p tcp -t pi -d 0 -4 -port 9999
// Measure TCP connection setup latency to www.github.com at port 443
./ethr -x www.github.com:443 -p tcp -t pi -d 0 -4
// Measure TCP connection setup latency to www.github.com at port 443
// Note: Here port 443 is driven automatically from https
./ethr -x https://www.github.com -p tcp -t pi -d 0 -4
// Measure ICMP ping latency to www.github.com
sudo ./ethr -x www.github.com -p icmp -t pi -d 0 -4
// Run measurement similar to mtr on Linux
sudo ./ethr -x www.github.com -p icmp -t mtr -d 0 -4
// Measure packets/s over UDP by sending small 1-byte packets
./ethr -c 172.28.192.1 -p udp -t p -d 0Comparison with iPerf
Types of measurements
iPerf supports bandwidth (or throughput) measurement only. Ethr supports many other measurements in addition to bandwidth measurements such as packets/s, connections/s, round trip latency, ping/connection latency, traceroute and mtr/my traceroute.
Multiple clients
One instance of iPerf server supports only one client to connect to it. Ethr supports a single server to allow connections from multiple clients, making it easier to do performance testing from multiple clients. In addition, as Ethr support many types of measurements, it is possible to mix and match all this to create a test environment that is comprehensive. For example, using Ethr it is possible to run TCP/UDP bandwidth test, while measuring TCP connection latency in parallel.
Protocol support
iPerf supports TCP, UDP and SCTP. Ethr supports TCP and UDP only.
Control channel
iPerf uses a control channel to exchange data between client and server. Ethr does not use a separate control channel and exchanges some information over the same connection that are used for test measurements.
There are pros and cons of each approach:
- The benefit for a separate control channel in iPerf are that it allows server to send results back to clients. This is especially useful in UDP where server bandwidth may be lower than client, if there is loss in the network. In Ethr, one has to look at both client results and server results to deduce the same.
- The benefit of not using a separate control channel in Ethr is support for entities behind load balancers. You can use Ethr to target a load balancer IP (typically referred as virtual IP or VIP), where Ethr can be running as server on multiple backend machines.
External mode
Ethr has an external mode that allows it to do some types of measurements with non-Ethr servers. For example, Ethr can allow you to measure TCP connection latency to any non-Ethr server or your web services etc. Ethr can also do TraceRoute and MyTraceRoute to non-Ethr destinations.
Screenshots
References
Ethr is available under MIT license at: https://github.com/microsoft/ethr