docs: update benchmark and add about memory usage

This commit is contained in:
Yujia Qiao 2021-12-28 20:04:54 +08:00
parent 1180c7e538
commit 4a1da9be7f
No known key found for this signature in database
GPG Key ID: DC129173B148701B
10 changed files with 156 additions and 21 deletions

View File

@ -1,7 +1,7 @@
# rathole
![rathole-logo](./docs/img/rathole-logo.png)
A fast, secure and stable reverse proxy for NAT traversal, written in Rust
A secure, stable and high-performance reverse proxy for NAT traversal, written in Rust
rathole, like [frp](https://github.com/fatedier/frp) and [ngrok](https://github.com/inconshreveable/ngrok), can help to expose the service on the device behind the NAT to the Internet, via a server with a public IP.
@ -131,13 +131,14 @@ If `RUST_LOG` is not present, the default logging level is `info`.
## Benchmark
rathole has similiar latency to [frp](https://github.com/fatedier/frp), but can handle more connections. Also it can provide much better bandwidth than frp.
rathole has similiar latency to [frp](https://github.com/fatedier/frp), but can handle a more connections, provide larger bandwidth, with less memory usage.
See also [Benchmark](./docs/benchmark.md).
![http_throughput](./docs/img/http_throughput.svg)
![tcp_bitrate](./docs/img/tcp_bitrate.svg)
![tcp_latency](./docs/img/tcp_latency.svg)
![udp_bitrate](./docs/img/udp_bitrate.svg)
![mem](./docs/img/mem-graph.png)
## Development Status

View File

@ -0,0 +1,24 @@
#!/bin/sh
RATE="1 1000 2000 3000 4000"
DURATION="60s"
RATHOLE="http://127.0.0.1:5202"
FRP="http://127.0.0.1:5203"
echo warming up frp
echo GET $FRP | vegeta attack -duration 10s > /dev/null
for rate in $RATE; do
name="frp-${rate}qps-$DURATION.bin"
echo $name
echo GET $FRP | vegeta attack -rate $rate -duration $DURATION > $name
vegeta report $name
done
echo warming up rathole
echo GET $RATHOLE | vegeta attack -duration 10s > /dev/null
for rate in $RATE; do
name="rathole-${rate}qps-$DURATION.bin"
echo $name
echo GET $RATHOLE | vegeta attack -rate $rate -duration $DURATION > $name
vegeta report $name
done

View File

@ -0,0 +1,46 @@
#!/bin/bash
rm -v *-mem.log
echo frp
while true; do
ps -C frpc -o rsz= >> frpc-mem.log
sleep 1
done &
while true; do
ps -C frps -o rsz= >> frps-mem.log
sleep 1
done &
echo GET http://127.0.0.1:5203 | vegeta attack -duration 30s -rate 1000 > /dev/null
sleep 10
kill $(jobs -p)
echo rathole
pid_s=$(ps aux | grep "rathole -s" | head -n 1 | awk '{print $2}')
while true; do
ps --pid $pid_s -o rsz= >> ratholec-mem.log
sleep 1
done &
pid_c=$(ps aux | grep "rathole -c" | head -n 1 | awk '{print $2}')
while true; do
ps --pid $pid_c -o rsz= >> ratholes-mem.log
sleep 1
done &
echo GET http://127.0.0.1:5202 | vegeta attack -duration 30s -rate 1000 > /dev/null
sleep 10
kill $(jobs -p)
gawk -i inplace '{print $1 "000"}' frpc-mem.log
gawk -i inplace '{print $1 "000"}' frps-mem.log
gawk -i inplace '{print $1 "000"}' ratholec-mem.log
gawk -i inplace '{print $1 "000"}' ratholes-mem.log

View File

@ -0,0 +1,18 @@
set title "Memory Usage" font ",20"
set term png small size 800,600
set key box outside
set output "mem-graph.png"
set ylabel "RSZ"
set format y '%.0s%cB'
set ytics nomirror
set yrange [0:*]
plot "frps-mem.log" using 1 with lines axes x1y1 title "frps RSZ", \
"frpc-mem.log" using 1 with lines axes x1y1 title "frpc RSZ", \
"ratholes-mem.log" using 1 with lines axes x1y1 title "ratholes RSZ", \
"ratholec-mem.log" using 1 with lines axes x1y1 title "ratholec RSZ"

View File

@ -1,17 +1,19 @@
# Benchmark
> Date: 2021/12/14
> Date: 2021/12/28
>
> Arch Linux with 5.15.7-arch1-1 kernel
> Version: commit 1180c7e538564efd69742f22e77453a1b74a5ed2
>
> Intel i7-6600U CPU @ 2.60GHz
> Arch Linux with 5.15.11-arch2-1 kernel
>
> 20GB RAM
> Intel Xeon CPU E5-2620 @ 2.00GHz *2
>
> 16GB RAM
## Bitrate
## Bandwidth
![tcp_bitrate](./img/tcp_bitrate.svg)
![udp_bitrate](./img/udp_bitrate.svg)
rathole with the following configuration:
```toml
@ -19,14 +21,20 @@ rathole with the following configuration:
remote_addr = "localhost:2333"
default_token = "123"
[client.services.foo1]
[client.services.bench-tcp]
local_addr = "127.0.0.1:80"
[client.services.bench-udp]
type = "udp"
local_addr = "127.0.0.1:80"
[server]
bind_addr = "0.0.0.0:2333"
default_token = "123"
[server.services.foo1]
[server.services.bench-tcp]
bind_addr = "0.0.0.0:5202"
[server.services.bench-udp]
type = "udp"
bind_addr = "0.0.0.0:5202"
```
@ -41,16 +49,20 @@ token = 1233
# frpc.ini
[common]
server_addr = 127.0.0.1
#server_addr = 47.100.208.60
server_port = 7000
authentication_method = token
token = 1233
[ssh]
[bench-tcp]
type = tcp
local_ip = 127.0.0.1
local_port = 80
remote_port = 5203
[bench-udp]
type = udp
local_ip = 127.0.0.1
local_port = 80
remote_port = 5203
```
```
@ -71,17 +83,50 @@ For frp benchmark:
$ iperf3 -c 127.0.0.1 -p 5203
```
## Latency
## HTTP
nginx/1.20.2 listens on port 80, with the default test page.
frp and rathole configuration is same with the previous section.
Using [ali](https://github.com/nakabonne/ali) with different rate.
[vegeta](https://github.com/tsenart/vegeta) is used to generate HTTP load.
### HTTP Throughput
The following commands are used to benchmark rathole and frp. Note that if you want to do a benchmark yourself, `-max-workers` should be adjusted to get the accurate results for your machine.
e.g. for rathole 10 QPS benchmark:
```
ali -r 10 http://127.0.0.1:5202
echo 'GET http://127.0.0.1:5203' | vegeta attack -rate 0 -duration 30s -max-workers 48
echo 'GET http://127.0.0.1:5202' | vegeta attack -rate 0 -duration 30s -max-workers 48
```
![tcp_latency](./img/tcp_latency.svg)
![http_throughput](./img/http_throughput.svg)
### HTTP Latency
`rathole` has very similar latency to `frp`, but can handle more connections
Here's a table, latency is in ms
|QPS|latency(rathole)|latency(frp)|
|--|--|---|
|1|2.113|2.55|
|1000|1.723|1.742|
|2000|1.845|1.749|
|3000|2.064|2.011|
|4000|2.569|7907|
As you can see, for QPS from 1 to 3000, rathole and frp have nearly identical latency.
But with QPS of 4000, frp starts reporting lots of errors and the latency grows to even seconds. This kind of reflects the throughput in the previous section.
Thus, in terms of latency, rathole and frp are nearly the same. But rathole can handle more connections.
[Script to benchmark latency](../benches/scripts/http/latency.sh)
## Memory Usage
![mem](./img/mem-graph.png)
rathole uses much less memory than frp.
[Script to benchmark memory](../benches/scripts/mem/mem.sh)

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 56 KiB

BIN
docs/img/mem-graph.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.6 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 52 KiB

After

Width:  |  Height:  |  Size: 62 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 139 KiB

1
docs/img/udp_bitrate.svg Normal file

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 54 KiB