Performance Test: Does Site B Have Servers In The United States? Comparison Of Latency And Throughput At Different Nodes

explain the test goal: confirm whether bilibili (bilibili) has servers/nodes in the united states, and compare the network latency (latency) and throughput (throughput) of nodes in different regions. preparation work: at least one node in china or hong kong, one node in the east/west of the united states, and one vps in europe or other regions (trial machines such as aws/gcp/azure/linode can be used). install necessary tools: dig/nslookup, traceroute/tcptraceroute, mtr, ping, curl, wget, aria2, geoiplookup/ipinfo, optional iperf3.

purpose: determine whether it points to cdn or us ip by parsing the domain name. command example:

dig +short www.bilibili.com; dig cname www.bilibili.com (check whether there is a cname pointing to the cdn domain name). at the same time, use dig +trace to recursively trace the domain name and record the ip that is finally resolved.

use nslookup -type=any or curl -i https://www.bilibili.com to view the return header (may contain via / x-cache / server / cdn identification).

after obtaining the ip, use geoiplookup or curl ipinfo.io/ to determine whether the ip is located in the united states and the asn/cdn it belongs to.

if the parsing result shows that the cname points to akamai, fastly, tencent cloud cdn, baidu cloud cdn, etc., it means that a cdn is used. in most cases, the cdn will deploy nodes in the target country and not necessarily the original "business server".

execute on each vps: ping -c 20 record minimum/average/maximum/jitter; if icmp is filtered, use tcptraceroute or tcping (for example: tcptraceroute www.bilibili.com 80 or tcping -t www.bilibili.com 443).

use traceroute -n or mtr -r -c 100 -n <domain name> to check the routing path, delay of each hop, and identify whether it is directly connected across the ocean or through a third-party node (such as whether there is a local node in the united states).

find a large and stable static resource url (you can view video fragments.m4s, pictures or static cdn resources through the browser developer tools). run on different nodes:

curl -o /dev/null -s -w "%{speed_download} %{time_total} %{size_download}\n" "url" record rate (bytes/sec); or wget --output-document=/dev/null "url" and view the download speed.

use aria2c's multi-connection capability to simulate concurrency: aria2c -x 16 -s 16 -k 1m "url"; or use curl's range header to fragment parallel downloads (use & to execute multiple curl -r 0-999999... in parallel background).

note: if the cdn has cache, speed limit or connection limit, it is necessary to average multiple tests at different times and compare the improvement ratio of single connection and multiple connections.

if the target exposes the iperf3 service (usually it does not), you can directly use iperf3 -c -p 10 -t 30 measure tcp throughput; otherwise use http parallel downloads instead. also record bandwidth, packet loss rate and number of retransmissions (iperf will report).

write a script to run the above command regularly on each node, and write the output into csv: time, node, ping_min, ping_avg, ping_max, speed_single, speed_multi. example curl output parsing: curl -w "%{time_total},%{speed_download},%{size_download}\n" -o /dev/null -s url >> results.csv.

it is recommended that the test cycle covers peak and low peaks, and each node is tested at least three times a day for one week to smooth out time-varying fluctuations.

delay determination: if the average rtt from the us node to station b for ip resolution is significantly low (<50ms) and the route ends in the us, it means there is a cdn/caching node in the us. if the rtt to us is still high and the route is back to the country, there may only be service within the country or a cross-border link.

throughput comparison: compare single connection and multi-connection results in mbps. if the multi-connection throughput of the us node is significantly higher than that of a single connection, it means that the transoceanic link throughput can be improved concurrently; but if both are low, it may be due to remote speed limit or origin site bandwidth limitation.

do not conduct high-frequency and large-traffic concurrent tests to avoid triggering protection or affecting service stability (to avoid ddos behavior). read the target site's robots or terms of service before testing. record all test times and nodes to facilitate recurrence and appeal.

question: by resolving the us ip via dns, can we conclude that site b has a server in the us?

answer: not necessarily. dns resolution to a us ip usually indicates that there is a cdn node or transit node in the us, but the original business server (origin) may be in china. it needs to be comprehensively judged by combining traceroute, geoip and response header (x-cache/server).

q: during the actual measurement, it was found that the delay of the us node was low but the throughput was not high. how to determine the bottleneck?

answer: let’s first look at the difference between single connection vs multiple connections: if the multiple connections are significantly higher, it may be tcp slow start or single flow limitation; if the multiple connections are also low, it may be cdn restrictions, bandwidth quotas or transoceanic link congestion. combine mtr packet loss and iperf (if available) for further confirmation.

q: i don’t have multiple vpss. is there an online alternative testing platform?

answer: you can use multi-node testing of ripe atlas, fastly/cloudflare radar, cdnperf, and webpagetest.org, or you can use small trial instances of various cloud vendors (aws/gcp/azure) to run the same script sampling in different regions. although the results may be biased, they can provide a reference for regional comparisons.