[Cube] [0.14.21] Client uploads too much data while map download

[Progman]

When I connect to a server the client will use a lot of my upload bandwidth to transfer data to the server while downloading the map from the server. The upload bandwidth used is reaching the upload bandwidth I have on my internet connection (I'm having a DSL connection). During the time I'm downloading the map (and blocking the whole upload bandwidth) I cannot do anything else on my internet connection, a simple DNS lookup is stalled until the download is complete (and the upload bandwidth has stopped as well). This also has an effect that I sometimes get the "Server not responding" message in factorio WHILE downloading the map from the server, which makes no sense at all. In some cases I'm also missing a HeartBeat from/to the server because of the blocked upload bandwidth and then I'm out of sync with the server and will be dropped after I catched up on the server. When I'm fully connected to the server (without any heartbeat drops) everything works fine and the game uses only about <1KB/s of network traffic.

This is the result of my connection speed test on speedtest.net

Speedtest.net test

speedtest_net_label.png (45.95 KiB) Viewed 2495 times

And this is the result of my connection speed test on http://speedtest.t-online.de/ (my ISP)

Speedtest on speedtest.t-online.de (my ISP)

speedtest_tonline_label.png (62.28 KiB) Viewed 2495 times

This fits the download/upload limit I have for my internet connection subscription (~750KB download, ~140KB upload).
However when I'm downloading the map from any multiplayer server the download rate only reaches about 360KB/s (from the possible ~750KB/s) and an upload rate of about my upload limit is used at the same time as seen below:

Traffic during map download

traffic_during_map_download.png (61.13 KiB) Viewed 2495 times

The servers can handle a much higher transfer rate, I can see that when other players join the server in an instant with their faster internet connections, it is not an issue with any server.
Why does the client send so much data (like 1/3 data of the downloaded map size) back to the server? Shouldn't the client send anything back at all to the server (besides the ACKs for the map data/chunks)?

[Loewchen]

I happen to have almost exact the same bandwidth and while I experience the same lower than maximum download speed (360 instead of 700kB/s), my upload is way lower while downloading a map (about 20kB/s).

[Rseding91]

In order to download something you have to send the download requests (upload). That's expected. Download any file off the internet and you see the same thing.

[orzelek]

Can you run tracert to server host and see if you have packet losses?

It is possible that due to lost packets there are a lot of resend requests (assuming Factorio uses something like those).

You could try running Wireshark and see whats in those outgoing packets - it might result in quite big capture file. HAving them on hand could help devs with checking if there is indeed a bug or something wrong with connection.

[Progman]

When I use ping on any server I get 0% package loss.

I have attached a wireshark capture for the map download (my local IP is 192.168.0.58).

[bergi9]

In order to download something you have to send the download requests (upload). That's expected. Download any file off the internet and you see the same thing.

You mean as ACK-packets?
Downloading from internet via TCP (i.e. HTTP/S) i download with 1.6mb/s and the upload is at 70-80kb/s (for ACKs).
On Factorio i have 1.3-1.4mb/s download and 280kb/s upload.
So that mean: Factorio have terrible big ACK packets?

[Rseding91]

I've looked into the map network transfer and there's no additional data that we send except the exact map packets that the peer requests.

Either the packets are getting dropped so they get re-sent or the UDP packet overhead is what you're seeing.

[bergi9]

Either the packets are getting dropped so they get re-sent or the UDP packet overhead is what you're seeing.

Did you implement the TCP slow start like on factorio's UDP protocol too? That could reduce the excessive amount of resending the packets if they're getting lost.