Page 1 of 1

[Cube] [0.14.21] Client uploads too much data while map download

Posted: Mon Jan 02, 2017 8:35 pm
by Progman
When I connect to a server the client will use a lot of my upload bandwidth to transfer data to the server while downloading the map from the server. The upload bandwidth used is reaching the upload bandwidth I have on my internet connection (I'm having a DSL connection). During the time I'm downloading the map (and blocking the whole upload bandwidth) I cannot do anything else on my internet connection, a simple DNS lookup is stalled until the download is complete (and the upload bandwidth has stopped as well). This also has an effect that I sometimes get the "Server not responding" message in factorio WHILE downloading the map from the server, which makes no sense at all. In some cases I'm also missing a HeartBeat from/to the server because of the blocked upload bandwidth and then I'm out of sync with the server and will be dropped after I catched up on the server. When I'm fully connected to the server (without any heartbeat drops) everything works fine and the game uses only about <1KB/s of network traffic.

This is the result of my connection speed test on speedtest.net
Speedtest.net test
Speedtest.net test
speedtest_net_label.png (45.95 KiB) Viewed 3160 times
And this is the result of my connection speed test on http://speedtest.t-online.de/ (my ISP)
Speedtest on speedtest.t-online.de (my ISP)
Speedtest on speedtest.t-online.de (my ISP)
speedtest_tonline_label.png (62.28 KiB) Viewed 3160 times
This fits the download/upload limit I have for my internet connection subscription (~750KB download, ~140KB upload).
However when I'm downloading the map from any multiplayer server the download rate only reaches about 360KB/s (from the possible ~750KB/s) and an upload rate of about my upload limit is used at the same time as seen below:
Traffic during map download
Traffic during map download
traffic_during_map_download.png (61.13 KiB) Viewed 3160 times
The servers can handle a much higher transfer rate, I can see that when other players join the server in an instant with their faster internet connections, it is not an issue with any server.
Why does the client send so much data (like 1/3 data of the downloaded map size) back to the server? Shouldn't the client send anything back at all to the server (besides the ACKs for the map data/chunks)?

Re: [0.14.21] Client uploads too much data while map download

Posted: Mon Jan 02, 2017 9:19 pm
by Loewchen
I happen to have almost exact the same bandwidth and while I experience the same lower than maximum download speed (360 instead of 700kB/s), my upload is way lower while downloading a map (about 20kB/s).

Re: [0.14.21] Client uploads too much data while map download

Posted: Mon Jan 02, 2017 11:03 pm
by Rseding91
In order to download something you have to send the download requests (upload). That's expected. Download any file off the internet and you see the same thing.

Re: [0.14.21] Client uploads too much data while map download

Posted: Tue Jan 03, 2017 5:22 pm
by orzelek
Can you run tracert to server host and see if you have packet losses?

It is possible that due to lost packets there are a lot of resend requests (assuming Factorio uses something like those).

You could try running Wireshark and see whats in those outgoing packets - it might result in quite big capture file. HAving them on hand could help devs with checking if there is indeed a bug or something wrong with connection.

Re: [0.14.21] Client uploads too much data while map download

Posted: Tue Jan 03, 2017 6:12 pm
by Progman
When I use ping on any server I get 0% package loss.

I have attached a wireshark capture for the map download (my local IP is 192.168.0.58).

Re: [0.14.21] Client uploads too much data while map download

Posted: Tue Jan 17, 2017 2:45 pm
by bergi9
Rseding91 wrote:In order to download something you have to send the download requests (upload). That's expected. Download any file off the internet and you see the same thing.
You mean as ACK-packets?
Downloading from internet via TCP (i.e. HTTP/S) i download with 1.6mb/s and the upload is at 70-80kb/s (for ACKs).
On Factorio i have 1.3-1.4mb/s download and 280kb/s upload.
So that mean: Factorio have terrible big ACK packets?

Re: [Cube] [0.14.21] Client uploads too much data while map download

Posted: Fri Mar 17, 2017 1:15 pm
by Rseding91
I've looked into the map network transfer and there's no additional data that we send except the exact map packets that the peer requests.

Either the packets are getting dropped so they get re-sent or the UDP packet overhead is what you're seeing.

Re: [Cube] [0.14.21] Client uploads too much data while map download

Posted: Fri Apr 21, 2017 12:36 pm
by bergi9
Rseding91 wrote:Either the packets are getting dropped so they get re-sent or the UDP packet overhead is what you're seeing.
Did you implement the TCP slow start like on factorio's UDP protocol too? That could reduce the excessive amount of resending the packets if they're getting lost.