[raiguard] [2.0.11] Large in-game downloads could be much faster

[DeHackEd]

For context, I have fiber to the home at high speed (gigabit or so). But while inside factorio, downloads (eg: large mods, or game updates) are often self-throttling to about 3 megabytes/second. I know it could go faster because the TCP receive buffers are highly full when I check them:

Code: Select all

tcp   2357492      0 192.168.0.14:54156      138.199.57.66:443       ESTABLISHED 3985/bin/x64/factor 

I also tried direct downloading the same content from a linux shell (factorio logs contain the URL, and I have my own authentication token). I can hit full speed consistently. However the game itself throttles.

My speculation as to a reason is that the menu screen still runs at 60 fps and as a result the network buffers are only checked periodically, resulting in a lot of available bandwidth being left on the table.

[Rseding91]

I am not able to reproduce any such issue when downloading huge mods in-game. I suspect this might be operating system socket specific (I'm on windows)?

[raiguard]

I also cannot reproduce because Factorio is able to saturate my 200 Mbit/s internet connection. Your conjecture might be correct - we update libcurl on the main loop which only runs 60 times per second. Perhaps moving it to a dedicated thread would help things, but that is a fairly invasive change to fix a relatively minor issue.

[DeHackEd]

Based on the netstat output, I suspect that you're not fully draining the TCP buffer each cycle. Are you sure you've pulled as much data as you possibly can from curl before you move on to other game update code? That might help without impacting the code nearly as much, and provide a speed improvement for people who can make use of it. I'm not sure which curl API you're using but there should be a means for it to provide feedback when waiting is *required*. Ideally, keep calling that function until either it reports waiting required, or maybe some amount of time elapses (eg: 5 or 10 milliseconds). I can't imagine how fast your internet would need to be to have the CPU be the bottleneck even processing SSL traffic, but anything's possible I guess.

[Rseding91]

If I could reproduce any issue I would attempt to debug and fix it but I am not able to reproduce any issue. For me, all downloads go at max speed in the current version of the game.