Friday Facts #136 - Map Transfers

[slpwnd]

Learn about technical aspects of new multiplayer map transfer mechanism: http://www.factorio.com/blog/post/fff-136

[FacToRizeD]

Would it be possible to use a peer-to-peer mechanism for sending the map?
This would allow multiple players to send the map data at once while ensuring the correctness of the data.

[cube]

Would it be possible to use a peer-to-peer mechanism for sending the map?
This would allow multiple players to send the map data at once while ensuring the correctness of the data.

It should be working that way now if you enable the P2P option when starting the game. There might be bugs with that though.

[Woehler]

What, no fancy pictures showing the difference between the new and old transfer protocol D:

[Phillip_Lynx]

This post will be more technical than usual, so let us know if you found it interesting.

I love this type of FFF. Keep on making such posts .

[tyaigan]

But i loved the TL;DR

Too technical for me (coder and FFF lover.)

[Yttrium]

cube spent some time doing computer stuff

Very informative Friday Fact indeed! Keep it up Kappa

[malecord]

And so you started with CUBIC and ended with Westwood. You defeated history sir XD.

Anyway, this FFF made me smile. My master thesis work was about measuring a bunch of innovative cc algorithms when they have to live along each other. You brought me back the memories of the nights I spent metabolizing that RFC stuff you linked. Thanks God is over now.

[Arch666Angel]

All hail the cube!

[Xterminator]

While I didn't understand a ton of the technical stuff, it was still fun to read and gave me a better idea of how things work behind the scenes.

Very glad the transfer speed should be faster now though!

[ratchetfreak]

BTW wifi has its own retransmission setup on packet loss to avoid TCP being throttled way too much. So microwaves are less of a problem than your think (they still are but not as much).

[Meritic]

Well actually me and my friend dont see any difference. Download speed begins at about 1,5kB/sec yet rapidly drops to 0B/s. Actually it is worse then 12.30 because it never returns back as it did in 12.30. Currently we both got 12.31

[Smarty]

All hail the cube!

but you got the color wrong it should be blue






Too technical

[MatthewGP]

I was curious if you were using a Network Emulator for your testing. Something like 'netem' on a linux system or one of many Network / WAN Emulators is invaluable when analyzing how latency, loss, and re-ordering affect your application.

Matthew

[djnono]

I actually love the technical fff
And as i happen to be a sysadmin/network eng., I actually understood all of that

Though I must say the rfcs are one of the worst things ever, they are very informative and contains all the informations, but they sometimes are painful to read.
also there are a few tools that are invaluable (wireshark comes first to mind to debug something like that), and you can easily write a script that send a known data sample to the network as a loop, and one that can receive it, and check it agains a local stored copy of the original. (I had to do that once to find a transient error between several of our systems, I used a 10 gig file containtng pi decimals, i still have nightmare about it )

[Ratzap]

Somewhat light on the technical side but then I'm not really the target audience. It shows how long things take to get into common practice though doesn't it. RFC in 1999 and it's still working it's way into actual use. It's found it's way into some NFS and web server implementations though as options.

[mtocrat]

What I don't understand is why the maps need to be so big. Wouldn't a seed and a diff be both sufficient and relatively small?

[RackAttack]

What I don't understand is why the maps need to be so big. Wouldn't a seed and a diff be both sufficient and relatively small?

I would imagine they don't want to use a seed because that will fail for any sort of custom map, and it will fail if their map generation algorithm ever changes.

[ikiris]

You really shouldn't have done this the way you did it. Either use TCP or something like QUIC, but honestly you should probably go with something distributed like bitswarm considering the potential size of the transfers, and the requirement for speed.

The TCP punching argument is actually pretty wrong. Based on this I get the feeling you're using IP literals, which is pretty broken on its own. See ipv6 and XLAT for examples as to why. Multiprotocol nat punching is no harder than single flow nat punching so I don't understand this argument at all, especially considering you can't ever guarentee you can even reuse a nat state with non specific tuples.

Lastly, for the love of god, please implement a basic UPNP requester, this isn't 1993. There should be 0 need to do this stuff manually with forwarding and firewalls. You're a network client / server app right now. Act like it.


(this rant from a slightly frustrated network/systems engineer)

[Narnach]

Thanks, I enjoyed reading it. I think you managed to communicate a technical topic in a fairly accessible way. That said, I'm a programmer as well, so I might be overestimating the grokkability of this

Funny how we always end up doing a 90% re-implementation of something else (TCP in this case), due to slowly discovering all the edge cases it actually solves.