CPU performance benchmarks

[madpav3l]

The improvement in the "568 hours factory" is huge. Here is the save saved in Factorio version 0.16.22 because loading version older than 0.13.00 is disabled in Factorio version 0.16.XX.

Copy/paste from my reddit post where I tested 4 maps for improvements:
i7-8700K OC 5.0 GHz / Cache 5.0 GHz, 16 GB DDR4 3800 MHz 16-17-17-36-1T. Game and system installed on Samsung SSD 960 PRO (M.2) NVME (read 3500 MB/s, write 2100 MB/s).

0.15.40 - 40,5 to 42 UPS
0.16.0 - 170 to 180 UPS

I doubt that Ryzen will catch up, Ryzen's poor performance in Factorio is due to high latency to RAM.

[lexx]

Same here on my ryzen system it's doing 60 ups now

I did not think about uncapped ups I be back 2 hours and I can uncap the ups to see what I am getting

Ryzen 1800x 8 core 3.92 (reported voltage under load 1.4v) , 32gb ram 16cl @2933 as system wount post at 3200 (corsair 3200 LPX 16cl 2x16gb kit)

I was getting around 22-24 ups 0.15.40 and max 60 on 0.16 (after 10-30 seconds, I was not counting and it was a rush check as had to do work)

OK I looked at the bottom of that reddit post I run all them maps as well with 15.40 and 16.0 at 50x game speed to see how my ryzen system stands up

The big issue was the ram controller been hammered when in late game with large base so if that's been Grateley reduced it be intresting what I get (considering I am still hampered by clock speed and 2933 ram)

as it was getting to the point where I was considering replacing my ryzen system for 8700k 3600/15cl Just because of one game (other games and programs work as expected) but don't really have the money for it (well I could but that limit me getting a monitor and video card)

[lexx]

102 zoomed out max and 110 default zoom (only done the game speed command)

map view dropping as low as 70 UPS

[hoho]

102 zoomed out max and 110 default zoom (only done the game speed command)

map view dropping as low as 70 UPS

That's quite likely due to having to blend together a metric ton of tree and shadow sprites and not so much about CPU limitations.

I wonder how much overdraw factorio does when rendering a forest with trees, their shadows, cloud shadows and player lights.

[aober93]

102 zoomed out max and 110 default zoom (only done the game speed command)

map view dropping as low as 70 UPS

Looks like the relation hasnt changed much then. But an OC coffee lake is expensive, mostly due the OC ofc.

[bergi9]

i7-8700K OC 5.0 GHz / Cache 5.0 GHz, 16 GB DDR4 3800 MHz 16-17-17-36-1T.

Does the CPU Cache Ratio overclocking affect Factorio? Because mine i7-6850K is OC to 4.4GHz but Cache Ratio is set as Default.

Edit: i overclocked my CPU Cache ratio from 31 to 38 and gained nice boost on transferspeed between the caches and memory. Also the latencies in the caches and memory decreased too.

[madpav3l]

@bergi9
I did not measure it exactly but I think it does because affect the performance. As you said the latencies are lower.

[dragontamer5788]

I would expect a game like Factorio to be memory-latency bound. AMD Ryzen has ~70ns of DRAM -> Register latency. Intel Skylake has~45ns of DRAM -> Register latency.

The prevailing theory is that Ryzen's Cross-CCX layer is adding a bit of latency. Note that Ryzen's memory organization is DRAM -> Infinity Fabric -> Core Complex (L3 -> Core (L2 -> L1 -> Register)). The parenthesis are there to help, because its a bit complicated. There are 2x Core Complexes on "big" Ryzen 1800X or 1700X chips. The smaller APUs (laptops) have only one Core Complex (CCX) and have lower latencies. Ryzen has doubled the size of the L2 cache to try and compensate for this fact (512kb on Ryzen but 256kb on Skylake), but Factorio has a huge number of updates per game-tick. There's no way it'd fit into L2 or L3 RAM.

There are 2 core complexes, and 4-cores per complex. That's why AMD was able to get such high core counts. Its an innate tradeoff: they have higher latencies but way more cores than Intel's design. There are also some interesting cost-savings with the way AMD is doing their system. Intel's organization is DRAM -> L3 -> Core (L2 -> L1 -> Registers), which is a more direct path but probably harder to build.

On a per-core and per-instruction basis... Ryzen is surprisingly competitive vs Skylake (within 10%). Its the latency in the memory system that seems to be hampering Ryzen in this particular game. Skylake has better AVX units (probably not used in Factorio??) and better DRAM -> Register Memory Latency.

Anyway, memory latency is when you read something from memory, change it a little bit, and then write it back out to memory. Memory latency hasn't changes in decades, it has always been around 50ns for computers. (Even as computers get faster and faster, latency has remained steady). One thing that HAS changed is how "parallel" DRAM can be. DDR4 can have 16 requests in parallel (due to the internal 16 banks), and with 2 channels, you can have 32 requests in parallel. With a large amount of cache (L1, L2, L3) on modern systems, as well as "out of order" execution, the growing latency can be "hidden" through parallel requests.

Basically, the optimization focus for modern systems is to focus on these "latency hiding" features of modern CPUs. Do whatever you can to assist the branch predictor, the cache, and yes: speculative execution. The 50ns of latency has been in existence for decades, and its unlikely to change anytime soon.

---------------

For the players: Other "simulation" games like Dwarf Fortress have also been primarily latency-bound. So of course, buy low-latency systems (ie: Skylake over Ryzen). If you still want Ryzen (its faster in some tasks), then do what you can to minimize latency. High MHz RAM works for Ryzen, particularly because the "infinity fabric" syncs to the memory clock. The faster you run the memory, the faster "infinity fabric" operates.

Buy low-latency RAM. There's 3 major operations for DRAM. Page Hit (tCAS Latency), Page Empty (tRAS + tCAS Latency), and Page-Miss (tRP + tRAS + tCAS Latency). There are other timings, but those three are the big ones. You'll want to buy RAM with the lowest latency (CAS Latency is usually easy to find), although the marketing numbers are a bit weird. CAS Latency is measured in clock cycles, not in nanoseconds for some reason. So be sure to divide by the MHz.

DDR4 (which has higher MHz and allows for more parallel accesses than DDR3) may help if the code allows the branch predictor and caches to do its job.

[NimbleCZ]

Anyone Ryzen 3700x benchmarks?

[slippycheeze]

I would expect a game like Factorio to be memory-latency bound. AMD Ryzen has ~70ns of DRAM -> Register latency. Intel Skylake has~45ns of DRAM -> Register latency.

Basically, the optimization focus for modern systems is to focus on these "latency hiding" features of modern CPUs. Do whatever you can to assist the branch predictor, the cache, and yes: speculative execution. The 50ns of latency has been in existence for decades, and its unlikely to change anytime soon.

Also remember that Spectre and friends mean that some of the speculative execution benefits are now disabled, because they could also be used to read arbitrary memory from an unprivileged process. So, y'know, JavaScript in the browser could scan through the kernel and find cryptographic keys, fun times.

Anyway, I think that latency at a human scale is a great document if anyone wants to understand a little more about exactly how bad this latency is: it doesn't address AMD vs Intel, but it does address why memory is so bad.

The executive summary is: if one CPU cycle - roughly one instruction - is 1 second long, it would take around four minutes or so for data to get from memory to the CPU if you get stuck waiting on it. (Don't even ask about your SSD: that'd be several days away.)

[lexx]

It would be interesting, map test on the 3700x and 3900x and 3950x

One important note is that you don't want the game to be chip let/ccx hopping on the 3700x/3900x windows 10 task Scheduler is still not keeping work on one 8 core ccx group, so more ideal if you limit it to first 8 threads for factorio or games that don't use more then 8 threads so does not have to do L3 cache copy to other ccx,

If you streaming on the 3950x (16 core) is best to place obs on last 8 threads and game on first 16 threads (factorio/ksp/oxegen not included on first 8 threads ti stay within one ccx group), 3700x split it 8 game 8 obs), the odd ball one is the 3900x 12 core as you have only 3 cores per ccx group so game ideally be first 6 threads game and last 6 threads for obs for optimal performance

The latencies from L2, L3 and ram controller been slighter higher then 2000 (about 10 ns on ram controller) cpus I don't expect miracles here on the 3700x and 3900x, but other IPC improvements to a load of stuff on L1 L2 L3 seems to negate most of it and still ends up been better then the 2700x (apox 15% or higher) but yet to see if that 10ns actually causes it to performance the same or slower on factorio (should still be better then 2000 cpus)

https://www.anandtech.com/show/14605/th ... -the-bar/2

does support 3600 ram and that's with infinity fabric at 1:1 ratio (3733 and higher requires 2:1 ratio on infinity fabric witch basically nullifies the higher ram speed in latencies)

A lot of tests seem to have it very matching to 9900k or just not far off it but it be interesting to see how factorio will react to it on a very heavy map (my 1800x even thought acts more like a i7 4th gen K cpu in factorio has not really given me problems because most headless servers run on a xeon with ecc ram witch automatically makes it slower then my cpu)

From what I can see, if you just compare the max fps of the 2700, 3700 and 8700k/9900k Intel cpu, the 3700x seems to get very close

The the lower max fps of the 1800x, 2700x is very reflective of factorio actual performance scaling and the 3700x seem to be very close to a 9900k so should be interesting

[orzelek]

I'm kind of itching to get one but might wait a bit since there is no good info on 3800X still - it seems it's not available yet.

[lexx]

I'm kind of itching to get one but might wait a bit since there is no good info on 3800X still - it seems it's not available yet.

3700x vs 3800x at best is 2% difference for £100 more not really worth it unless you want he best (just use a better cooler and the 3700x will clock to same as 3800x) the actual difference between the 3700x and 3800x is not the cpu but the included cooler (one is 65w rated other one is 95w rated) actual power draw between the 3700x and 3800x is only 4w difference when using there included coolers (so the 65w 3700x is not following its TDP limit)

I currently own 1800x so really thinking of pulling the trigger on the 3700x but my ram will only do 2933 on this motherboard (might do 3200 with 3700x but the Asus prime pro x370 ram traces are not optimised for high speed RAM)

So bordering on the idea of getting the x570 board witch may end up me buying low latency 3200-3300 ram as well and clocking it to 3600 cl16 or what ever I can get away with (at this rate might end up buying a full system lol) but I must refine from doing that (at the moment)

[orzelek]

If people are intersted and have some heavy up-to date save I can run some tests on Ryzen 3700X with 3200 ram.

[azesmbog]

People are VERY interested.
If the proposal is still valid, I would like to check my game saving on modern processors from AMD, but it is also possible from Intel 8700/9700/9900.
On my current configuration, FPS / UPS does not rise above 47 anymore, and this is with minimal graphics settings.
If possible, then providing fresh freshness.
Version current 0.17.69, vanilla.

And one more question after it - is it possible to test the game for maximum performance, what is the system capable of, so that the FPS is ABOVE 60 ??? What is waiting for this to enable / disable / give the necessary command?

[Bilka]

I benchmarked my 2700x and 3900x both with the same memory (3200MHz, CL14) on this map: https://www.reddit.com/r/factorio/comme ... _with_way/

The 2700x gets 73 UPS (stock clocks), the 3900x gets 84 UPS (overclocked to 4.7 GHz)/ 81 UPS (stock clocks). An overview of general benchmark results can be found here: http://factoriobox.1au.us/results?map=c ... 2&sort=ups

Testing was done with the script here: http://factoriobox.1au.us/

[azesmbog]

Thank. But this is a very very very small base.
On it, my UPS / FPS is certainly not ideal 60, sometimes it drops to 54, with my own card 47 at best, and so it drops to 30-40
Thanks for the script, I'll try to figure out the launch
as I understand it, the script downloads some kind of its map and runs in my conf?
I don’t know if he worked correctly, but if 82 UPS is the result, then is this a pretty good result?
upd2: With repeatability, the test is somehow bad, I ran it again - 87 UPS

[ChoMar]

I wanted to try my 3900x with the newly arived 3600 RAM but I cant get the script. Any other way to test with uncapped UPS?

[jape3]

I wanted to try my 3900x with the newly arived 3600 RAM but I cant get the script. Any other way to test with uncapped UPS?

If you're running linux, copy that "10k megabase" test map to your saves folder first. Then you can run on terminal (folder /factorio/bin/x64) the following:

./factorio --benchmark "stevetrovs_10K_mega_belt_base.zip" --benchmark-ticks 1000 --disable-audio --mod-directory /dev/null --benchmark-runs 5

that will output the results on your terminal, you can pick the execution times from there manually and from those calculate the ups.
On windows it looks like you need to replace /dev/null after --mod-directory with something else. Haven't tried it myself.

Just tried it with my e3-1231 v3 w. 16GB of 1600MHz DDR3, got 54UPS.

[azesmbog]

Just tried it with my e3-1231 v3 w. 16GB of 1600MHz DDR3, got 54UPS.

Thanks for the command line
For windows all the same, I got this:

factorio --benchmark "stevetrovs_10K_mega_belt_base.zip" --benchmark-ticks 1000 --disable-audio --mod-directory /dev/null --benchmark-runs 5 >test.txt

Just redirected the output to a text file.
It turned out two tests, one with this standard base, one with mine
Can you help me interpret the results in the UPS?))

Performed 1000 updates in 31970.164 ms
avg: 31.970 ms, min: 19.713 ms, max: 1136.875 ms

Are these the final results ??