Optimizing Factorio on Linux with new scheduling framework

[dvernet]

TL;DR

I need a saved game file with a very large Factorio instance / world, so as to investigate optimizing Factorio with different scheduling policies on Linux.

What ?

A colleague and I have been working on a new pluggable scheduling framework in Linux called "sched_ext". sched_ext allows scheduling policies to be implemented as BPF programs, and we've been able to use it at Meta to optimize our key web workload by 1.25 - 3% throughput, and 3 - 6% p99 latency. We're actively rolling it out to production at Meta, and at the same time are working on getting it merged upstream. The most applicable work we can do at the moment to get it merged upstream is to showcase its value, and optimizing gaming on Linux is a large class of applicability.

Request

That brings me to why I'm posting this question. Factorio presents a compelling possible use case for investigating how different scheduling policies could optimize gaming on Linux. Very large factories would likely be CPU bound, as the game will have to continuously process all of the operations that should take place in parallel across the factories in the world.

With all that said, would the admins or anyone else be willing to share a save file that contains a very large instance of a Factorio game? Meaning, a game which has a very large number of factories and components throughout the world, and which requires heavy use of CPU to run smoothly? If someone would be willing to share that, I could use it as my benchmark for investigating scheduling policies that would improve Factorio on Linux.

These tests would (for now) be run on a 16-core AMD Ryzen 9 7950X, with an MSI S360 AIO Liquid Cooler, an AMD Radeon RX 7900 XT, and 64 GB of RAM. If that goes well, I'll probably also buy and experiment with different scheduling policies on a Steam Deck, as well as one of the new exotic AMD Ryzen 9 7950X3D processors (the ones with a 3D-vcache on top of only 1 of the 2 CCXs on the chip): https://www.amd.com/en/products/apu/amd-ryzen-9-7950x3d

Relevant links

Latest upstream patch set (v3): https://lore.kernel.org/all/20230317213 ... ernel.org/
Github repo: https://github.com/sched-ext/sched_ext
sched_ext subreddit: https://www.reddit.com/r/sched_ext

Why ?

This has the potential to significantly improve the performance of Factorio on Linux. See https://www.anandtech.com/show/18747/th ... ocessor/11 for example, where AnandTech mentions the following:

As we can see in our Factorio benchmark, we saw massive gains of over 100% when forcing the Ryzen 9 7950X3D to use the CCD with the 3D V-Cache, as opposed to letting AMD's PPM Provisioning and 3D V-Cache Optimizer drivers do their things automatically. Otherwise, when left to their own devices, X3D software stack – and specifically, the Xbox Game Bar – weren't able to recognize our Factorio benchmark run as a game that warranted intervention.

CPUs are getting a lot more interesting. Factorio is in a nice position for us to test out different scheduling algorithms for games in that it can easily become CPU bound. This work has the potential to of course benefit Factorio, but also the Linux gaming community more broadly.

Thanks for your time!

[Tertius]

There is a thread about large saves, I'm certain some of the saves are still available:
viewtopic.php?f=5&t=105733

[jodokus31]

Here is a collection of saves for performance investigation:
https://factoriobox.1au.us/

[flame_Sla]

I wonder how you will deal with cache misses from the processor?

[sirwalter]

Have you looked into the mimalloc2 memory allocator on linux for factorio? It seams to give some meaningful improvements aswell.(at least for factorio)

Also I have this repository which can compare different allocators. (The factorio benchmark isnt documented in the readme but I think it's part of the hlp option. )https://github.com/suisseWalter/mimalloc-bench it contains a factorio benchmark. It currently uses a small/medium sized map. (You would have to change the download link to a bigger map.)
ignore all of this i was stupid.


Also from some of my testing it seams like different kinds of maps (train/belt/bots) have different kinds of cpu bottleneck, so they might also be affected differently. Also modded maps, that have a lot of scripted actions, might react even more different. But you should find all the possivle maps on factoriobox.