ARM Build

[Hexicube]

Make it so that the biter movement starting is handled by the host, which signals for a biter to move (which will jump ahead according to lag compensation) a set time after the path is worked out (or after all clients signal they've worked out a path), and only allow a certain amount of work to go into the pathfinding each tick (such as 2500 tiles checked) so that there's no desync with paths calculated using different tile data.

Why would we do that?

It would allow threaded pathfinding, which is another step towards using multiple cores better.

[taiiat]

excuse me while i also slightly cry in the corner about direct Processor Core and Hertz comparisons, especially across completely different Architectures.
:v

It would allow threaded pathfinding, which is another step towards using multiple cores better.

a single step is a lot of work for no tangible result at that point.
AI is Single Threaded in EVERY game for good reason - it has to be. nobody on the planet earth has yet to figure out a way to split the calculations to other threads without blowing up some or all of the AI processing. no matter what you split off you create a problem because Processor Cores don't technically talk to each other.

if you split pathfinding, AI will collide with each other constantly and get stuck because they don't know the other AI exist.
actually frankly, if you split basically any segment, this is what happens for everything. once it's in a different Thread, the AI no longer knows that part of the processing exists. and trying to communicate the data back makes the AI Thread even slower, when it's already often a nightmarishly slow Thread in games as it is.

as ever, the story is "making really cool complex AI that acts dynamically to everything is really easy - making that AI actually run on any computer in the world is a different story".

this is why 99% of Video Games still run on 1/2 Threads. usually one Thread for Game state, AI... Physics... the kitchen sink... and one Thread for smaller modules that don't really need to be synced to the core of the game, like User Interface.
inb4 "but i see all 1598276 of my Processor Cores being used, so it must support and utilize that many Threads" - incorrect. software can only utilize as many Threads as it knows how to.
having more Processor Cores doesn't change that. period. a game that can utilize 4 Threads is already something of a rare miracle. pending genre ofcourse. a game that doesn't have any AI to worry about has a huge load lifted and is a lot easier to Multi-Thread.


that's probably more than enough off topic stuff though.

[quadrox]

Since sharing state between threads is an issue for parallellism, wouldn't it be possible to make each thread use the previous state as input? Since the state from the previous tick will no longer be updated, it can safely be shared across threads. This would introduce a tiny amount of lag before collisions and similar would be resolved, but should otherwise work fine.

As for collisions, they will only be detected after the fact and the previous position should be restored, but this should not matter much. I am somewhat confident that other special cases could be resolved as well (e.g. two inserters inserting into the same chest with only one slot left, etc.) through a small amount of clever code (e.g. inserters reserving a slot before inserting). Some of these tricks would reduce performance slightly, but the net gain should still be positive.

This would preserve determinism 100%. As I don't have access to the codebase, I can't show off a prototopye implementation though

EDIT: My inserter example would, in fact, not preserve determinism. hmm... I suppose you could implement a priority system based on the id/memory address of the involved objects. The higher/lower number wins the race, and the other must restore previous state.

Please note that I am not saying this should be implemented, I just can't help trying to come up with solutions to problems Implementing something like this would most likely require a huge amount of effort, and the gains remain questionable.

[ratchetfreak]

Since sharing state between threads is an issue for parallellism, wouldn't it be possible to make each thread use the previous state as input? Since the state from the previous tick will no longer be updated, it can safely be shared across threads. This would introduce a tiny amount of lag before collisions and similar would be resolved, but should otherwise work fine.

As for collisions, they will only be detected after the fact and the previous position should be restored, but this should not matter much. I am somewhat confident that other special cases could be resolved as well (e.g. two inserters inserting into the same chest with only one slot left, etc.) through a small amount of clever code (e.g. inserters reserving a slot before inserting). Some of these tricks would reduce performance slightly, but the net gain should still be positive.

This would preserve determinism 100%. As I don't have access to the codebase, I can't show off a prototopye implementation though

EDIT: My inserter example would, in fact, not preserve determinism. hmm... I suppose you could implement a priority system based on the id/memory address of the involved objects. The higher/lower number wins the race, and the other must restore previous state.

Please note that I am not saying this should be implemented, I just can't help trying to come up with solutions to problems Implementing something like this would most likely require a huge amount of effort, and the gains remain questionable.

It's possible but will increase the memory footprint and require quite a bit of duplicate calculations.

Or some extra lag.

For example an inventory can check which inserters are trying to pull from it and then assign the stack they pull as appropriate. The next tick the inserter sees what it pulled and then goes to deposit to the destination.

There is also the option to use message queues, each entity has a message queue that can be pushed to each tick from any thread. The next tick it will sort the messages deterministically and then process them, possibly sending out messages of its own. In this method a pull from an inventory would be a multi stage process: first the inserter sends WantToPull (possible detailing which items can be pulled), then the inventory gets the message and checks whether the pull request can be handled. After it completes it'll send a reply PullSucceeded with the details of the item pulled.

[quadrox]

It's possible but will increase the memory footprint and require quite a bit of duplicate calculations.

Absolutely!

Or some extra lag.

With an update frequency of 60Hz, lagging for one tick should not be noticeable.

For example an inventory can check which inserters are trying to pull from it and then assign the stack they pull as appropriate. The next tick the inserter sees what it pulled and then goes to deposit to the destination.

Yeah, this could work quite well I imagine.

There is also the option to use message queues, each entity has a message queue that can be pushed to each tick from any thread. The next tick it will sort the messages deterministically and then process them, possibly sending out messages of its own. In this method a pull from an inventory would be a multi stage process: first the inserter sends WantToPull (possible detailing which items can be pulled), then the inventory gets the message and checks whether the pull request can be handled. After it completes it'll send a reply PullSucceeded with the details of the item pulled.

I think such a solution would be too heavy, both memory and performance wise. But yeah, possible.

[ratchetfreak]

there is another factor to consider

A thread-safe engine like that would need a specialized moding api that can only read and request changes to entities getting the success value through a callback the next tick. The current one won't work or be the major bottleneck.

[AsherMaximum]

WHAT: With the addition of a headless server in 0.12, I think an ARM processor build for the server would make sense.

WHY: Cheap server option for playing with friends without requiring one person to keep the game running all the time.

Since a server doesn't have to run the graphics, and depending on how the backend is built might not even have to run the game, (just serve the map file and check synchronization of clients), the resources on small ARM based computers like the Raspberry Pi or Pine A64 should be plenty to run the game, at least for maps that are not gigantic, and it would make it cheap and easy for people to start a server and play with friends.

[bobingabout]

It does run the game, it's the one that redistributes the map etc... It needs to be in sync with what's going on in the game.

[SyncViews]

Is an RPi really powerful enough? I was monitoring the CPU load of my MP game recently, and it is pretty demanding, using 50+% of a core on my 3.5GHz PhenomII server. Was trying to see if an AWS T2.micro will be enough (works out something like $40/year for 24/7, and a lot less if you just run as needed), but honestly I am thinking not, that load would just eat through the CPU credits...


Just distributing and syncing the map with a peer-2-peer mode might work, so it does not really run the game, but just doing an ARM build will not achieve that, it is a much bigger change for the devs (also if the dev's ever did do it, they could then "host" servers on your behalf, which would make the whole MP thing a lot simpler, although thats somewhat a different suggestion/idea).

[immibis]

If you really did want a minimal server, here's what it would need to do: (Note: without the server running the game, you have little security against hackers)

• Coordinate new player connections (so when a player connects to the server, they really connect to one of the other players)
• Periodically download the game state from players (also while the last player disconnects), and store the last game state.
• When a player connects to an empty server, send the last known game state to them.

[AceScottie]

So i know there are probably hundreds of posts about this already.
I have been searching for days for any signs of an ARM capable server with no luck, not even unofficial server builds.

Firstly i know Factorio can be resource intensive at times, and i know its single core only (currently)
Im not asking for a full stable server build for ARM, just a server build, if it breaks fine, if it lags that's ok.

Im not looking to build a mega base with billions of belts and hundreds of trains.

Currently i have 3 options for hosting a server.
Option A. run it on my main PC (because a 8 core CPU with 16GB RAM is not over kill to run 24/7 on a small 4 player max server)
Option B. Run it on a dedicated server i have to pay for ... im not rolling in money over here ...
Option C. Spin up my 24 core, 64GB Server and run it on that (this is like killing a fly with a star overkill)

Currently i have about 10 SBC, including Raspberry Pi (B+, 2, 3, zero) Odrioid C2, Banana Pi, Orange Pi etc...

I have all these devices and can only use the stupidly big ones to run such a little task.
So please, pretty please, release a barely working ARM server or put the sources for the binary up so we can compile them ourselves. I dont even want support with it, more of a "if it dont work for you then tuff" statement will do.

[HurkWurk]

option D, purchase an old refurb business class pc to use as a server. should not cost more than $100.

[netmand]

while you're at it, please compile a version for watchOS! I'd love to be able to host a Factorio game on my apple watch!
(sorry couldn't resist)

[Daid]

They dropped 32bit support for a reason, this same reason rules out the ARM version.

[OdinYggd]

They dropped 32bit support for a reason, this same reason rules out the ARM version.

ARM is used in more than just phones and tablets though. It is a powerful enough architecture to be used in small energy-efficient servers for tasks like web hosting and NAS devices.

I could see an ARM based server being strong enough to run a headless Factorio.

Problem is, building Factorio for armhf instead of amd64 would require some degree of library shuffling and may introduce code bloat into both versions of the game in order to deal with architecture specific optimizations and library requirements.

While it would be nice, that's a lot of effort for a usergroup that is rather small at the present time.

You can get a secondhand Intel Atom or Intel Xeon server for no more per month than it would cost to get dinner for 2 at a burger joint. Such an arrangement would have far better latency and reliability than anything you could host in your house.

[TruePikachu]

The main reason that only x86-64 is supported at this moment is because it is a single architecture, compared to there being multiple different architectures to consider. Here is a list of reasons given in FFF158 for moving away from x86:

• Extra issues related to determinism between 32 bit and 64 bit systems that would have to be solved.
• Release times (compile, upload, updates, server storage space)
• Occasional bugs that we don't notice as no one uses 32 bit system here.
• Confusion of some people that download 32 bit by accident and run out of memory soon.

All but the last point are valid reasons for not having an ARM build; the determinism issues in particular are important to note -- something you don't want with a dedicated server.

Additionally, keeping to a single architecture (x86-64) and compiler (gcc) allows tighter control over optimization; for the three platforms, the only differences are the operating system interfaces. If something not dependant on those interfaces is written in some way, it will almost certainly compile to the same instructions for all three platforms; in contrast, x86-64 and ARM have very different instruction sets, so the actual compiled code will be different and behave differently from a performance standpoint (and potentially a behavioural standpoint as well, given the first item on the quoted list).

[torne]

Supporting 32-bit ARM would presumably be a nonstarter for the same reason that 32-bit x86 is not supported: problems with multiplayer consistency between the 32-bit and 64-bit client. So realistically you'd be talking about 64-bit ARM, which excludes some (but not all) of the devices mentioned in this thread. There are some possible sources of multiplayer inconsistency between x86 and ARM as well though: floating point works somewhat differently for a start (the exact semantics of SSE vs NEON aren't quite the same). I have no idea how much this would affect Factorio specifically, but it's definitely a problem that exists in general. If you're after a server-only version, then it's even *more* important that multiplayer doesn't desync often, especially since pretty much all the clients will be x86 machines.

You also need all your libraries to work (and, yaknow, for a commercial product ideally be fully supported and actually tested by their developers) on ARM, though probably some of them aren't needed for a headless server.

And then after all that.. the performance isn't going to be good, probably. ARM single-core performance is not the best, and many ARM devices in the wild don't have sufficient cooling to actually run a maxed out single-core workload for more than a couple of seconds without going into thermal throttling and significantly reducing the CPU speed (most mobile devices go into thermal throttling in <200ms at full load, and many NAS/microserver style devices are not much better).

[AceScottie]

option D, purchase an old refurb business class pc to use as a server. should not cost more than $100.

$100 might not be much but then add the noise (it is in my bedroom where i have to sleep) and the power usage which is like what 400w.

To give a quick and dirty calculation, a 400w pc running at about 350w (optimal) meaning 0.35kwh running 24 hours a day so 8.4kwh per day.
I pay 13.21p per kwh (in UK pence) so thats 13.21*8.4= 110.964p per day, so basically £1.11 per day ... That amounts to £31 a month if i left it running 24/7...

however lets calculate a SBC as a server.
so they use 5v @ 1.2amp (my own) but to make it a more applicable to everyone, i will calculate to a 2 amp plug. so using Ohms law 5v*2a=10w, so thats 0.010kwh which is simply 0.24kwh per day. again using 13.21p/kwh so 0.24*13.21 3.1704 pence per day. a monthly cost of 89 pence per month

So quite simply the difference between running a full PC as a server vs SBC is about 35.6 times the running cost.

[orzelek]

You are not counting one thing.
Even if you would have ARM build it wouldn't be able to handle big factory.
Due to how game works server runs it's own simulation - that kind of small server wouldn't have that much of computing power I guess.

So after a lot of work to port server to ARM you'd get one that would be able to run small/med factory and UPS would tank after that most likely.

[AceScottie]

You are not counting one thing.
Even if you would have ARM build it wouldn't be able to handle big factory.
Due to how game works server runs it's own simulation - that kind of small server wouldn't have that much of computing power I guess.

So after a lot of work to port server to ARM you'd get one that would be able to run small/med factory and UPS would tank after that most likely.

But do we need more than that ? im not a fan of huge mega factories as quite frankly i dont have the time, but thats one of my main gripes with this game... Time... how many hours upon hours have i wasted waiting for that next research to upgrade to the next step or waiting for some speed moduals to fully service a product line...
If i could just stick it on one of these SBCs for a night running at 10 or 20 ups i would be more than happy, could add a mod to queue up research and let it have at it, or leave it on while i stockpile materials for my next build.

How long does it take to make 100 turbines and a and a reactor ? too long to do by hand but not long enough to automate the process and integrate it into your factory, something like that could be pseudo automated and left to run at night while you sleep, then when you wake up and start playing its done and you can continue to expand.

I know there will be a point where the ARM CPU will not be able to handle a factory, but thats late game 30-40 hours down the line, to me thats where the fun starts but before that you have to get through the time sync.