Logistic freeze

[grobyc]

The biggest freeze in game are causes by logistic robots. When logistics robots do their job and all of them are flying its ok, but if they stack over one chest (example 100 of them) game start freezes and till they move you have nice "freeze-land". Maybe its because of their animation, if yes maybe they could stop fly up and down when they are over the chest and all hang in one up position.

Edit: If more U have them (over 500) game start to be upsetting.

[SilverWarior]

The problem is not in their animation but in pathfinding and probably searching for another posible job.

As soon as those logistic robots get some item sfrom the chest they then need to calculate proper path to their destination. Based on my observation I belive Factorio is using A* pathfininding algorithm which can become quite slow on large levels. Not so slow that you would notice for a single logistic robot, maybe a few milliseconds difference in needed time for finding proper path, but when this needs to be done for may logistical robots the cumulative efect can be quite noticable.
So it would be much better to use heuristic pathfinding algorithm but it is a bit more dificult to implement it and if it isn't implemented corectly it can be even slower than plain A* pathfinding algorithm.

And if all those logistic robost have just finished some job they may be checking every logistic chest you have to find the nex job. So if you have many logistic chest it can take some time for logistic robot to check if any of them has a job to be done. Again similar as in pathfinding the large number of chests won't cause noticable delay for one single logistical robot, but the cumulative efect for many logistical robots can again be quite noticable.
@Developers
As a game developer myself I had been thinking about how to implement a good and effective job-worker related system for one of my future games. The way I imagined it would work there shouldn't be any noticable performance drop when number of available jobs or idle workers becomes large (checking for available jobs or available workers is not done on time intervals but instead when jobs gets created or certain woker finishes his current job).
I haven't managed to test my idea yet (game development hasn't reach the phase where I could use it) but if yu are interested I can write a short paper explaining logics used behind my idea so you could go and implement it by yourself.

[slpwnd]

The biggest freeze in game are causes by logistic robots. When logistics robots do their job and all of them are flying its ok, but if they stack over one chest (example 100 of them) game start freezes and till they move you have nice "freeze-land". Maybe its because of their animation, if yes maybe they could stop fly up and down when they are over the chest and all hang in one up position.

This is not an expected behavior. How many logistic chests did you have (more or less) ?

The problem is not in their animation but in pathfinding and probably searching for another posible job.

Logistic robots don't use any pathfinding. When they know from where to where to go they just go there straight. They have no collisions. However the searching for another possible job could be a problem if there is a lot of logistic (requester / provider / storage) chests around.

Based on my observation I belive Factorio is using A* pathfininding algorithm which can become quite slow on large levels.

Yes it does, however only for creepers and trains. For creepers there are a lot of small tricks how to avoid freezes (using path cache, interrupting path finder after specified amount of steps and continuing the next tick, etc.). However there is still a lot of space for improvements (I was toying with idea of using Jump Search heuristic for a while).

As a game developer myself I had been thinking about how to implement a good and effective job-worker related system for one of my future games. The way I imagined it would work there shouldn't be any noticable performance drop when number of available jobs or idle workers becomes large (checking for available jobs or available workers is not done on time intervals but instead when jobs gets created or certain woker finishes his current job).

We also figured this is a problem. So for instance every turret is checking whether there is someone in the vicinity it can shoot at. However this can hardly be made event based because then all of the moving units would have to emit events about position change. That could get even more performance heavy.

We do get around some important stuff. Like for example serializable path finding I mentioned. It should make little difference whether 50 or 500 units are dispatched. In theory

I suppose there are multiple solutions for the logistic robots:
1) Do limited amount of checks in the tick and serialize the state.
2) Check the "transitions". That means only trigger some action when new stuff is put / taken from the logistic chest (or a chest is build, etc.).
3) Use some heuristics for planning what needs to be taken from where to where. This search could be the core of the slowdown.

[SilverWarior]

The solution I had in mid was something like this:
You have two list. First lists contains data for all idle workers (logistic robots) and second lists contains data for all available jobs.

**worker part**
When you create new worker or existing worker finishes current job you check if there is any available job for this worker in Job list. If there is no job available for the worker you go and add that worker to Idle workers list. But if there is job available you go and assign that job to that worker.

**job part**
When new job becomes available you check to see if there is any idle worker in Idle worker list. If thre isn't any idle worker available you add that new job to Job list. But if there is available worker you go and assign the job to it. Here you also have ability to actually assing job to several workers if the job requires (moving 10 items when each worker can only move one item a time) and there is enough available workers. And if there arent enough workers to comepleetly fulfill the job (moving 10 units and only 5 workers available) you can assign that five idle workers to go and move the 5 items, and add modified job to job list which requires moving of rest of the items.

Now as you see this system doesn't need interval checking for idle workers or available jobs becouse all the checkings are only made when the status of idle workers or available jobs gets updated.

Now if you have different kind of jobs it would be best to have multiple lists (one for each job). The same goes if you have different worker types which can only do certain jobs.


Now the only problem for implementing this into Factiorio right now (the one that only ocured to me right now) is the need to have job system designed in a way so that at any time the game knows which jobs can be compleeted (necessary resources available). Implementing this might be a bit more complicated but it is doable.

First you would need some system which would alow you to quickly check if the necessary resources are available.
Here I would suggest some data tree like implementation:
- tree root contains information for entire stock of every logistic provider chest
- first level tree braches contains information for entire stock of every logistic provider chest in certain map chunk
- second level tree braches contains information about stock for every single logistic provider chest.
This system should update itself everytime when some item is either put or taken out of logistics provider chests.
First level of tree braching could come verry useful for finding nearby chests so logistic robots won't go get some item on the other end of the map when they can get it somewhere nearby. And you won't have to iterate through every chest that do coneint that item and calculate which is closer.

Second you would need a system which would automaticly reserve some needed items when a job is taken by some worker to preven multiple workers trying to compleete mupltiple jobs using same item.

As third you would need a mechanizm which would be evaluating which jobs can be compleeted.
Since we have quick system for checking for available resources you could simply make sure that jobs are being evaluated on certain intervals. This shouldn't cause major slowdowns unles you have huge number of pending jobs.
Another more eficient but harder to implement system should reevaluate jobs only upon sotck changes. But this would require dividing jobs into another data tree or multiple list so you can quickly find the jobs which require certain item (limiting reevaluation of only those jobs that require such item).


If there is some part about my idea isn't understandable please do say so and I'll try to explain it better.

DISCLAIMER: When I'm talking about lists I don't have in mind lists of objects but instead lists of pointers to objects. Why?
1. Since you are only operating with pointers the lists operations are usually much quicker
2. You get ability for two or more lists to refer to same object - can be quite useful in certain scenarios

[grobyc]

Problem solved, and I was create this problem setting in requesert chest maximum value.

I set request chest and require "it must be full of materials" and in filter set copper ore and move slider to maximum (20,000) when box can handle only 3,072. 100 logistic bots was taking 4 copper ore each (400 together) and they fly to one chest and try put inside materials when there was place only for 4 copper ore, they arrive to chest and drift with materials over one chest. When I down slider to 3,000 in requester chest logistics robots work properly.

P.S. I use a lot of logistic chest, about 50 provider and 400 - 500 requester

[SilverWarior]

Problem solved, and I was create this problem setting in requesert chest maximum value.

I set request chest and require "it must be full of materials" and in filter set copper ore and move slider to maximum (20,000) when box can handle only 3,072. 100 logistic bots was taking 4 copper ore each (400 together) and they fly to one chest and try put inside materials when there was place only for 4 copper ore, they arrive to chest and drift with materials over one chest. When I down slider to 3,000 in requester chest logistics robots work properly.

P.S. I use a lot of logistic chest, about 50 provider and 400 - 500 requester

So there is a bug afterall! UI for requester chest alows setting desired amount to a higher value than its capacity its when the maximum number of desired amount of items should be only the maximum capacity of the chest.
So I suggest to developers to fix that and I also suggest that GUI should properly limit the maximum number of desired items when having more than one item type selected (if item type has already selected quantity of 1000, item type two maximum desired quantity shouldn't be higher than chest storage minus item type one desired quantity).

[kovarex]

Hmm, so first step of optimisation of logistic manager made it more than 100X faster (for big logistic system).
I believe it can be still 100X faster, but I will leave that for bigger systems

[grobyc]

It's good logic system got a bust and will work properly and after filter in storage chest it will be awesome.