Friday Facts #203 - Logistic buffer chest

[loganb]

Hello,

This new buffer chest is excellent! Though one challenge I did like was figuring out how to supply distant bases. I am a little disappointed that this change will make it so that there's no reason not to run one humongous logistic network across the whole map and never solve the problem of delivering diverse resources via trains to distant areas.

What if the power consumption of a logistic network increased with the square of the roboports. A "communication" overhead if you will. Or maybe increases with the area of the bounding box of the network. That way it's Prohibitively expensive to just stamp out roboports along the rail paths, and there's at least some reason to build multiple logistics networks. It's a great puzzle, so I hope y'all don't totally nerf it away.

[Selvek]

I'm sorry, but I fail to see how the buffer chests would be better than adding filters to storage chests.

The buffer chest looks to me like a storage chest with filters, and I can't seem to tell the difference between the two. Could someone please enlighten me?

But I do agree the concept of a filtered storage/buffer chest is very useful. Thanks for adding it.

It's easy. Storage chests are yellow. Buffer chests are green.



My guess is that the real difference is buffers will actually have item requests with desired amounts. A storage chest with filters would prevent the wrong items from being placed in it, but would not actually have items intentionally delivered to it.

[Selvek]

Now we only need a way to make sure the bots fly sensible routes, without passing over things like power backout zones, or behemoth spitter nests

This. PLEASE!!!

I've always wondered what the "logistic area" and "build area" of a roboport actually mean physically, since bots have no problem wondering outside of them whenever you have a U-shaped robo network with large biter concentrations in the middle

[IronCartographer]

I've always wondered what the "logistic area" and "build area" of a roboport actually mean physically, since bots have no problem wondering outside of them whenever you have a U-shaped robo network with large biter concentrations in the middle

"Leave these areas and you're on your own, dumb bots."

There's not much that can be done. The bots go in straight lines for performance reasons, with no consideration of whether they'll survive the journey.

[QGamer]

and I can't seem to tell the difference between the two.

It's easy. Storage chests are yellow. Buffer chests are green.

All right...I guess I should learn to ask better questions...

My guess is that the real difference is buffers will actually have item requests with desired amounts. A storage chest with filters would prevent the wrong items from being placed in it, but would not actually have items intentionally delivered to it.

Couldn't you just use active providers to "push" the items into the filtered storage chests?

Anyways, the buffer chest idea is a cool idea.

Now could you do something about construction robot logic? I was playing yesterday* and I ordered my construction robots to build a solar farm. I also ordered them to deconstruct some solar panels that were in the wrong place. The robots took the panels to the storage chest, put them in, and then immediately took them out and placed them in the farm I wanted built. They wasted about three seconds visiting the chests for no reason.

*Or maybe is was the day before; time flies when you're having fun.

[FreezyExp]

Will there be a way to load balance the buffer chests?
for instance:
red at factories -> yellow/green in main base -> green at key perimeter points -> green chests at turret spots for resupply and repair.

It might be useful to be able to spread out a varying supply over a certain set of chests.
perhaps greens could get a priority (low priorities would fill up last, like in base buffers)

Bot flight paths should limit themselves to node based path finding, with the nodes being the roboports.
A small variance is allowed, but no long treks over non-supported areas. we have landfill now, we can use it to place roboports anywhere.

This has the benefit of also being able to show a hot-map of bot flight paths, because they would be consolidated.
This in turn would allow us to refine the network by expanding the number of nodes.
Perhaps we could also set roboports to spread out more evenly and even trade bots or have item hand-off mid-air for speeding up delivery times by not waiting for recharge.

A green chest mechanic in the roboport perhaps?

[Xiphoris]

Typically when you set up all the provider chests, they are spread out across your whole factory. When you return to base for a resupply, you end up waiting for a long time while the bots travel from all over the factory with items. By using a buffer chest, you can setup a dedicated 'supply area', where the buffer chest will already contain all the typical items, and the bots can quickly top-up your inventory.

I've been wanting something like this for a long time - this is amazing! I've tried to simulate it with circuit networks and have largely failed. I'm really glad this is going to be part of the game.

[Engimage]

Thank you guys for buffer chests. Glad you keep listening to players.
I would like to extend possible improvements to logistic system
Currently problems arise cause of generic use of chests. If we could configure request and provision purpose/group this might help. For example we could set passive provider chest only to provide items to logistic requests and not to construction bots. Same with requests. Totally we can split it to 3 types - player, logistics, construction
With this settings we could fine tune bots behavior.

[Sir Arcana]

A while ago I was thinking that you could simply wire together two chests to force them to ignore each other, so that a requester could pull any item from any other supplying chest except that one, and that a supplier could push any item to any chest except that one. The buffer chest seems to work much better, though. A bit less fiddly.

[TheTom]

Decouple draw logic.

It already is.

Decouple routing decisions. Keep a separate path network (robots, TRAINS) and handle pathfinding on separate threads.

Not feasible. Routes become invalid during game updates which means you can't calculate a path while the game updates. Routes also effect the cost of the next route so they can't be done in parallel.

Decouple AI At least high level (i.e. groups of biters).

Exact same problem as trains.

Logical non squitur, sorry.

Routes become invalid ANYWAY - regardless whether they are perfectly calcualted or not. Deadlocks may develop, trains get stuck, the player changes the network. In fact, the current "precalulate a train route" is a problem already because it bypasses every train on the network - which sadly are moving.

Once you accept you can not rely on a route to be perfect, this is totally doable.

* Main thread posts "network update" messages into a queue.
* Separate thread regularly applies those to a separate network map.
* Threads calcualte route proposals as asked.
Simple. Accept that a route may become invalid during an update - just reschedule a reroute.

The requirement that a route is "valid" is comical - and I do not mean this as insult. Ever used a navigation system with trafffic information for a longer trip? Routes as planned turn invalid quite frequently. Traffic jams develop, get removed, the navi will pull up changes.

Accept that longer routes are "Proposals to be reevaluated later" and suddenly - using threads is possible. And allows to use much more cpu consuming algos.

This is why I put up trains and "biter strategiy" - not individual biters - as examples. Train networks do NOT change brutally between updates (yes, you may drop a piece of train track, but that will not change the network in most cases and if it does - gues what, not all trains got the update, happens in real life. The map structure also does not change brutally between updates. Small updates - are ignored until the entity asks for a new route.

Great job on the rest, btw - look forward to .16.

[Keks]

after having slept over the initial excitement I do have a small request could we get presets for the Buffer chests?
There would be 3 choices: default 1, 2 and 3 + individual requests.
We set the desired amounts then save as default 1, in the future any buffer chest placed down would start with the default 1 setting (for example yellow and red Belt stuff, electrical poles and some raw Resources) then you could add individual items (the oil area would have refineries + Chemical plants , buffers near the main bus would have additional belts etc).

If you want to upgrade your standard eg. you don't want blue factories anymore, you don't have to walk up to every buffer chest and change it manually you simply change the default setting.
If you mainly want individual buffers you leave default 1 empty and only set individual requests (you can still use default 2 and 3 for quickly setting up buffers for frequently used items).

It would be far more useful in a heavily modded game where there are lots of tiers to everything then to a vanilla game but you have a track record of treating even excessive Idiocy (MMO Factorio ) very seriously.

btw. I very much enjoy playing Factorio with 200+ people. the reason I call it excessive idiocy is cause when I bought Factorio way back I didn't even think there would ever be a MP. I thought I had found a great game that's easily worth twice the asking price.
Then! I found the Mods,... and look where we are now.

I have my hopes high cause you never disappoint and always exceed.

[kpreid]

If I understand the described behavior correctly, then buffer chests will not actually work well for the last use case of "dedicating part of your storage for specific things", because they make requests, so they will fill up as fast as possible from PP chests and won't take random items.

Example: Right now in a couple maps, my main area I have trains coming in with stone. The stone is unloaded into passive providers as well as belts because it has small uses that logistic robots are good for.

I also sometimes use miners going to active provider chests to remove tiny little deposits that are in areas I want to build in. Or, I go out and deconstruct a depleted mine and pick up some stone from splitters, or from mining rocks that are in the way of new construction. Any which way, I end up with stone in my inventory.

If I were to set up dedicated storage what I would want is that the stone (or coal, this happens a lot with coal too) that gets put into the logistic network from active providers or character trash slots goes to a specific area of storage. But the new buffer chests will immediately fill up with stone from the passive providers, so they won't be taking the miscellaneously added stone after they're full. So they don't look like an organization solution unless you also avoid passive providers or use conditions to limit input to your PP chests (which is not great because it means you have to update the numbers as your base scales up).

Just to be clear, the new buffer chests will be great for improving response times when doing construction or defense repair, and I don't think they need to also be a storage organization solution.

[Muche]

* Main thread posts "network update" messages into a queue.
* Separate thread regularly applies those to a separate network map.
* Threads calcualte route proposals as asked.
Simple. Accept that a route may become invalid during an update - just reschedule a reroute.
...
Train networks do NOT change brutally between updates (yes, you may drop a piece of train track, but that will not change the network in most cases and if it does - gues what, not all trains got the update, happens in real life. The map structure also does not change brutally between updates. Small updates - are ignored until the entity asks for a new route.

Let's imagine the following scenario:
Bots are removing/placing rails which changes the rail network causing rerouting.
On your beefy 8-core machine the recalculation happens immediately, a train repaths and goes his merry way. A bot places another rail piece.
On your friend's lowly 2-core machine, the recalculation happens in the following tick, during which the bot places the rail piece. The train repaths and chooses different path (using the newly placed rail).
Congrats, you now have a desync.

The difference from real life is that all players are mere clients logged into the universe server which is simulating the reality, whereas in Factorio all players simulate their own universe.

[Serenity]

So am I seeing this right that this would finally allow simple recycling of intermediate items (like yellow and red belts)? You set the buffer chest to request an extremely large amount of items, so that it gets basically everything. And then you set the inserter emptying the assembler to only work when it's relatively empty. But at the same time the chest also acts as a provider.

[Boardy]

As cool as the buffer chest sounds, I've always felt that what the game needs is a passive requester chest.

To keep the place tidy and the storage chests empty, I like to recycle as much as possible. Deconstructing a full belt will just fill up the storage chests (with coal, for example). I always like to put requester chests at certain spots to bring these items back into the system. This works well as long as there are no provider chests with those items (creating loops).

Another example is when upgrading red belt to blue belt. This can result in tons of red belt in storage, while the red belt factory is straining to push out as many new red belts for the blue belt factory, stealing the much wanted gears in the process.

A passive requester would only recieve items from the storage chests (or brought direct from things being deconstructed).

[pleegwat]

Let's imagine the following scenario:
Bots are removing/placing rails which changes the rail network causing rerouting.
On your beefy 8-core machine the recalculation happens immediately, a train repaths and goes his merry way. A bot places another rail piece.
On your friend's lowly 2-core machine, the recalculation happens in the following tick, during which the bot places the rail piece. The train repaths and chooses different path (using the newly placed rail).
Congrats, you now have a desync.

The difference from real life is that all players are mere clients logged into the universe server which is simulating the reality, whereas in Factorio all players simulate their own universe.

That's easily resolved by defining a delay (say, 6 ticks) after which changes are applied. As long as the recalculation takes less than those 6 ticks, there is no impact.

The larger problem is that you also need to snapshot the rail infrastructure so the routes are calculated on a consistent baseline grid. One solution there would be to batch all rail network changes until after the route calculations have been completed.

Applying the same trick to biter AI is probably significantly more difficult because the amount of state to be snapshotted is larger.

[TheTom]

Let's imagine the following scenario:
Bots are removing/placing rails which changes the rail network causing rerouting.
On your beefy 8-core machine the recalculation happens immediately, a train repaths and goes his merry way. A bot places another rail piece.
On your friend's lowly 2-core machine, the recalculation happens in the following tick, during which the bot places the rail piece. The train repaths and chooses different path (using the newly placed rail).
Congrats, you now have a desync.

The difference from real life is that all players are mere clients logged into the universe server which is simulating the reality, whereas in Factorio all players simulate their own universe.

That's easily resolved by defining a delay (say, 6 ticks) after which changes are applied. As long as the recalculation takes less than those 6 ticks, there is no impact.

The larger problem is that you also need to snapshot the rail infrastructure so the routes are calculated on a consistent baseline grid. One solution there would be to batch all rail network changes until after the route calculations have been completed.

Applying the same trick to biter AI is probably significantly more difficult because the amount of state to be snapshotted is larger.

Or you start using the server as a server. Train route calculations happen on the server, the ROUTES are sent to the clients. Or not - if it syncs train positions that is anyway all that is needed.

[jackaroo]

the buffer chest is an option to filter your storage but why can't we just filter storage chest with a logistic or circuit condition?

[Adil]

So, basically you've split requester chests into "requester chest lvl 1" and "requester chest lvl 2" so that "lvl 2" can pull items from "lvl 1" chests.
Can we also get chest levels 3,4,5... ?

[tolomea]

From the diagram it looks like when players, requestors or construction bots are pulling items they will favor buffer chests over passive providers. I think it would be more useful to have buffers and providers have the same priority so that it pulls from the nearest.

Imagine you are manufacturing ammo somewhere in the middle of your base and putting it in a passive provider and it's distributing from there to requestors all round the perimiter. You notice there is a big latency to this process on one side of your base so you put a buffer over there to cut the latency. If buffers always take priority over providers then the opposite side of your base will now start pulling from the buffer instead of the closer provider which is probably not what you want.

I guess a counter argument to this is if you are using buffers in place of storage chests and so want to clear the storage buffers before pulling from the provider and triggering more construction. This usecase seems unlikely to me, I imagine generally buffers as storage will be for raw resources not manufactured goods.

(obviously active providers and actual storage chests should still take priority, this is just about passive providers vs buffers)