Friday Facts #203 - Logistic buffer chest

[Frightning]

https://www.factorio.com/blog/post/fff-203

Edit: 1st example is also solvable with requestor chests.
2nd example is already solvable with requestor chests (unless you had active providers with repair packs in them; but there's no reason to do that).
3rd example would be solvable with slot filtering, so that only the chosen item can be in a given Storage chest. (Edit 2: Slot filtering for chests would be handy for a lot of other reasons as well tbqh)

My favorite clever use of active v. passive providers was something I did in my 0.12 kilobase for the purpose of automating Steel barrel production for crude oil transport, I wanted an all-bot solution that would add barrels as needed, but not allow for excess barrels to end up in storage. The solution was to use passive providers for filled barrels (so they can never end up in storage), and active providers for emptied barrels (so they would be preferentially taken over newly made barrels, which were also available from a passive provider). This functioned as a logibot analog of mechanical priority (as is frequently done with sideloading), but it can't be extended beyond 1 tier (so no source 1>source 2>source 3, or longer chains of priority). Depending on how these buffer chests are implemented, maybe 2-tier will be possible now, but not beyond that, which can still be a problem (e.g. bot operated fuel priority with Wood, Coal, Solid fuel, and Rocket fuel).

[Alice3173]

Why do they want to add a new chest type? Can't we just use the storage chest with "requests" set for this? As long as nothing is set, it keeps the old behavior, once items are specified (maybe even by circuit network) non matching stuff is cleared and... but i guess having the feature somewhat hidden might be less user friendly than having more options to understand initially.

A more pressing question is: Does them adding a new type of logistics chest somehow negatively impact the game for you? Nobody is forcing you not to use the method you described after all and clearly many people wanted this new chest.

[ili]

You don't really need the Logistic buffer chest you can solve many of this looping problems with the Circuit network like this
viewtopic.php?f=18&t=51904&p=303453#p303453

[cpy]

F5 time! F5F5F5F5F5F5F5F5F5F5F5F5F5F5F5F5F5F5!

Waiting for new FFF is so hard

[Pure Ownage]

Ridiculously excited about the buffer chest!

Speaking of logistics I would like the robots to prioritise delivery over recharging. It's really annoying to see 200 robots turn around one meter from you to go recharge 4 at a time.

[neilon96]

I'm literally sitting here Celebrating the new Chest!

[Engimage]

Actually Buffer chest without any active request can act like storage chest so why not just merge them?

[Klonan]

Actually Buffer chest without any active request can act like storage chest so why not just merge them?

Buffer chest without any request will not accept items, it will act as a passive provider

[Engimage]

Actually Buffer chest without any active request can act like storage chest so why not just merge them?

Buffer chest without any request will not accept items, it will act as a passive provider

Ok, Passive provider it is.
So why not merge with passive provider then If it can 100% replace one?

[Klonan]

Actually Buffer chest without any active request can act like storage chest so why not just merge them?

Buffer chest without any request will not accept items, it will act as a passive provider

Ok, Passive provider it is.
So why not merge with passive provider then If it can 100% replace one?

Then it would not provide the desired functionality

[Engimage]

Actually Buffer chest without any active request can act like storage chest so why not just merge them?

Buffer chest without any request will not accept items, it will act as a passive provider

Ok, Passive provider it is.
So why not merge with passive provider then If it can 100% replace one?

Then it would not provide the desired functionality

Man you are really saving words
But I got it. To fulfill the request buffer chests needs other type of provider.

[jerocom]

That is a great idea, but there is a active provider chest too, and what would do that? (it is not in the scheme)

[Jap2.0]

That is a great idea, but there is a active provider chest too, and what would do that? (it is not in the scheme)

Active providers push all items in them to other chests (with the priority system, of course). So they are used sometimes in, for example, bot-based train unloading systems where they will push ores out (to requester chests, then buffer chests, then storage chests, if I got my priorities right) so the unloader doesn't get clogged.

[Keks]

That is a great idea, but there is a active provider chest too, and what would do that? (it is not in the scheme)

Active providers push all items in them to other chests (with the priority system, of course). So they are used sometimes in, for example, bot-based train unloading systems where they will push ores out (to requester chests, then buffer chests, then storage chests, if I got my priorities right) so the unloader doesn't get clogged.

if there's a filter set to allow that item

not sure if that filter is binary or you can limit it to a certain number though.

[Jap2.0]

That is a great idea, but there is a active provider chest too, and what would do that? (it is not in the scheme)

Active providers push all items in them to other chests (with the priority system, of course). So they are used sometimes in, for example, bot-based train unloading systems where they will push ores out (to requester chests, then buffer chests, then storage chests, if I got my priorities right) so the unloader doesn't get clogged.

if there's a filter set to allow that item

not sure if that filter is binary or you can limit it to a certain number though.

I assume you can filter them to any number, like requester chests, but I may be wrong.

[Klonan]

That is a great idea, but there is a active provider chest too, and what would do that? (it is not in the scheme)

Active providers push all items in them to other chests (with the priority system, of course). So they are used sometimes in, for example, bot-based train unloading systems where they will push ores out (to requester chests, then buffer chests, then storage chests, if I got my priorities right) so the unloader doesn't get clogged.

if there's a filter set to allow that item

not sure if that filter is binary or you can limit it to a certain number though.

I assume you can filter them to any number, like requester chests, but I may be wrong.

That's right, the requesting is identical to requester chests

[PunkSkeleton]

* Main thread posts "network update" messages into a queue.
* Separate thread regularly applies those to a separate network map.
* Threads calcualte route proposals as asked.
Simple. Accept that a route may become invalid during an update - just reschedule a reroute.
...
Train networks do NOT change brutally between updates (yes, you may drop a piece of train track, but that will not change the network in most cases and if it does - gues what, not all trains got the update, happens in real life. The map structure also does not change brutally between updates. Small updates - are ignored until the entity asks for a new route.

Let's imagine the following scenario:
Bots are removing/placing rails which changes the rail network causing rerouting.
On your beefy 8-core machine the recalculation happens immediately, a train repaths and goes his merry way. A bot places another rail piece.
On your friend's lowly 2-core machine, the recalculation happens in the following tick, during which the bot places the rail piece. The train repaths and chooses different path (using the newly placed rail).
Congrats, you now have a desync.

The difference from real life is that all players are mere clients logged into the universe server which is simulating the reality, whereas in Factorio all players simulate their own universe.

You totally miss the point of multi-threaded programming. It's not like: make something happen sometime in the past. It's like: we've got to do some work before next game update, let's divide the work and everyone tell me when it's done. There's really no difference between hardware - the update cycle will finish when all work is done. What he is suggesting is to have a separate object in memory responsible for train network. Update this at the beginning of frame cycle and then push train recalculation to another thread.

If routes become invalid during game updates, then how are train paths kept? How are biter paths cached (this was an optimization you guys did and explained in a previous Friday facts)? Even if routes become invalid at game updates, that just suggests that you should use a different algorithm that doesn't suffer from the problem. I don't see any reason why you can cache all these paths and update them only when the appropriate entities are added or removed, and just keep using the cached path info most of the time.

The problem here is that you have no separate copy of the train network. If you do the update in parallel then the entities will "advance to the next frame" independently of train network update. Apart from possible crashes (get pointer value, try to access memory but the memory has already been released because an entity was destroyed) that would lead to desyncs if on machine A route X changing entity update happens before route X and on machine B it happens after route X calculation. Then you have 2 different train paths and a desync.
Also this seems like a main problem of multi-threading in Factorio - you have a huge list of entities which affect other entities and so on.

[c0bRa]

I really need the bufferr chests in my new factory <3

Please release them

[JohnyDL]

Okay I think I've got this so for request priority the most important to least important is

Player/Requester chests (closest)
Buffer chests
Storage chests

for provider priority

Player (Trash)/Active providers (closest)
Storage Chests
Buffer/Passive providers (closest)

This makes me think there's going to be a few odd moments

say for example an expandable solar array is part of my base and due to bad planning it sits between solar production and rockets the rocket will grab resources I'd wanted to be on standby for the solar array and it has to request more from the place they're produced, I suppose this does the job but depending on setup that could cause 2 or 3 bots to do what had been the job of one bot.

another example is a centrally located buffer area for things the player might want access to by hand or by bot if it's between production of something and a requester/assembler that uses that same something the buffer chest might not remain full of the requested items

Recycling seems to be made easier by buffer chests but there might be problems with it here's my suggestion keep the request priority the same but tweak provider priority giving different orders to construction, player requests and logistics and also allow a little buffer to buffer action

The buffer to buffer action I'm suggesting shouldn't cause loops because it's not every item in a buffer chest that's eligible it's only those above the request level and to have a looping effect the providing buffer would have to drop below it's request level while the receiving buffer would have to rise above its request level since the requesting buffer chest only asks for as many as it needs and the providing buffer only provides down to it's request level that shouldn't happen (assuming bots will take only as many as are needed/permitted and not their maximum carry load between buffers). Along side this a more complicated idea might be setting a buffer to request 0 of an item would make it a storage chest for that/those item(s) so it requests all of them it can hold, in this mode it shouldn't request accept anything from other buffers (to prevent looping) and it would output the items like a storage chest to buffers requesters or the player

For logistics I'd suggest
Player (Trash)/Active providers (closest)
Storage Chests/Buffer (excess items even to other buffers) (closest)
Passive providers
Buffer (items that have been requested)

This would allow the buffer chest to be a true buffer within the logistics system and for items only to go out into the logistics world from the buffer as a last resort generally increasing the buffer chest's logistical power (though not without providing interesting complexities to deal with I'm sure) and reducing the number of logistics bots doubling up on doing the same job moving by preferring provider requester routing for items over provider buffer requester routing.

For construction/player
Active providers
Storage
Buffer/Passive providers (closest)

This is because the intent of buffers seems to be to move items closer to where they're needed for players' benefit and construction so prioritising based on chest type for them would actually reduce their usefulness in this capacity, the closest item that does the job is the best item to do the job in these cases but still with the intent that the storage chests and active providers should be emptied and that the player trash is an undesirable target to get items from.

[seePyou]

"The memory transfer speed itself is not that slow, but the waiting (latency) time between ordering and receiving it is. "

Doesn't that describe the memory value "CAS Latency" ? Or am I wrong? If I'm not wrong, why have memory modules gone to such a high CAS latency value in this day and age? I am a senior in the world of PC and gaming, so I do remember the times when memory modules with 4 CAS latency were prized! Now they are non-existent.

What happened?