Working with palleting mods

[Omnifarious]

I'm using Deadlock's Crating Machine, and with that mod (and other paletting mods) you end up with these pallet items you have to return to the provider in some way. My approach before LTN has been to always unload a train and simultaneously load the appropriate number of empty pallet items onto the train. And, conversely, at the provider station, load a train while simultaneously unloading all the empty pallet items.

Now, if a stop is simultaneously asking for x number of filled pallets and advertising the availability of x number of empty pallets, and another stop is doing the reverse, LTN will schedule two trains to handle this. This isn't really what I want.

If I rethink things and instead say "An empty train on this network has every cargo train full of the appropriate number of empty pallets". That can work. Except that LTN won't do anything to enforce this. I have to carefully set up stop conditions to load up just the right number of empty pallets into the train, and in some cases there is a risk that the train might be slightly over or underfilled with empty pallets. For my non-LTN stations I handle this by having one of the train conditions for leaving a requester station being that it contains exactly the correct number of empty pallets, and that works. But there is no way to make LTN do this.

The problem is, how to handle this flexibly.

One idea is to have LTN record the contents of a train when it creates a schedule for a train at a depot and have a requirement that the train return to that state at the requester station before it leaves. This behavior could be triggered by turning on bit 31 of the 'this is a depot' signal.

It would be the player's responsibility to make sure that all trains participating in a network that had a particular idea of what 'empty train' meant all returned to depots that had this bit set.

The reason I'm considering LTN at all is that I was previously using a 'all stations that need the same product have the same name' strategy, and only enabling a station when it had a need for the product. But this would result in multiple trains making their way towards a station when it enabled only to have one train abort halfway through and return to the originating station. This was much less than desirable. It wasted a lot of fuel, clogged train networks unnecessarily, and required that I have refueling stations at far-flung mines.

[Optera]

I heard that request before for barrels.
Chained deliveries are too much work for serving just one edge case.

btw why transport crated items in trains? Didn't Deadlock balance it so crated items take the same space in chests/wagons as non crated items?

[Omnifarious]

I heard that request before for barrels.
Chained deliveries are too much work for serving just one edge case.

Well, I'm not requesting chained deliveries because that does seem like a pain, and also hard to implement in any reasonable fashion. What I am requesting is that the depot record the train state in the depot as 'empty' and put in train schedule requirements at the requester station that the train return to the 'empty' state before it leaves the station. That doesn't seem like it would be that hard, and it would solve the issue for people using barrels as well.

btw why transport crated items in trains? Didn't Deadlock balance it so crated items take the same space in chests/wagons as non crated items?

The crating machines use very large amounts of electricity. It's good to have as few of them as possible. Crating at mines is dubious, but in almost all other cases, it makes the most sense to crate products when they're produced, not just before they're consumed.

Additionally, crated products can be loaded five crates (equivalent to a full stack of items) at a time, and unloaded at the same speed.

Also, the main reason I want to use crates is that I really hate having giant balancers and tons of belts and all the other bric-a-brac that comes with having individual items on belts. Having some gigantic balancer mess at every train stop just to load items into crates so they can move a short way and then be uncrated seems to defeat the purpose.

Edit: Yes, looking at the code, this shouldn't be too hard if reading train contents isn't hard. I'll have to look at the API to see how possible this is. You NewScheduleRecord function is a little convoluted, but I suspect it can handle this as well, maybe with a little tweaking. I suspect though that your special case for fluid wagons would mean that a mixed pallet/fluid train might not work. And from a game mechanics perspective, it may not be possible for it to work.

[Optera]

I don't get what you ask for.

What I am requesting is that the depot record the train state in the depot as 'empty' and put in train schedule requirements at the requester station that the train return to the 'empty' state before it leaves the station.

Trains return to the depot when they finished their delivery. At this point anything left in trains is considered residue and should be cleaned out at the depot to ensure providers on the next delivery can load the train to its capacity.

Reading inventories is a fairly slow function. The only time train inventories are read by LTN is when a train leaves a provider to update delivery counts.

Additionally, crated products can be loaded five crates (equivalent to a full stack of items) at a time, and unloaded at the same speed.

Items stacking 100+ or more into one crate, will certainly see an increase in loading speed. I never saw loading as bottleneck though, 12 fully upgraded stack inserter unload 8000 green circuits in a few seconds into chests.

Also, the main reason I want to use crates is that I really hate having giant balancers and tons of belts and all the other bric-a-brac that comes with having individual items on belts. Having some gigantic balancer mess at every train stop just to load items into crates so they can move a short way and then be uncrated seems to defeat the purpose.

From my tests I found that for 3 belt upwards Miniloader and wide chest are the most ups friendly belt balancer. viewtopic.php?f=6&t=61381&p=371056#p371040

[Omnifarious]

Well, I have the code written already. It now works and serves my purposes well. I issued a pull request, though if I were to work harder on it, I would make it a lot nicer and handle more edge cases (like fluid wagons) in a reasonable fashion. It will only really work for trains that consist entirely of cargo wagons.

[mrvn]

I heard that request before for barrels.
Chained deliveries are too much work for serving just one edge case.

Well, I'm not requesting chained deliveries because that does seem like a pain, and also hard to implement in any reasonable fashion. What I am requesting is that the depot record the train state in the depot as 'empty' and put in train schedule requirements at the requester station that the train return to the 'empty' state before it leaves the station. That doesn't seem like it would be that hard, and it would solve the issue for people using barrels as well.

Well, I am.

What would be so hard with adding a chained-provider-limit. Then when a train is scheduled to arrive somewhere that provides more than chained-provider-limit items check if anyone requests those and add another stop to the schedule. Repeat till no more items are chain provided or nobody requests them.

It's not just one edge case anymore. You have barrels and pallets or wooden chests for crating and the same could be used for every production line. E.g. bring iron plates to the gear wheel factory, take gears and deliver to electronic circuit factory, take electronic circuits and deliver to advanced electronic circuit factory, ... The code would be a simple recursive call and stop when it gets too hard to figure out a follow up stop. But the schedules this could come up with would be so much more efficient and much more fun to watch.

[Omnifarious]

What would be so hard with adding a chained-provider-limit. Then when a train is scheduled to arrive somewhere that provides more than chained-provider-limit items check if anyone requests those and add another stop to the schedule. Repeat till no more items are chain provided or nobody requests them.

It's not just one edge case anymore. You have barrels and pallets or wooden chests for crating and the same could be used for every production line. E.g. bring iron plates to the gear wheel factory, take gears and deliver to electronic circuit factory, take electronic circuits and deliver to advanced electronic circuit factory, ... The code would be a simple recursive call and stop when it gets too hard to figure out a follow up stop. But the schedules this could come up with would be so much more efficient and much more fun to watch.

I'm looking at how this works, and thinking through your problem...

So, on the surface, it seems simple. But there are some interesting problems that can crop up here.

The first issue is timing. Generally, for example, a delivery is scheduled and a train schedule constructed immediately upon sufficient demand or supply being available to fulfill the request. It seems unlikely that a chain schedule could be constructed then. All deliveries involved would have to become possible on the same tick.

You could keep track of 'trains in flight', and try to update the schedule of a train that's going to soon arrive at a requester stop that now also has sufficient material to supply somewhere else. But there are still tricky issues of timing there as well. Also, train schedules are typically not updated on a running train. How is the game going to handle that? In all likelihood it's going to stop accelerating while the game constructs a new path which will slow things down.

Another solution would be to have separate thresholds for convincing a new train to leave the depot vs. a chain delivery. But that adds a lot of complexity to an already complex mod. People already have a really hard time wrapping their heads around the idea that the network id is a bit set instead of a single integer, or understanding the bit mask representation of carriage types.

And then, the next level of difficulties present themselves. There might be several possible branches to a chain. How to pick which branch? Suddenly you're in the position of possibly having to solve the travelling salesman problem. And the mod simply can't get enough information to even make heuristic decisions that don't result in significantly sub-optimal paths. If I'm not mistaken, it uses the 'crow-flies' distance between things to make a decision about even the simple schedules it now creates. The proper metric would be the train pathing cost, but that metric isn't available to mod authors, as far as I can tell. And I don't know that it would be a good idea to use anyway. The train pathing code is already a significant CPU sink. Running speculative path calculations on the behalf of mods doesn't really seem like a good idea. Additionally, that cost is dynamic as the system takes into account all the current train paths and the value of all the different train signals.

Grappling with even just the simultaneity problem with regards to even simple "provider and requester trade items one for one" type schedules was worrying me. I would've had to make sure that a request and supply offer reached the threshold on the same tick for it to even be at all reasonable that the mod would generate a 'trade' schedule rather than a regular 'move items from A to B' schedule.

[Optera]

It's all a matter of station design.
I got LTN 1.7.13 to ship full barrels between provider and requester and the same amount of empty barrels back to provider through depot.

Download down here: viewtopic.php?f=214&t=61818#p374114

[Omnifarious]

It's all a matter of station design.
I got LTN 1.7.13 to ship full barrels between provider and requester and the same amount of empty barrels back to provider through depot.

Yes, using two trains, which seems wasteful and unnecessary in addition to having a time delay involved in which a station won't have empty barrels to fill up with liquid until the empty barrel delivery arrives. It will work, and I might've chosen to run the network that way until I had the 'in this network, empty means having the correct quantity of empty crates/barrels/pallets on hand' idea.

[Optera]

Yes, using two trains, which seems wasteful and unnecessary in addition to having a time delay involved in which a station won't have empty barrels to fill up with liquid until the empty barrel delivery arrives. It will work, and I might've chosen to run the network that way until I had the 'in this network, empty means having the correct quantity of empty crates/barrels/pallets on hand' idea.

If you had bothered checking the save you'd find it running on one train doing all shipping in a single loop.
This does all you wanted it to do without changing anything in LTN.

[Omnifarious]

If you had bothered checking the save you'd find it running on one train doing all shipping in a single loop.
This does all you wanted it to do without changing anything in LTN.

I'm sorry. I missed the save. :-/

Yes, it works.

I wasn't able to decipher all the circuit logic, but I've tried setups like this without LTN, and it turned out that every once in awhile the inserters would overfill with empties (which can't happen with barrels because full and and empty barrels stack the same). My solution to that was to have an inserter that would remove the empties if the train was overfilled. But, without a condition asserting that the train have exactly the right number of empties, the train would often leave before that inserter got the chance to remove the extras.

Oddly, it turned out that once I added the condition about having the exact right number, the inserter that removed any extras was never activated again..

And yes, I dynamically reduced the stack size on inserters to try to prevent overfilling. But, while that greatly reduced the problem, it still didn't completely go away.

So, that experience is why I wasn't willing to just trust to fate and let the train return to the depot full of empties without a condition on how many empties there should be before the train left the depot.

[Optera]

I tuned it a bit.

• Now two L-3C constantly roam around
• Depots top off trains with empty barrels
• Provider only unloads as many empty barrels as it will load filled barrels
• All stack inserters have been set to stack size of 10 ensuring a even loading even with barrel stack sizes > 10

With these changes the amount of barrels each station has shouldn't change.

I'll update the design thread with these two stations with explanations what the circuitry parts do.

[mrvn]

What would be so hard with adding a chained-provider-limit. Then when a train is scheduled to arrive somewhere that provides more than chained-provider-limit items check if anyone requests those and add another stop to the schedule. Repeat till no more items are chain provided or nobody requests them.

It's not just one edge case anymore. You have barrels and pallets or wooden chests for crating and the same could be used for every production line. E.g. bring iron plates to the gear wheel factory, take gears and deliver to electronic circuit factory, take electronic circuits and deliver to advanced electronic circuit factory, ... The code would be a simple recursive call and stop when it gets too hard to figure out a follow up stop. But the schedules this could come up with would be so much more efficient and much more fun to watch.

I'm looking at how this works, and thinking through your problem...

So, on the surface, it seems simple. But there are some interesting problems that can crop up here.

The first issue is timing. Generally, for example, a delivery is scheduled and a train schedule constructed immediately upon sufficient demand or supply being available to fulfill the request. It seems unlikely that a chain schedule could be constructed then. All deliveries involved would have to become possible on the same tick.

I realized that already. That's why I called it chained-provider-limit. You need the extra limit so a lower threshold is used for chain deliveries and requesting new trains. Otherwise, as you say, it would have to be the same tick to work. You would e.g. configure your station to request 1000 iron plates, 2000 copper plates, chain provide 1000 electronic circuits and provide 2000 electronic circuits. So if 1000 electonic circuits have been build then the next train coming in will chain deliver them to the next station. And if 2000 electronic circuits have collected then a separate train will be scheduled.

There could also be a chain-requester-limit. Similar idea. If a train is going to a provider station anyway then let it go an extra hop to deliver an amount you wouldn't normally waste a train on.

You could keep track of 'trains in flight', and try to update the schedule of a train that's going to soon arrive at a requester stop that now also has sufficient material to supply somewhere else. But there are still tricky issues of timing there as well. Also, train schedules are typically not updated on a running train. How is the game going to handle that? In all likelihood it's going to stop accelerating while the game constructs a new path which will slow things down.

Updating train schedules in the GUI while driving has no effect as long as you don't change the station it is driving too. Might be harmless to do it through scripts. And you can always change the schedule of the train when it arrives/sits at the unloading station.

Another solution would be to have separate thresholds for convincing a new train to leave the depot vs. a chain delivery. But that adds a lot of complexity to an already complex mod. People already have a really hard time wrapping their heads around the idea that the network id is a bit set instead of a single integer, or understanding the bit mask representation of carriage types.

And then, the next level of difficulties present themselves. There might be several possible branches to a chain. How to pick which branch? Suddenly you're in the position of possibly having to solve the travelling salesman problem. And the mod simply can't get enough information to even make heuristic decisions that don't result in significantly sub-optimal paths. If I'm not mistaken, it uses the 'crow-flies' distance between things to make a decision about even the simple schedules it now creates. The proper metric would be the train pathing cost, but that metric isn't available to mod authors, as far as I can tell. And I don't know that it would be a good idea to use anyway. The train pathing code is already a significant CPU sink. Running speculative path calculations on the behalf of mods doesn't really seem like a good idea. Additionally, that cost is dynamic as the system takes into account all the current train paths and the value of all the different train signals.

LTN sorts requests by priority and round-robin. Afaik only train selection and possibly provider stations use the 'crow-flies' distance, right? Both the train and the provider would be determined already. So it's simply a matter of picking the requester. Anyway, whatever logic LTN uses now to pick the stations can be adapted to picking a requester for a fixed train and fixed provider. It should actually be simpler since the search space goes from 3 dimensions to 1.

The way I suggested it it wouldn't become a traveling salesmen problem. Each hop in the chain is decided on it's own and simply as a target of opportunity. "I'm going to X anyway, can I go somewhere else after that too?" There is no optimization going on to find the best 10 hop schedule or something. Anything like that would become computational impossible for even just 3 or 4 hops.

Grappling with even just the simultaneity problem with regards to even simple "provider and requester trade items one for one" type schedules was worrying me. I would've had to make sure that a request and supply offer reached the threshold on the same tick for it to even be at all reasonable that the mod would generate a 'trade' schedule rather than a regular 'move items from A to B' schedule.

Actually going back to your point about modifying schedules in flight. If you allow modifying schedules of trains in the station than this could be even simpler to implement.

When a train arrives and unloads the factory will start producing goods. If it then hits the provider limit (or someone hits the requester limit) while the train is in the station then LTN can modify the trains schedule instead of requesting a new train. For that the modification should be small. LTN picks a requester and provider. It then checks if the provider has a train present that has the depot as next stop. If so use that train instead of finding a depot.


You mentioned network IDs. Here is another wrinkle for chaining deliveries. The first delivery might have network ID 1 and the second network ID 2. Does the train remember the network ID of the depot it came from? Chained deliveries might not be allowed because trains from different network IDs are supposed to be used. Didn't think about that before.