Feature Request: Relative Request Threshold Signal

[jaakon]

Can we get an additional "Request Threshold" Signal, which sets the amount of missing items in percent?
If we have a station requesting different items in vastly different quantities, it's always a problem (at least for me?) to set the correct threshold amount. Setting it too high and one resource may now be delivered in time or at all (threshold >= request). Setting it low enough for this resource may result in half filled trains for other requested items in the same station, especially if you consider the different stack sizes.

If we could set the threshold to e.g. 50%, if would work smoothly for a high quantity item with big stack size and a low quantity item with low stack size in the same station.

Or is there another solution to this problem?

[Optera]

Would certainly make some multi provide or request designs simpler, but it's just not feasible to implement in any manner I could think of.

Thresholds as absolute values don't need a point of reference and can be quickly calculated by comparing current value and threshold.
To calculate 50% LTN would require a reference point to an absolute value in percentage meaning I'd have to have 2 inputs for e.g. iron ore, one as current value and one to know how much iron ore is 100%.

1st Option:
Feeding current values through green wire and 100% value through red wire into the lamp.
This would not only break all existing builds, but also would be quite a change from how LTN sees wires as to how they are for any other circuit connected entity.

2nd Option:
Another option would be to have a dedicated 100% virtual signal for every possible item and fluid in the game.
That's a huge number of additional virtual signals. With Bobs and Angels potentially more than Factorio can handle.

3rd Option:
Add a 4th entity for the sole purpose of reading 100% signals to LTN Stops.
Connecting to one of LTN stops 3 entities is already hard and confusing to players.
Scanning for signals is currently faster than ever, but polling still still takes 58 out of the 60 ticks.

[jaakon]

Bummer. But at least I made it on the "rejected features" list

[DaleStan]

Idea. Possibly a completely bogus idea, but idea nonetheless.

I don't know how often LTN scans the stops to create deliveries, but I'm going to assume for the purposes of this idea that it happens once a second. At each requester stop, put the cargos into an appropriate number of groups. For this example, I'm going to use three groups, "high demand" (request threshold 10000), "medium demand" (request threshold 1000), and "low demand" (request threshold 100). Add a clock near the stop that counts from 1 to 180 (3 seconds), and two decider combinators. Pretend for the moment that you have three LTN stops, not just one. Wire the low demand cargos directly to the light for the existing LTN stop. Wire the medium demand cargos to the input of one of the two deciders, exactly as if that were the light for the medium demand stop we're temporarily pretending you have. Do the same for the high-demand cargos and the other decider. Also wire the clock to the input of both deciders. Set the medium demand decider to output everything if the clock signal is greater than 60, and the high-demand decider to output everything if the clock signal is greater than 120. Connect the output of both deciders to the light on the LTN stop.

Now do some minor trickery. Set the threshold that's input to the medium-demand decider to 900 (1000-100), and set the threshold input to the high-demand decider to 9000 (10000-1000). Now, for one second out of every three, only the low-demand cargos will be requested at the stop, along with a threshold of 100. The next second, the low and medium demand cargos will be requested, with a request threshold of 100+900=1000. And the third second, all cargos will be requested, with a threshold of 100+900+9000=10000.

I could imagine this trickery messing up the "Expected train contents" outputs from the stop, but I'd just go with "expected train contents: 0", regardless of whether or not the expected contents output is correct.

[Optera]

I never tested how ltn would react to constantly changing item and threshold signals. It should work but may as well summon the Kraken or an ancient evil.

A sturdier approach would be to use circuit logic division to create steps in desired delivery sizes.
For example 10k delivery size regardless of threshold can be done with 2 arithmetic deciders:
iron plate / 10k = iron plate >> iron plate * 10k = iron plate >> LTN

For a building store it may be better to first group it through a whitelist filter and use each/size*size after the filter.

[redis]

LTN is a great system, but here is a seemingly normal use case scenario where it is terrible and could possibly be addressed in future. The problem is with shared requesters.

Setup: Multiple providers of different items (providers setup don't matter). Lets use iron plates and speed module as an example. We have one shared requester station requesting those items. Here is the problem:

These items have drastically different volumes and it seems impossible to request appropriate volumes for them into 1 station. For example, you may want to send(request) 1 wagon train for speed modules and 5 wagon for iron. Because the station threshold setting is the same for both of these items you end up either having to low orders for iron with almost empty trains. Or you end up keeping huge volume of speed modules and it may dispatch several one wagon trains once the level drops below threshold. I have tried all possible combinator contraptions to bypass this problem, but it is not possible if the threshold is fixed and the same for all item types.

The addition of STACK threshold addressed exactly this problem, but only for stack variability. This could be solved by giving the threshold in PERCENTAGE terms of the amount set for each item on the constant combinator. If you set -16k for iron and -2k for modules and set your threshold at 50% then it would request at -8k and -1k respectively giving you exactly behavior you want.

Thanks.

[Optera]

moved to correct topic

[redis]

How about this idea?

This would not be as complicated as having to track separate signals for each item, but would still approach solving the issue looks like a lot of people trying to figure out. What if it was possible to have much simpler request with always "full cargo inventory" condition instead of the exact item difference. It would allow to have the shared threshold low for low volume items, but trains would also not dispatch half empty for large volume items. This would effectively be implementation of 50% threshold relative to the 2x train size, which is basically what is needed. This could be a setting in the mod or a true/false signal on requesters since requesters are responsible for managing buffers.

Edit: If interested I could try doing it and send a pull request for review (programmer here).

Edit#2: After more digging I found that the setting "Finish Loading" attaches Inactivity 2s allowing to fully load a train. Bingo! However, I find this is very suboptimal to have such convoluted condition, wait needlessly 2s at both stations and execute a bunch of other code checks. Instead I added a few lines to NewScheduleRecord at the beginning of the first if block (possibly can be done better) and update the count of the order to the size of the train )so it does not send more trains)

Code: Select all

 if finish_loading then 
	if itype == "fluid" then	
		loadingList[1].count = trainInventorySize
	else
    		loadingList[1].count = game.item_prototypes[iname].stack_size * trainInventorySize
	end	
  end
....
elseif finish_loading and condType == "item_count" then 
	if condComp == "≥" then
		record.wait_conditions[#record.wait_conditions+1] = {type = "full", compare_type = "and"}
	elseif condComp == "=" then
		record.wait_conditions[#record.wait_conditions+1] = {type = "empty", compare_type = "and"}
	end  
		

I know there is a difference between inactivty and full/empty conditions so I would recommend to add a config option to allow to generate simple full/empty schedules (instead of me hacking "finish loading" flag). I am sure many will appreciate this addition.

[Optera]

Full load will break merged deliveries and is not going to happen.
"Finish loading" prevents stuck inserters. 2s inactivity is required for stack inserters loading from belts.
Either can already be bypassed by turning circuit conditions on.

If you read a bit further, you'd see LTN already updates deliveries with train inventory when trains leave a provider.

PS: I consider LTN feature complete and will reject any pull request that isn't fully tested to work with all features of LTN.

[redis]

Full load will break merged deliveries and is not going to happen.
"Finish loading" prevents stuck inserters. 2s inactivity is required for stack inserters loading from belts.
Either can already be bypassed by turning circuit conditions on.

If you read a bit further, you'd see LTN already updates deliveries with train inventory when trains leave a provider.

PS: I consider LTN feature complete and will reject any pull request that isn't fully tested to work with all features of LTN.

Merged deliveries are irrelevant if going for full train delivery so should just skip that section in that case (which is actually potentially cpu heavy) and it will not break. Stuck inserters are also only relevant only if you have multiple items in the train or multi item providers. This is an alternative and much simpler mode of requests generation. LTN has a problem with a different size trains use cases and this solves that problem. Of course need to be tested, but please think about this problem.

Edit: I do not see where it actually affects any of existing feature. It just different way to generate requests which can be disabled or enabled.

[redis]

If you read a bit further, you'd see LTN already updates deliveries with train inventory when trains leave a provider.

This may need to be done immediately and not at provider otherwise it can generate another order before the first train even reaches provider. Because if you send for a Full train , but it says still thinks it is fetching your number of threshold items, but you are down 3-4 times that threshold it will send another train(s). So you must update it to match your oncoming delivery. Plus the message print outs are correct.

[redis]

PS: I consider LTN feature complete and will reject any pull request that isn't fully tested to work with all features of LTN.

You seem to be very protective with this statement. So you are against changes or alternative ideas because of strong opinion that you know it better. However you have the mod under not permissive license to modify and if you were to be gone as a resource for community you would take this contribution with you. Me or others now or later would want to create a mod to LTN with a different dispatcher logic for example.

How would you recommend to proceed? Could you expose this via API where you can plugin your own ProcessRequest or may be even consider changing the license? Thanks.

[lois lane balancer]

There's no need for additional code, this is trivial to do right now.
Set no item requests (negative values) in your Constant Combinator. You'll add them via circuit invididually. Set Request Threshold to 1. Yes, 1. You don't need to do this if you've put the default request threshold to 1 in the mod settings (like I did and forgot about, mea culpa there).
Isolate the LTN Station Lamp from the rest of the circuit via a diode (an Arithmetic Combinator set to Each+0 -> Each), with the output going to the lamp.
For each item you want to request, have a Decider subcircuit with logic [ITEM]<= [THRESHOLD] -> [ITEM @ REQUEST QUANTITY]. Have the input for this be on the chest side of the diode and the output going to the lamp. [ITEM @ REQUEST QUANTITY] is of course whatever you would have put in your Constant Combinator, such as -5000 IRON to request 5k Iron Plates.
This will cause the station to only receive the request when the item is below the threshold you set in the relevant combinator, and this scales to any quantity of items. You now have individual request thresholds on a per-item basis, and with just a very simple bit of circuitry.

PS: I consider LTN feature complete and will reject any pull request that isn't fully tested to work with all features of LTN.

You seem to be very protective with this statement. So you are against changes or alternative ideas because of strong opinion that you know it better. However you have the mod under not permissive license to modify and if you were to be gone as a resource for community you would take this contribution with you. Me or others now or later would want to create a mod to LTN with a different dispatcher logic for example.

How would you recommend to proceed? Could you expose this via API where you can plugin your own ProcessRequest or may be even consider changing the license? Thanks.

You're being very aggressive here. If you want to freely modify the mod, write your own version or work with a similar mod that has a license that allows 3rd party modification. That's just how licensed software works. Additionally, Optera has implied they will approve PRs that have been tested.
But the mod works right now, and they don't seem to want to compromise a stable piece of software for a feature that's not critical. So their stance, as I understand it, is "I'm not making changes unless I know those changes are bug-free". Which is a very sensible stance to take when the software works.

[Optera]

However you have the mod under not permissive license to modify and if you were to be gone as a resource for community you would take this contribution with you. Me or others now or later would want to create a mod to LTN with a different dispatcher logic for example.

How would you recommend to proceed? Could you expose this via API where you can plugin your own ProcessRequest or may be even consider changing the license? Thanks.

LTN has been developed and maintained for over 3 years and I don't plan on leaving. (Even when debates about my licensing make me want to.)

I would only pass my mods on to someone who has already proven to be a skilled and reliable developer, otherwise I'd rather take them to my grave than see them butchered into badly maintained forks.

[redis]

There's no need for additional code, this is trivial to do right now.
Set no item requests (negative values) in your Constant Combinator. You'll add them via circuit invididually.
Isolate the LTN Station Lamp from the rest of the circuit via a diode (an Arithmetic Combinator set to Each+0 -> Each), with the output going to the lamp.
For each item you want to request, have a Decider subcircuit with logic [ITEM]<= [THRESHOLD] -> [ITEM @ REQUEST QUANTITY]. Have the input for this be on the chest side of the diode and the output going to the lamp. [ITEM @ REQUEST QUANTITY] is of course whatever you would have put in your Constant Combinator, such as -5000 IRON to request 5k Iron Plates.
This will cause the station to only receive the request when the item is below the threshold you set in the relevant combinator, and this scales to any quantity of items. You now have individual request thresholds on a per-item basis, and with just a very simple bit of circuitry.

PS: I consider LTN feature complete and will reject any pull request that isn't fully tested to work with all features of LTN.

You seem to be very protective with this statement. So you are against changes or alternative ideas because of strong opinion that you know it better. However you have the mod under not permissive license to modify and if you were to be gone as a resource for community you would take this contribution with you. Me or others now or later would want to create a mod to LTN with a different dispatcher logic for example.

How would you recommend to proceed? Could you expose this via API where you can plugin your own ProcessRequest or may be even consider changing the license? Thanks.

You're being very aggressive here. If you want to freely modify the mod, write your own version or work with a similar mod that has a license that allows 3rd party modification. That's just how licensed software works. Additionally, Optera has implied they will approve PRs that have been tested.
But the mod works right now, and they don't seem to want to compromise a stable piece of software for a feature that's not critical. So their stance, as I understand it, is "I'm not making changes unless I know those changes are bug-free". Which is a very sensible stance to take when the software works.

If you do not understand the problem in this thread you can skip it. Now, regarding your "solution". Combinators contraption will not work because the issue is not solvable by sending whatever values of signal into the input. The threshold itself must be different for each item for correct size deliveries. If you are suggesting sending different threshold values this is impossible. You can not send 2 (or more) thresholds at the same time using one value for a threshold. If you try to alternate it will break too (read the code).

This is a problem because it requires storing those relative thresholds and I would agree with Optera here there is just no place for it. This problem with the threshold however exists because how LTN creates orders with precise calculation of amounts for delivery and uses that threshold. So this issue is solvable if the threshold is simply ignored and Full train deliveries are (optionally) allowed. This is the only spot where TSM is better (but not good in many others compared to LTN).

You seem salty because I speak the truth that software license makes it difficult to make improvements, which would benefit others. As a developer myself I understand the author position too. However, my opinion is that after many years LTN has been popular if it was released say under GPL one day the author could not lose any credit anymore (unlike in early stages) for his work because the mod is so well known.

But for now the only way is to convince the developer to look into the problem and implement the code himself or allow a mod for it. None of that is happening here so far. No amount of proving "tested" will be sufficient until the author decides to do it himself. It took ages to convince him to implement network id even though it was clearly a great idea, so I do not see ice moving here much with this "edge case".

[lois lane balancer]

If you do not understand the problem in this thread you can skip it. Now, regarding your "solution". Combinators contraption will not work because the issue is not solvable by sending whatever values of signal into the input. The threshold itself must be different for each item for correct size deliveries. If you are suggesting sending different threshold values this is impossible. You can not send 2 (or more) thresholds at the same time using one value for a threshold. If you try to alternate it will break too (read the code).

This is a problem because it requires storing those relative thresholds and I would agree with Optera here there is just no place for it. This problem with the threshold however exists because how LTN creates orders with precise calculation of amounts for delivery and uses that threshold. So this issue is solvable if the threshold is simply ignored and Full train deliveries are (optionally) allowed. This is the only spot where TSM is better (but not good in many others compared to LTN).

If you're a "developer" you should understand the importance of clear problem statements. If I don't understand the problem, then you and the OP are the ones to blame for shoddy problem statements.
As I understand the problem statement, your problem is that the desired behavior of

if ITEM1 <= THRESHOLD1 then REQUEST QUANTITY OF ITEM1 else REQUEST NO ITEM1
if ITEM2 <= THRESHOLD2 then REQUEST QUANTITY OF ITEM2 else REQUEST NO ITEM2

is impossible due to only having one request threshold signal, so THRESHOLD1 and THRESHOLD2 cannot be different values? If that's wrong, correct me now.

My described circuitry does exactly that. If your stock of ITEM1 is below the THRESHOLD1 you plugged into the relevant Decider Combinator, the subcircuit sends the REQUEST QUANTITY OF ITEM1 to the lamp. Otherwise the station isn't requesting ITEM1. Repeat with a different subcircuit for each other item. No mucking around with the request threshold signal, we're manually implementing threshold behavior in circuit instead. Which you don't seem to understand. Should I be saying "If you don't understand the solution in this thread you can skip it"? In fact, given how you're making such bold claims about impossibility, prove it by giving a failure case my described circuitry can't handle.

[redis]

However you have the mod under not permissive license to modify and if you were to be gone as a resource for community you would take this contribution with you. Me or others now or later would want to create a mod to LTN with a different dispatcher logic for example.

How would you recommend to proceed? Could you expose this via API where you can plugin your own ProcessRequest or may be even consider changing the license? Thanks.

LTN has been developed and maintained for over 3 years and I don't plan on leaving. (Even when debates about my licensing make me want to.)

I would only pass my mods on to someone who has already proven to be a skilled and reliable developer, otherwise I'd rather take them to my grave than see them butchered into badly maintained forks.

Ok well I guess you are here for us until your apprentice shows up who will be a new jedi master. Now to business. I am sure you understand the issue here and why full train deliveries can solve it. What are you arguments that it is bad for LTN? Why not consider this mode of operation for dispatcher? Here are my for points.

1. Solves the problem of different sizes for trains at requestors and item volume management. Allows full trains to run deliveries in you factory, which improves efficiency for many types of builds.
2. Makes scheduling simpler with much fewer calculations required.
3. Gives more freedom and flexibility to manage volumes at your requestors.
4. This is relatively easy to do with your clean code and safe feature because it is just a new version of scheduler with minimal change. There is nothing to break or conflict with if the entire scheduler operation is different. A user can have an option to choose one of two. Standard for ant style factory with merge deliveries and fancy things and the full deliveries for the blunt TSM style.
Thoughts?

[redis]

If you do not understand the problem in this thread you can skip it. Now, regarding your "solution". Combinators contraption will not work because the issue is not solvable by sending whatever values of signal into the input. The threshold itself must be different for each item for correct size deliveries. If you are suggesting sending different threshold values this is impossible. You can not send 2 (or more) thresholds at the same time using one value for a threshold. If you try to alternate it will break too (read the code).

This is a problem because it requires storing those relative thresholds and I would agree with Optera here there is just no place for it. This problem with the threshold however exists because how LTN creates orders with precise calculation of amounts for delivery and uses that threshold. So this issue is solvable if the threshold is simply ignored and Full train deliveries are (optionally) allowed. This is the only spot where TSM is better (but not good in many others compared to LTN).

If you're a "developer" you should understand the importance of clear problem statements. If I don't understand the problem, then you and the OP are the ones to blame for shoddy problem statements.
As I understand the problem statement, your problem is that the desired behavior of

if ITEM1 <= THRESHOLD1 then REQUEST QUANTITY OF ITEM1 else REQUEST NO ITEM1
if ITEM2 <= THRESHOLD2 then REQUEST QUANTITY OF ITEM2 else REQUEST NO ITEM2

is impossible due to only having one request threshold signal, so THRESHOLD1 and THRESHOLD2 cannot be different values? If that's wrong, correct me now.

My described circuitry does exactly that. If your stock of ITEM1 is below the THRESHOLD1 you plugged into the relevant Decider Combinator, the subcircuit sends the REQUEST QUANTITY OF ITEM1 to the lamp. Otherwise the station isn't requesting ITEM1. Repeat with a different subcircuit for each other item. No mucking around with the request threshold signal, we're manually implementing threshold behavior in circuit instead. Which you don't seem to understand. Should I be saying "If you don't understand the solution in this thread you can skip it"? In fact, given how you're making such bold claims about impossibility, prove it by giving a failure case my described circuitry can't handle.

No your circuitry does not do that because there is only 1 threshold in existence on the combinator. There is also its variation called "Stack threshold".

"No mucking around with the request threshold signal"
This is where you do not understand. You can not change the threshold by manipulating the input for that level. The threshold number itself is used in the calculation of delivery size (in the code). Without changing it you are not able to create correct delivery size per item. This is basic math here. Optera understands the problem and the original poster too. If you think you are smarter than all then post your blueprint contraptions.

[lois lane balancer]

Small edit to my first post, forgot a part of the solution was in my mod configuration settings and thus needs to be explicitly stated.

No your circuitry does not do that because there is only 1 threshold in existence on the combinator. There is also its variation called "Stack threshold".

"No mucking around with the request threshold signal"
This is where you do not understand. You can not change the threshold by manipulating the input for that level. The threshold number itself is used in the calculation of delivery size (in the code). Without changing it you are not able to create correct delivery size per item. This is basic math here. Optera understands the problem and the original poster too. If you think you are smarter than all then post your blueprint contraptions.

My circuitry doesn't need more than 1 threshold signal. Reread the post.
Also answer these questions, they should help you get closer to the answer:
1: If the request threshold is 0 and all item signals are positive, how much of what item(s) does LTN send to the station?
2: If the request threshold is 0, the signal from a chest with N, where N is less than 5000, of an item is sent to the lamp's network and a signal of -5000 of an item is also sent to the lamp's network, what is the resultant item signal in the lamp's network? How much of that item will LTN send to the station, if any? What will be the final amount of that item in the station after that shipment arrives (assuming no other changes to chest contents)?

Why would I bother with a blueprint? This is basic circuitry here. If you can't understand it, you don't have any grounds to talk about this being "impossible" to solve via circuits.

[redis]

Small edit to my first post, forgot a part of the solution was in my mod configuration settings and thus needs to be explicitly stated.

No your circuitry does not do that because there is only 1 threshold in existence on the combinator. There is also its variation called "Stack threshold".

"No mucking around with the request threshold signal"
This is where you do not understand. You can not change the threshold by manipulating the input for that level. The threshold number itself is used in the calculation of delivery size (in the code). Without changing it you are not able to create correct delivery size per item. This is basic math here. Optera understands the problem and the original poster too. If you think you are smarter than all then post your blueprint contraptions.

My circuitry doesn't need more than 1 threshold signal. Reread the post.
Also answer these questions, they should help you get closer to the answer:
1: If the request threshold is 0 and all item signals are positive, how much of what item(s) does LTN send to the station?
2: If the request threshold is 0, the signal from a chest with N, where N is less than 5000, of an item is sent to the lamp's network and a signal of -5000 of an item is also sent to the lamp's network, what is the resultant item signal in the lamp's network? How much of that item will LTN send to the station, if any? What will be the final amount of that item in the station after that shipment arrives?

Why would I bother with a blueprint? This is basic circuitry here. If you can't understand it, you don't have any grounds to talk about this being "impossible" to solve via circuits.

LTN ALWAYS REQUESTS the THRESHOLD AMOUNT (+ extra a few items drop during time to get to issue an order).

Results of this:
1. if you have threshold 0 it means it will immediately issue request if it drops to <0 with train(s) arriving with couple of stacks only and keep doing so every damn few seconds. You will get near empty trains with 0 threshold!
2. Opposite if you have large threshold of 50000. It means small items signal need to wait to drop to -50000 too to get order issued. You may not want to stack up on that many and if you use small trains for small items it will send 5 trains right away to fullfil such large order.

Sending Full trains actually would allow to set the small threshold (aka 0) and the train would not arrive with only a couple of stacks, but FULL and the next order would be issued after all the items are used up. If you can show circuitry which works with 0 threshold as is and sends full trains you are welcome rock star.


Edit: Now quoting Optera here
"A sturdier approach would be to use circuit logic division to create steps in desired delivery sizes."
This technique (division or multiplication) wiil somewhat work with circuitry allowing to "pretend" proportionally higher/lower level of items than actually is, but this is an ugly park of combinators (on top of you already use for something) and convoluted approach to report "wrong" values into the LTN input to solve a simple problem if you have many items types.