Optimization idea: abstraction

[StephenLynx]

I just can't think of a way where this helps UPC. To me this sort of ask will hurt UPC and require more processing power.

Instead of processing hundreds of entities like inserters, items, belts, assemblers, you process one thing.

[Deadlock989]

You just have to believe.

[netmand]

I just can't think of a way where this helps UPC. To me this sort of ask will hurt UPC and require more processing power.

Instead of processing hundreds of entities like inserters, items, belts, assemblers, you process one thing.

How so? Are you telling the devs to make assemblers that only make Advanced Circuits? How are you connecting them? Are these connections pre-built too? Will they accommodate all permutations of logistical delivery? Of course not, right? Then you are wanting the program to identify and abstract on the fly, tasking the program to build and unbuild these abstractions as needed. All that takes processing power too.

[StephenLynx]

How so? Are you telling the devs to make assemblers that only make Advanced Circuits? How are you connecting them? Are these connections pre-built too? Will they accommodate all permutations of logistical delivery? Of course not, right? Then you are wanting the program to identify and abstract on the fly, tasking the program to build and unbuild these abstractions as needed. All that takes processing power too.

Because you can infer the result and you can cache this inferential information. And no, you don't have to account for literally everything in the game. You can limit the cases. I am repeating myself here, read the thread.

[Jap2.0]

I, too, think that if we believe hard enough we can use machine learning to make blockchain-based abstractions.

There's a cadre of players that will rebel if the game is made any simpler.

You should read the thread. I opened by saying this wouldn't change anything at all as far as features or mechanics go.

Yet you continue it by saying that we have to make assumptions which we do not know will be true to implement this, thus inherently changing the mechanics.

[ssilk]

I'm sorry, but as a modder that is all laughable to be polite. No personal offense meant, i realize you're not a modder and just enthusiastic.

Sorry, but that's quite offensve. I've modded several stuff, helped improving many mods and libs. And wrote a little mod myself.
(Maybe I'm a bit out, of what can be done with a mod, since I have no time for that since 2 years, but the abilities of mods have improved, not declined.)

Of course I know how difficult this is, but to say "impossible" is too simple, after scribling my ideas about it on half a page.
That's just offensive.


Laughable is, how you think that a non-deterministic behavior should be simulated in a game, where nearly everything is about that; it's the essence of Factorio. Devs are with my opinion. (*)

This is a too theoretical discussion, so I deleted my statements to your arguments (some have already been answered), because nobody knows how it really is, until someone tried. Yes, I know that this is far beyond, of what a mod should do (but does anybody mean, this will be implemented into vanilla the next 2-3 years???) and there are many difficulties, but still possible and no need for offense.


(*) And to repeat the idea (because I think it's misunderstood):

Detect that part of production, that is deterministic. That is only possible for belt driven production and when that is full production or no production (and some more preconditions). Simulate full production and look, if this fragile deterministic state is left somehow to switch simulation off.

And you might see now (**), this is a kind of game in game to keep that fragile determinism running. And not many would love that, because it is quite complex. But that part of game in game is the only sense I can see in this suggestion, I called that concept "factory streets" and it could be read as fifth post in this thread.

... And it would help, if the devs would implement, that you can turn off complete parts of a factory just by turning the power off (inserters doesn't turn off when unpowered). Because belts are not power driven, the simulated part could recover and simulation can restart.

(**) it is difficulty to see, because it is even a game in game, without the whole optimization stuff: A factory street - in my sense of view - is some factory part, that needs to be marked out and feed with full input belts and there is no full compression on output belts. And in that case this part has an inbuild production/speed/efficiency-bonus.

[Deadlock989]

This is just a flood of gibberish.

Detect that part of production, that is deterministic.[

It is literally all deterministic. Because if any part of it wasn't then none of it would be. That's what deterministic means.

[bobucles]

I'm pretty sure we had this discussion a long time ago. The big issue with abstracting the factory out is that you no longer have a factory to look at. It's just a black box with inputs and outputs. You can do the same thing with gigantic magic cheat boxes, but who wants to build their factory out of those?

The closest equivalent is Factorissimo, where you plop down big boxes that have mini factories inside. The only thing that matters on the outside are inputs and outputs. Abstracting it down would basically remove the inner world and not render it at all.

[netmand]

Because you can infer the result and you can cache this inferential information. And no, you don't have to account for literally everything in the game. You can limit the cases. I am repeating myself here, read the thread.

Check out your terms: infer, cache, limit. These all take as much if not more processing.
You are also misleading yourself in assuming that I haven't read all posts here. We should progress the discussion into how to make abstraction work, not repeat an idea that doesn't.

[StephenLynx]

The big issue with abstracting the factory out is that you no longer have a factory to look at. It's just a black box with inputs and outputs.

Read the thread. My suggestion doesn't change literally anything as far as the player is aware.

[StephenLynx]

Check out your terms: infer, cache, limit. These all take as much if not more processing.

How does a cache uses "as much if not more processing"? You could argue it uses more ram. But how does it uses more processing? Why do you think we have cached shaders, cached web pages both on servers and clients?

And how does a limit would use more processing than not limiting it?

I know I have enough software development experience to talk about the things I am talking about. Do you know enough to refute it?

[Deadlock989]

I know I have enough software development experience to talk about the things I am talking about. Do you know enough to refute it?

There's nothing to refute. It's all just waffle.

[netmand]

How does a cache uses "as much if not more processing"? You could argue it uses more ram. But how does it uses more processing? Why do you think we have cached shaders, cached web pages both on servers and clients?

And how does a limit would use more processing than not limiting it?

Regarding cache:
Adding more data to cache increases processing requirements. The need to "infer" or rather rebuild non-pre-defined entities to represent pieces of the factory, storing them in cache, tracking which parts are abstracted, then merging results with the rest of the unabstracted factory presents another level of processing that is not being accounted for here. You can't treat an abstraction like an assembling machine; a train could be running through it. We still need to measure the production and consumption of the items within the abstraction. Dare I mention, fluid dynamics?

Regarding limits:
We can write a function that yields a result, but if the results need to be limited, the function will also need to check values along the way and add processing to yield results within the limit.

[netmand]

If abstraction as proposed were viable, the result would be a gateway to parallel processing. Which isn't happening. Not for a long time.

So then we're left with abstraction resulting in a simplification...

[StephenLynx]

Regarding cache:
Adding more data to cache increases processing requirements. The need to "infer" or rather rebuild non-predefined entities to represent pieces of the factory, storing them in cache, tracking which parts are abstracted, then merging results with the rest of the non-abstracted factory presents another level of processing that is not being accounted for here. You can't treat an abstraction like an assembling machine; a train could be running through it. We still need to measure the production and consumption of the items within the abstraction. Dare I mention, fluid dynamics?

Regarding limits:
We can write a function that yields a result, but if the results need to be limited, the function will also need to check values along the way and add processing to yield results within the limit.

While calculating what has to be added to the cache surely requires processing, it would heavily compensate to the data that doesn't need to be processed anymore due to the optimization. We are talking about dozens and dozens of entities being checked several times per second. Fetching from the cache is trivial because you can index the abstractions by map areas being entered/left. The only merge with the rest of factory that would happen would be putting the abstraction products into a belt. And you can isolate areas that can be treated like some sort of assembling machine. For example: you check if the part being abstracted has to interact with a logistics network. If so, you don't abstract. Mind you, just because you are abstracting entities in an area, that doesn't mean you have to abstract the whole area. It's not hard to see what's interact with what. If the train is just going through the area but doesn't interact in any shape or form with the assemblers, directly or indirectly, then you just ignore the train. And measuring the production and consumption of 1 thing is much faster than processing 30, 100, 200 things. About fluid dynamics, if the flow is consistent, you can still abstract it.

And checking the limit is still faster than processing everything outside that limit in this case. You place something. This something checks if it interacts with something else. If in this interaction group you find something that will not be abstracted, like a combiner, you mark that group as not being able to be abstracted. That is not hard and if you know about that group and you place something else that interacts with the group, you stop right there.

You are ignoring how much stuff is processed every single cycle of the game. Any optimization must be compared against the work that will be saved by the abstraction. But I see lots and lots of people here completely forgetting that and just saying that this abstraction will also require processing. Forgetting all the inserters, all the belts, all the assemblers, all the individual items going around them that have to be processed dozens of times per second. While this abstraction doesn't even have to be calculated immediately. You could just perform the initial calculation on the first time the area is a given distance from vision and that would be it. Reusing that data afterwards would be trivial.

[netmand]

You could just perform the initial calculation on the first time the area is a given distance from vision and that would be it. Reusing that data afterwards would be trivial.

I was hoping to have an academic discussion about your idea of abstraction, and how we could use it to optimize the game. It appears I'm not going to get it. When the idea is challenged, you're just saying the same thing over and over instead of having a working discussion that overcomes challenges put to the idea. While you assert your beliefs and brag about your experience, the reality is very very different.

You must think I'm totally ignorant and must be tired of repeating yourself. So I guess I'll just leave it at that... Except for one parting comment: If you think you can ignore a train, any active part of the factory doesn't interact with the logistic network, and fluid flow can be consistent at all; you are ignoring how much stuff needs to be processed every single cycle of the game.

[bobucles]

My suggestion doesn't change literally anything as far as the player is aware.

Magic makes a lot of things possible. That doesn't mean it's possible. There are existing solutions like clusterio, that's as close as it gets to magic box factory abstraction.

[ssilk]

This is just a flood of gibberish.

Detect that part of production, that is deterministic.[

It is literally all deterministic. Because if any part of it wasn't then none of it would be. That's what deterministic means.

Well, I mixed deterministic for "calculateable". You can calculate the throughput of a belt based construction for maximum filled belts quite precise. Versus you cannot do that if you want to calculate the throughput if the belts are below the maximum filled, because the throughput then will flutter too much.

[Zanthra]

If you have a set of assemblers making red science from iron plates and copper plates, what is the method to provide the abstraction.

The challenge I see is how to determine the state of the intervening iron gear belt from a given tick in the future. Say the player walks away from the factory, it gets abstracted away, but then they come back and decide they need iron gears for something, and are coming to the red science to pilfer them from the belt. When and how is the exact position and number of iron gears sitting on the intervening belt calculated. If you know of a way to do that from a given time in the future, cheaper than actually simulating the production of the iron gear assemblers and the consumption of the red science assemblers, along with the timing of inserter arms and the like, then there may be potential here.

[StephenLynx]

If you have a set of assemblers making red science from iron plates and copper plates, what is the method to provide the abstraction.

You see how many buffered inputs you can have and at which point these inputs turn into a subproduct, like the gears. When you stop the simulation, you place the buffered inputs further from the end as possible to avoid rushing the production. Then you also see how long does it take for items to start coming out of the simulation and add that as a buffer for finished products. The rest is just producting items in the same rate with the same productivity bonus.

You don't need to place the gears and iron plates exactly where they would be, you only need to place them somewhere the production wouldn't be sped up.