Friday Facts #148 - Optimizations for 0.14

[Clane]

Oh Twinsen, I love that game so much...

[kane.nexus]

Regarding the belt/pipe segment improvement.
Have you guys thought about making those segments as long as they can be until the split or make a connection?
So that means that pipes are treated as one segment until they attach to a machine or until they split to 2 or more pipes? For belts that would mean until they reach a splitter or an inserting interacting with the belt.
For your mentioned issues it solves quite a few of them:
- Splitters can still function as before since they won't be part of the segments.
- Saving the transport data can be per segment, since the segments will be the same for all clients as well as between save/load because there are simple rules that decide what is part of which segment that will be the same for everyone.
- Responsibility for saving can probably be solved by marking each segment by their top/left most location id (or something similar) since it would be the same for all involved. Then that entity handles all saving.
- Because of these clear rules of what is which segment that should also be making it fairly easy to merge/split/create them.
- Same thing for migration the old style to this new style.
- Underground belts are obviously no problem since they will simply count as part of an uninterrupted segment.

An image to illustrate:


Obviously this needs more thought to work, but I think this could help solve some of the issues.

One thing that I would add to this would be for the collision checks would happen at the end tile of the segment. Once there is a collision in that tile then that becomes a collision segment and the previous tile to that waits for a collision. When the tile is "full" then the tile stops and waits for space to move. This way only collision segments are stopped and items can move once an item is removed then the collision check wait resumes. Segments would be shortened instantly but lengthened slowly to help memory rewrites.

[Sogrog]

Another improvement could be some mechanism that automaticaly splits these huge segments around the player and bitter's paths, so any sudden change wouldn't required recalculation of whole block.

[AnarCon]

https://www.factorio.com/blog/post/fff-148

fun read and interesting challenges u got ahead of u, i wish u the best of luck n hope u come up with a elegant solution(s) and looking forward to fun trainstats in 14

[Xterminator]

Really enjoyed this FF. That is a ton of hours logged on a fairly short period, you have been busy.

The future optimizations look quite promising and the idea of making Belts even less performance heavy is music to my ears! I'm sure all the mega base builders here were happy to see this stuff.

Already can't wait for 0.14. haha

[Ohz]

Another improvement could be some mechanism that automaticaly splits these huge segments around the player and bitter's paths, so any sudden change wouldn't required recalculation of whole block.

I had this in mind too

0.13 is absolutly brillant and amazing guys, thank you so much for your amazing work

[massey]

i'm no programmer but wouldn't a fix for this be some kind of long distance belt?
It could be a part of the game mechanic.
cheaper or slightly faster covered belt (to encourage use) that cannot be split or interacted with by inserters. oh, and it also wouldn't move the player around! I hate getting flung around my main bus while trying to place things. then these sections, defined by the payer could be taken as a whole piece.
Do underground belts suffer from the same issues as normal belt?
If i underground long straight sections of belt will i get better performance?


obviously your current idea would fix more scenarios..

[scatterbraincc]

That looks like a fun thing to optimize. I used to program C with pointers, memory management, etc over a decade ago. You just made me miss it. Though I don't miss the day-long hunt for a memory leak; there is something cathartic, for me, thinking about data structures at that level. Probably why I liked Objective-C more than my coworkers.

[Fortanono]

Can we PLEASE make 0.14 the update where you add more energy generation methods? I don't care what it is, but something to bridge the gap between solar and steam and maybe for other stages of the game. A few ideas:

• Wind: Would need to either use Lubricant or replaceable parts to be balanced, but could be an alternative to an advanced solar panel.
• Biofuels: Would probably be an alternative way to make solid fuel rather than entirely new power, but would be nice to have.
• Upgraded boilers with less pollution, so using fuel is easier to live with.

[ChurchOrganist]

Can we PLEASE make 0.14 the update where you add more energy generation methods? I don't care what it is, but something to bridge the gap between solar and steam and maybe for other stages of the game. A few ideas:

• Wind: Would need to either use Lubricant or replaceable parts to be balanced, but could be an alternative to an advanced solar panel.
• Biofuels: Would probably be an alternative way to make solid fuel rather than entirely new power, but would be nice to have.
• Upgraded boilers with less pollution, so using fuel is easier to live with.

Sounds as though you should look at Klonan's mods......
https://mods.factorio.com/mods/Klonan/KS_Power

[Fortanono]

Can we PLEASE make 0.14 the update where you add more energy generation methods? I don't care what it is, but something to bridge the gap between solar and steam and maybe for other stages of the game. A few ideas:

• Wind: Would need to either use Lubricant or replaceable parts to be balanced, but could be an alternative to an advanced solar panel.
• Biofuels: Would probably be an alternative way to make solid fuel rather than entirely new power, but would be nice to have.
• Upgraded boilers with less pollution, so using fuel is easier to live with.

Sounds as though you should look at Klonan's mods......
https://mods.factorio.com/mods/Klonan/KS_Power

That mod looks amazing. However there's a difference between modded and vanilla. I love mods, but this is something that everyone should be able to experience. It would be an overall improvement to the game, which is why I suggested it.

[pr0n]

We really need nuclear where things like control rods or turbines and water are controlled through circuits. Similar to Big Reactors in minecraft if anyone's used that, only this should be more advanced and it should blow up if done incorrectly. It would also be interesting to have to deal with nuclear waste. It could do damage if not handled correctly etc. Nuclear material could be hidden in coal or stone deposites or something similar, like a 1% chance of extracting it every drill process, creating a new interesting thing that has to be planned for in your material processing.

[Fortanono]

We really need nuclear where things like control rods or turbines and water are controlled through circuits. Similar to Big Reactors in minecraft if anyone's used that, only this should be more advanced and it should blow up if done incorrectly. It would also be interesting to have to deal with nuclear waste. It could do damage if not handled correctly etc. Nuclear material could be hidden in coal or stone deposites or something similar, like a 1% chance of extracting it every drill process, creating a new interesting thing that has to be planned for in your material processing.

Nuclear would be nice, but it seems like a big undertaking for the devs. Not to mention power gen is meant to be simple in Factorio, and making a simplistic nuclear reactor wouldn't do the concept justice. Biofuel would be my idea of choice. Could add farming into the game, such as my algae idea here. But maybe a bit different so there's no pollution from the generators. Or a wind turbine that uses Lubricant like fuel, but doesn't pollute much (besides the obvious requirement for a refinery and chemical plant).

[Zeblote]

Nuclear would be nice, but it seems like a big undertaking for the devs. Not to mention power gen is meant to be simple in Factorio

Why? I'd rather build a cool/complex power plant layout than build 1000 solar panels... and 1000 more... and another 1000...

[Fortanono]

Nuclear would be nice, but it seems like a big undertaking for the devs. Not to mention power gen is meant to be simple in Factorio

Why? I'd rather build a cool/complex power plant layout than build 1000 solar panels... and 1000 more... and another 1000...

You have a point. I think it could be done.

[Optera]

that train kill statistic made me chuckle
Integrate Honk mod and add a victory honk after a kill

[ChurchOrganist]

Can we PLEASE make 0.14 the update where you add more energy generation methods? I don't care what it is, but something to bridge the gap between solar and steam and maybe for other stages of the game.

Sounds as though you should look at Klonan's mods......
https://mods.factorio.com/mods/Klonan/KS_Power

That mod looks amazing. However there's a difference between modded and vanilla. I love mods, but this is something that everyone should be able to experience. It would be an overall improvement to the game, which is why I suggested it.

I too am hoping that the dev team will implement Klonan's power stuff into the vanilla game at some point.

It would be nice to have in 0.14, but I suspect they already have enough to implement, with the stuff they didn't have time for in 0.13.

Power, as you say also needs looking at, but also fluid handling, which is a right royal pain atm - only being able to barrel crude oil for transport is causing me real problems in my vanilla playthrough of 0.13. I will be glad to have the Fluid Barrel mod back after my first vanilla playthrough.

[RobertTerwilliger]

Regarding the belt/pipe segment improvement.
Have you guys thought about making those segments as long as they can be until the split or make a connection?
So that means that pipes are treated as one segment until they attach to a machine or until they split to 2 or more pipes? For belts that would mean until they reach a splitter or an inserting interacting with the belt.
For your mentioned issues it solves quite a few of them:
- Splitters can still function as before since they won't be part of the segments.
- Saving the transport data can be per segment, since the segments will be the same for all clients as well as between save/load because there are simple rules that decide what is part of which segment that will be the same for everyone.
- Responsibility for saving can probably be solved by marking each segment by their top/left most location id (or something similar) since it would be the same for all involved. Then that entity handles all saving.
- Because of these clear rules of what is which segment that should also be making it fairly easy to merge/split/create them.
- Same thing for migration the old style to this new style.
- Underground belts are obviously no problem since they will simply count as part of an uninterrupted segment.

An image to illustrate:


Obviously this needs more thought to work, but I think this could help solve some of the issues.

I only want to add: use array-based segments with calculations only at inputs, outputs, and instead of every item moves by (distance) per tick, BELT COUNTER changes, but array stays inchangeable.
What I mean: you have items on belt:

Code: Select all

---A--BA----CDCE--AFBB--C

instead of moving EVERY SINGLE ITEM to the next slot PER UPDATE, like this:

Code: Select all

---A--BA----CDCE--AFBB--C
B---A--BA----CDCE--AFBB--
DB---A--BA----CDCE--AFBB-
(B, D have came from previous segment, C goes to the next segment)

array itself keeps being the same. Next tick will be:

Code: Select all

---A--BA----CDCE--AFBB--C#
---A--BA----CDCE--AFBB--#B
---A--BA----CDCE--AFBB-#DB
Where # isn't an array item - it's visualization of array counter position

This way, instead of moving 25 items (for this example, including empty slots) per update, calculations are made only for input, output, counter, which is... 3 - almost 10 times lesser, for this tiny segment.
Calculations will be a bit more complex, as each time system will need to find where are "the start" and "the end" of segment in array, but I think,
less of complex calculations is better than much more of simple ones

Also I haven't shown the fact, items take more than single position - they can be moved few pixels back and forth without another item able to fit between (due to hitbox), this makes the example just a bit more complex in implementation.
Also compressing such belt may cause some more heavy calculations, but I'm sure it will still give performance boost.
Inserters taking/puting items also will require finding belt's "current spot" in array, etc.

I hope this idea will help at least somehow.

[kovarex]

I only want to add: use array-based segments with calculations only at inputs, outputs, and instead of every item moves by (distance) per tick, BELT COUNTER changes, but array stays inchangeable.
What I mean: you have items on belt:

Code: Select all

---A--BA----CDCE--AFBB--C

instead of moving EVERY SINGLE ITEM to the next slot PER UPDATE, like this:

Code: Select all

---A--BA----CDCE--AFBB--C
B---A--BA----CDCE--AFBB--
DB---A--BA----CDCE--AFBB-
(B, D have came from previous segment, C goes to the next segment)

array itself keeps being the same. Next tick will be:

Code: Select all

---A--BA----CDCE--AFBB--C#
---A--BA----CDCE--AFBB--#B
---A--BA----CDCE--AFBB-#DB
Where # isn't an array item - it's visualization of array counter position

This way, instead of moving 25 items (for this example, including empty slots) per update, calculations are made only for input, output, counter, which is... 3 - almost 10 times lesser, for this tiny segment.
Calculations will be a bit more complex, as each time system will need to find where are "the start" and "the end" of segment in array, but I think,
less of complex calculations is better than much more of simple ones

Also I haven't shown the fact, items take more than single position - they can be moved few pixels back and forth without another item able to fit between (due to hitbox), this makes the example just a bit more complex in implementation.
Also compressing such belt may cause some more heavy calculations, but I'm sure it will still give performance boost.
Inserters taking/puting items also will require finding belt's "current spot" in array, etc.

I hope this idea will help at least somehow.

This is part of the plan (and reason this will be effective). This is what was meant by the sentence in the FFF:

Instead of moving every item in the segment, we can just change the overall segment offset as long as the belt exit is not blocked

[RobertTerwilliger]

This is part of the plan (and reason this will be effective). This is what was meant by the sentence in the FFF:

Instead of moving every item in the segment, we can just change the overall segment offset as long as the belt exit is not blocked

Ooops, I have to read more carefully
Okay, an amateur probably shouldn't give advices to professionals... At least same but independent idea means you're at the right way (wink-wink)