Performance optimization - post your saves

[FunMaker]

Determinism does not prevent multithreading - i know it is pain in the ass to do it right and synchornization issues might eat up all the performance gain but
[easyspeakingmodeon]
Robots that are:
non power-loosing
not in range of biters
not in range of drop off or pick up situations
would be the same outcome - no matter the order they are calculated
[easyspeakingmodeoff]
Well i know i ignored the fact that some quadtree/gridfile/other spartial data structures for storing robots is one of the most influencing bottleneck for multithreading and determinism (Simple as: Insertion order into a list). And surely i missed some aspects that additionally might break the determinism.

So don't take my "it's easy as getting butter on your bread" talk too seriously I know that many implications are there that break up good ideas. Just keep up the good work increasing performance

[Melfish]

My current save.

https://www.dropbox.com/s/h839rg1smb25v ... 2.zip?dl=0

Nowhere near my goal of 20m circuits but it has quite some beacons so it should be usefull.

[Rseding91]

My current save.

https://www.dropbox.com/s/h839rg1smb25v ... 2.zip?dl=0

Nowhere near my goal of 20m circuits but it has quite some beacons so it should be usefull.

Your save runs at 1.5 MS/tick for me so there's not a lot to be found slow in it

[Frightning]

I noticed the FPS slowdown due to steam engines seems to have been solved with 0.14 (ty), however, the slowdown I get in dense forests (at like 50-70% of the way zoomed out) remains. Curiously the problem vanishes at 100% zoomed out, and when zoomed in enough. (So maybe LOD related?) I can reproduce this issue on many different maps (note, these are maps gen'd in 0.13 though, maybe that has something to do w/ it? Haven't gen'd fresh 0.14 map and explicitly confirmed that the issue is still present)

[Rseding91]

I noticed the FPS slowdown due to steam engines seems to have been solved with 0.14 (ty), however, the slowdown I get in dense forests (at like 50-70% of the way zoomed out) remains. Curiously the problem vanishes at 100% zoomed out, and when zoomed in enough. (So maybe LOD related?) I can reproduce this issue on many different maps (note, these are maps gen'd in 0.13 though, maybe that has something to do w/ it? Haven't gen'd fresh 0.14 map and explicitly confirmed that the issue is still present)

That's just the graphics card being overloaded and not able to handle rendering all of the trees. Do you have tree mipmapping enabled in graphics settings? That can help.

[d4rkpl4y3r]

What graphics settings are you using?

I made a short video to show off my problem and the [graphics] section of the config.ini: https://youtu.be/cXq95aZeuw4

Something to note is, that I still use the multisampling setting that is no longer supported. But I tried it with it set to 0 and had exactly the same performance but everything was awfully pixellated on zoom in.

[Frightning]

I noticed the FPS slowdown due to steam engines seems to have been solved with 0.14 (ty), however, the slowdown I get in dense forests (at like 50-70% of the way zoomed out) remains. Curiously the problem vanishes at 100% zoomed out, and when zoomed in enough. (So maybe LOD related?) I can reproduce this issue on many different maps (note, these are maps gen'd in 0.13 though, maybe that has something to do w/ it? Haven't gen'd fresh 0.14 map and explicitly confirmed that the issue is still present)

That's just the graphics card being overloaded and not able to handle rendering all of the trees. Do you have tree mipmapping enabled in graphics settings? That can help.

I find it surprising to hear that a sprite based game can overload a GTX 550, and part of why I am skeptical of that explanation is that it runs flawlessly when I am zoomed all the way out, but tanks as I start to zoom in until I am viewing a small enough area that the tree count isn't so high, it's a dramatic shift and it's 100% controlled by number of trees on the screen+being at less than fully zoomed out, why would it behave this way? It's what I would expect from poorly optimized rendering code (it's textbook symptoms for it, sudden and dramatic FPS drop under specific conditions, that goes away as soon as the relevant conditions end).

[keyboardhack]

Observed problems:

• FPS loss in the productions tab. Usually halves the fps to open the production tab.
• FPS loss in the train preview when not set to map mode. The more the train preview camera is zoomed out the worse the fps gets.
• When placing tracks that connects to the railway the whole game stops for a short moment.
• Deconstructing or construction anything with a few hundred items will slow the game to 8-10ups.
• Running this command

  Code: Select all

  /c local surface = game.player.surface; for coord in surface.get_chunks() do surface.pollute({coord.x * 32, coord.y * 32}, -10000000) end

  usually increases the ups by ~5.

What graphics card do you have? Most of those sound like your graphics card just isn't powerful enough since they're almost all render related.

I have a R9 290 which should be more than capable.

• FPS loss in the productions tab. Usually halves the fps to open the production tab.
  Here is two ss showing the difference in timings.
  
  
  As can be seen the GUI render goes from 8ms to 37ms when opening the production window. This didn't happen before the production window was updated.
• FPS loss in the train preview when not set to map mode. The more the train preview camera is zoomed out the worse the fps gets.
  This lags the game even if the preview is of nothing but empty land.

config.ini

Graphics settings

(5.74 KiB) Downloaded 206 times

[Rseding91]

I noticed the FPS slowdown due to steam engines seems to have been solved with 0.14 (ty), however, the slowdown I get in dense forests (at like 50-70% of the way zoomed out) remains. Curiously the problem vanishes at 100% zoomed out, and when zoomed in enough. (So maybe LOD related?) I can reproduce this issue on many different maps (note, these are maps gen'd in 0.13 though, maybe that has something to do w/ it? Haven't gen'd fresh 0.14 map and explicitly confirmed that the issue is still present)

That's just the graphics card being overloaded and not able to handle rendering all of the trees. Do you have tree mipmapping enabled in graphics settings? That can help.

I find it surprising to hear that a sprite based game can overload a GTX 550, and part of why I am skeptical of that explanation is that it runs flawlessly when I am zoomed all the way out, but tanks as I start to zoom in until I am viewing a small enough area that the tree count isn't so high, it's a dramatic shift and it's 100% controlled by number of trees on the screen+being at less than fully zoomed out, why would it behave this way? It's what I would expect from poorly optimized rendering code (it's textbook symptoms for it, sudden and dramatic FPS drop under specific conditions, that goes away as soon as the relevant conditions end).

That's how mipmapping works. Once you pass a specific zoom level it starts using the scaled down sprites and so it's less load on the GPU.

[tha_snoetje]

@ Rseding, Just a thought, What if you grab a slower pc for testing?
Then the bottleneck becomes better noticable?

[bk5115545]

@ Rseding, Just a thought, What if you grab a slower pc for testing?
Then the bottleneck becomes better noticable?

Shoot just virtualize a linux and a windows image and then you can play with different hardware sets without buying anything new. VMWare Player is my favorite free hypervisor with KVM coming in second and lastly virtualbox.

[Rseding91]

@ Rseding, Just a thought, What if you grab a slower pc for testing?
Then the bottleneck becomes better noticable?

That doesn't give an accurate representation of what's slow. That just makes everything slower so everything takes the same percent time.

[Frightning]

I noticed the FPS slowdown due to steam engines seems to have been solved with 0.14 (ty), however, the slowdown I get in dense forests (at like 50-70% of the way zoomed out) remains. Curiously the problem vanishes at 100% zoomed out, and when zoomed in enough. (So maybe LOD related?) I can reproduce this issue on many different maps (note, these are maps gen'd in 0.13 though, maybe that has something to do w/ it? Haven't gen'd fresh 0.14 map and explicitly confirmed that the issue is still present)

That's just the graphics card being overloaded and not able to handle rendering all of the trees. Do you have tree mipmapping enabled in graphics settings? That can help.

I find it surprising to hear that a sprite based game can overload a GTX 550, and part of why I am skeptical of that explanation is that it runs flawlessly when I am zoomed all the way out, but tanks as I start to zoom in until I am viewing a small enough area that the tree count isn't so high, it's a dramatic shift and it's 100% controlled by number of trees on the screen+being at less than fully zoomed out, why would it behave this way? It's what I would expect from poorly optimized rendering code (it's textbook symptoms for it, sudden and dramatic FPS drop under specific conditions, that goes away as soon as the relevant conditions end).

That's how mipmapping works. Once you pass a specific zoom level it starts using the scaled down sprites and so it's less load on the GPU.

I verified that I do have mipmapping on, so perhaps, the 'roll-on' for the effect should start at a lower zoom level? Still seems like the kind of thing where doing benchmarks and figuring out what conditions cause the FPS to start to tank and optimizing the code against that could resolve its occurrence (I have a degree in mathematics, so I fully realize that optimization problems can be very hard to solve, just suggesting that this is something to look at optimizing so that the game runs consistently on a given machine spec and similar conditions; e.g. a megafactory can slow down a PC with a weak CPU, and that is to be expected, but having a rig with decently modern hardware run great until you find a dense forest zoomed most the way out is not something I would expect, e.g. nothing else, not even large biter bases or my kilobase cause any slowdown whatsoever on my rig).

[DaveMcW]

Decider combinators read every signal on the wire, even with a simple "if" condition that terminates after testing 1 signal. This makes large memory arrays very slow.

Save file: viewtopic.php?t=37490

[Roblin]

Background: I decided to mine out every single resource deposit in a large area to set up the core of my future mega base without having to worry about building on top of resources.

to do this the resources would obviously need to be stored somewhere (or destroyed, but I don't like waste, which is the reason why I'm not just building straight on top of resources in the first place).

so to organize the storage I decided to use logistic bots, as they solve the transport without having to do belt spaghetti and also sort the items for me as needed.

inevitably, the robots ended up being used by me in ways they were never intended to be, specifically, transporting large quantities of items over long distances (700+tiles)

in my naivity, I figured the throughput problem would solve itself if I just had enough bots, so I left a circuit condition on the bot production to continue as long as there weren't many available logistic robots.

the result, as I'm sure you have already guessed, is 35 000+ logistic robots in the network, of which about 25 000 of them are just standing in charging queues (yes, I have built a lot of extra roboports for charging and I have plenty of electricity to power the bots, the bottleneck is that the robots are really dumb when selecting ports to charge from and I probably still don't have enough ports in certain high-traffic areas.)

the frame rate seems to be throttled by the UPS, so despite there often being what looks like thousands of robots on the screen, there are no rendering issues.
my UPS is hovering (in the save file, which is about at its worst) at 33-35 UPS.

35k laggy idle robots.zip

35k laggy idle robots

(34.4 MiB) Downloaded 231 times

after the time of the save I started removing robots from the system and now I have only 11k robots and 50-55 ups with matching fps, which makes me strongly suspect that the idle robots standing in roboport queues were the performance culprits.

if you want to stress the system even harder there is a stockpile of an additional 20k-ish robots in the south-eastern corner of the main factory which, if released, should inflate the robot count to about 55k.

hardware:
Processor: Intel i7
Graphics Card: GTX 980
RAM: 8*2 GB

use the save if you wish, you guys are doing a great job and I wish you an excellent day
// Roblin

[Xarovin]

I'm not sure if you ended up implementing this or forgot about it (I haven't played in a while), but a while back you said you'd liked the idea of grouping construction robots by material needed, and I don't think I'd seen any change last time I played.

Like if you are missing concrete but have placed blueprints for 20k concrete, the robots will iterate through something like "is there concrete? no (x20000)" before the next item is searched for (which takes forever), as opposed to doing a "is there concrete? no: continue to next material / yes: run as normal"

[Roblin]

processor: intel i7
graphics card: GTX 980
RAM: 8*2 GB

this save was running on about 45 fps/ups and then I told construction robots to do many many things, after that it runs at 6 fps/ups

no mods, just vanilla.

too many construction orders.zip

too many construction orders/deconstruction orders

(75.99 MiB) Downloaded 234 times

[youdoomt]

We have this very large multiplayer map and it's easier to give the server address instead of me trying to download it and upload it somewhere:
as of writing this the server is running 14.20
name: "elricm.playat.ch"
password: "password"

Apart from the game slowed done to game speed .3 ish to be able to run the game here are some interesting things happening:

1.
When manual placing of a line of rail, the server goes "server not responding" for a couple of seconds(properly due to: for every rail the game has to calculate rail stuff in one game tick).
2.
any rail laying is also really laggy.
3.
We had some problems earlier on the map with power outage, and when we had them, we had insane amount of lag properly due to machines running low on power and lasers trying to defend off the aliens.

Otherwise I hope it will into optimizing the game.

[Klonan]

We have this very large multiplayer map and it's easier to give the server address instead of me trying to download it and upload it somewhere:
as of writing this the server is running 14.20
name: "elricm.playat.ch"
password: "password"

Apart from the game slowed done to game speed .3 ish to be able to run the game here are some interesting things happening:

1.
When manual placing of a line of rail, the server goes "server not responding" for a couple of seconds(properly due to: for every rail the game has to calculate rail stuff in one game tick).
2.
any rail laying is also really laggy.
3.
We had some problems earlier on the map with power outage, and when we had them, we had insane amount of lag properly due to machines running low on power and lasers trying to defend off the aliens.

Otherwise I hope it will into optimizing the game.

If you can upload it to this thread it would be better, the server might be gone once we get around to optimizations again

[Killerbee]

If you can upload it to this thread it would be better, the server might be gone once we get around to optimizations again

Here is the save I downloaded from his server, locally I only have performance issues(21 UPS) but don't notice any slowdown when laying rail.
So the problem must be on the server.

I also noticed some FPS slowdowns when zooming out, 14 FPS with 20 UPS.
You might want to investigate that too because I have a GTX 1080 and can't image how a scene like that can do that to a modern high end GPU.