Rseding91 wrote:You don't have to have 1 creative-mode item in the world for it to have the performance impact. That's why it's particularly bad. I also said nothing about the crash - that's unrelated to the performance issues.
Maybe I wasn't clear. I'm only getting poor performance when activly dealing with blueprints, with Creative Mode Installed or without it. The CM impact, without spawning any of it's entities in the world, is negligable.
These two statements prove that mods are unrelated to my crash/performance report:
- The performance is fine as long as I don't try to use blueprints even with mods like Creative Mode.
- The the ups drops and the game crashes without mods when activly using large blueprints.
The performance issues and the crashes happen at the same stresspoints. The issues are linked. When I disabled CM I got marginally better performance and then could preview the 1024-balancer for a few frames (in certain circumstances) without crashing for a short while. Basically the game crashes if a frame hasn't been drawn for X amount of seconds (I'm not saying the frame delay literally causes the crash though).
Basically if the blueprint string was read in chunks and ghost were built 10k/frame max then the game would probably not crash at any blueprint size that your compter can run. The issue is that blueprints are massivly more performance intensive than actual entities.
It might even be possible to make a blueprint mod in lua that performs better (doesn't freeze or crash the game by doing updates over time) than the in-game blueprint system when dealing with large blueprints. Can't remember on the top of my head if the Lua-API gives me access to file reading which would make it fairly easy. Back to the old times when blueprints were a mod implementation. The C++ in-engine implementation shouldn't ever be beaten by a lua mod though if it is designed with large blueprints in mind. I'm not saying that it has to be solved exactly by adding ghosts over several frames, but it is a solution if you can't come up with something better.
____________
So after writing that above I see new responses that I will adress:
daniel34 wrote:I tested it and it did not crash, but used up to 10.1 GB of RAM on that map. It probably crashes if you have less, but that's not (directly) Factorio's fault. It's not related to mods either - all my analysis was done on vanilla.
Yes, I assumed that was the case. How much memory did the game use after you had finished placing the ghost (assuming the game does free up unused memory)? If we relaxed the requirement to place everything at once, even for blueprints with 10^6 entities, we could reduce the memory requirement to something a bit lower, yes?
daniel34 wrote:The performance was very, very low while handling the 1024-belt-balancer (898k underground belts, 159k transport belts, 34k splitters). The framerate was very low (<0.5FPS) while opening the blueprint/seeing the preview, but also while placing it down. After placing it down the framerate returned to 60. All of the low framerate time was spent in "Gui render preparation" while seeing the preview and "Game render preparation" while placing it down. Note that its not the GPU that does the preparation and caused the lag, its the CPU.
Yeah those slowdowns make working with large blueprints very, very painful. I can deal with placing a blueprint down and having the actual construction being completely built minutes later (instant blueprint, with robots it would take days though but maybe a minute to get the ghosts down if optimised). But playing the game at 0 FPS as soon as your cursor just hovers over an icon or the entire time while positioning the blueprint is quite bad.
daniel34 wrote:
It makes sense that when looking at the blueprint preview the game has to include all the items in the blueprint and render a proper preview, but does it have to do that every tick? Blueprints are not inherently mutable, surely you could have a client database of blueprint pictures indexed by string/CRC instead of re-rendering it 60 times per second, or attach it to the blueprint entity (in memory) itself.
Then I tried to place the blueprint, but the same lag happened. The debug shows "Game render preparation", but there is only a small part of the blueprint displayed in the game window, most of it is off-screen. The game should only try to render the on-screen part and ignore the other parts. After placing the blueprint (laying down ~1 million ghosts, which actually doesn't take any noticeable time) and Q-ing the blueprint the game is back to 60 FPS/UPS.
I think your optimisation ideas would drastically enhance the usability of using large blueprints.
daniel34 wrote:So we can, apparently, lay down large blueprints and have no issues afterwards, but we have to endure massive lag while selecting and then placing the blueprint.
Yep, exactly.
___________________________________________
Rseding91 wrote:I looked at the crash and it's crashing because it's running out of RAM trying to render the blueprint preview.
Not that surprising to me since I could barely generate the blueprint on my computer without running out of RAM and since the game used up all of it trying to work wih the blueprint. But the blueprint library tries to do the simple and easy process of keeping it all in memory at once when. The game should be smarter about it
Rseding91 wrote:Just loading the save takes 5.4 GB of RAM, if you also have the large blueprint copied into the library that adds another 1.5 GB and rendering the preview takes between 1 and 2 GB. It fails to allocate enough memory for the render preparation data because there's not a large enough contiguous piece of memory available for the request and it crashes.
You'll get the same performance if you zoomed the normal game view out that far - simply we don't allow it in the normal world view because no computer that exists today can handle it without severe performance drops. When you put that much stuff into a blueprint it still tries to render it but you now get to experience the performance hit of doing so.
I placed the blueprint in the library to be able to hover over it to get the preview. Would it take less memory to place it in my regular inventory? I did notice that the game froze for a few seconds moving the item from my cursor to the game library.
With daniels suggestions, why even render the whole item at once? It's never displayed at zoom level 1 and the belt items aren't connected in the blueprint view. The image database could also be built up asyncronously over time. The preview window could then pop up immediatly showing item requirement list and so on while the render preview is updated piecewise while the game plays on at a nice and smooth 60 fps with low RAM requirements. The render prep wouldn't be so big if you never try to prepare to render all of it in one go.
Rseding91 wrote:
You'll get the same performance if you zoomed the normal game view out that far - simply we don't allow it in the normal world view because no computer that exists today can handle it without severe performance drops. When you put that much stuff into a blueprint it still tries to render it but you now get to experience the performance hit of doing so.
But if I did that I would actually want the game to render the whole view each frame. For a static blueprint shown a low resolution with low requirements on update speed and acceptable delay for the entity rendering part of the preview it's not at all the same thing. The tasks are completely different from a user perspective and that means relaxed requirements from a technical perspective. One shows
a picture and another shows an interactive game.
Rseding91 wrote:I found one small area to improve memory usage in rendering the blueprint but the real "fix" if you don't want it to crash is either get more RAM or don't make blueprints that large
Hey, I'm not giving up and daniel isn't giving up, so you can't either!
Did you forget that making oversized contruction is the point of Factorio? And you are the legendary optimizer on Wube destined to save us all? What about your reputation, the glory and half the kingdom?
Ok, since I don't want you to just dismiss it because of the sillyness... I actually think the blueprint previews, contruction, destruction etc does need improvements. Does the whole blueprint need to be expanded into memory completely to place/decon planned? No. And the preview doesn't need to render the blueprint at the same FPS as the rest of the world. The blueprint rendering of the blueprints entities can be completely asynchronous to the rest of the preview rendering that shows item requirement list and buttons (when you click it).