[2.0.66] Mod performance issue: Trouble migrating massive amounts of item prototype names

[Neutrality]

I have a mod that generates 59000 items viewtopic.php?f=190&t=106078 . I'm trying to migrate it to 2.0. One of the changes in 2.0 is that many non-alphanumeric characters are no longer allowed in prototype names. Many of my items have used the character "|" which was removed, which means that I need to migrate every item prototype name. I didn't see a way to do this in Lua so I have to do it in JSON.

This ends up taking an extremely long amount of time, almost 17 minutes for this single migration to run. During that time, the progress bar only changed once, from 0% to 100% instantly. Most of the items don't exist in the world in a given save file, and don't end up getting shown to the user on the migration screen since they're hidden.

I would really like to be able to migrate these without such a long delay with no feedback from the game, or at least some way to warn the user that it'll take so long. (Although really I'd prefer to be able to use "|" in prototype names)

I attached the log file (ignore the mod error at the end, that's a different WIP error), the migration file standalone, the WIP version of the mod, and a save file I'm trying to migrate

[Rseding91]

I did some improvements around this logic 2 weeks ago pending a release of the next experimental game version. It makes it faster - but it will always take some amount of time. The issue is compounded by migrations running for each blueprint in the save file and each blueprint in your library. There's simply no way to remove the migration cost of having that many migrations.

[Rseding91]

After the changes I made 2 weeks ago, the provided save file and mod load in a few (3) seconds on my laptop.

[Neutrality]

Ok cool thank you, 3 seconds is perfectly reasonable. Looking forward to testing it.

[Neutrality]

Still takes just as long in 2.0.68, ~17 minutes. I have many blueprints in my blueprint library, some of them even for use with this mod, but none of the entities in the blueprints have items being migrated. And in the previously attached save, there are no instances of migrated items in the world. Is there a way for the migration system to ignore blueprints when none of the migrated items place entities?

[Rseding91]

There is no way for the engine to know that information until after it has loaded the save/blueprint.

You’ll have to decide if you want 17 minute migrations or just let the old items/entities be removed from old saves.