Blueprint Diffusion Model

[Scorchchamp]

Hi,

I wanted to showcase a Blueprint Diffusion model that I have been working on for the past couple weeks.

Basically, the model takes a query as input, and uses a specifically-designed diffusion model that I developed to eventually generate a blueprint json (which is converted into a blueprint string). Below is an image of the best result thusfar:

02-17-2025, 22-56-26.png (1.36 MiB) Viewed 2625 times

Cool right?

Current issues:
1. Its a diffusion model, so it often results in a random building overlapping with another building 4x the size
2. It does seem to know when to use which belts as input, but often scrambles them up later on in the pipeline (yay for spaghetti base trainingdata)
3. It takes quite a bit of time to 'render' - currently it takes about 3 minutes to generate a full contraption (maybe because i'm running on cpu ;-;)
4. Not enough datasets to be 'creative' - I need more different types of trainingdata (other people's bases basically)

But yay for progress!

[eugenekay]

Neat concept! Now we can automate being drunk and making broken factory additions…


I would be interested in seeing the “thought process” and code involved… Screenshots of localhost looks like a good Minimum Viable Page.

How are you loading Input Data? Blueprint strings all the way down, or decoding Save files and modeling game state directly?

[ian5v]

Cool project! How do you ensure that the diffusion model outputs valid JSON?

[BraveCaperCat]

I'd like to see more of this. Anyway...
1. Diffusion models are generally used to produce high quality AI images and video. It probably just needs some more work.
2. Maybe don't only use a spaghetti base as training data?
3. Yeah, running it on a GPU rather than a CPU should make it much faster.
4. You should go to public, unmodded servers, download their save files and then train your AI on them. (maybe ask permission if anyone's online but if not, it's probably fine) Plus, there'll be people that want to give you their save files to train your AI on them. You can also go and grab a bunch of blueprints from Factorio Prints, since they have a lot of them. (there's even an API from the looks of it, so you don't have to go scraping the website)

It might also be good to open source the project or at least make it available to the public.