Factorio Forums

Hi there,

I haven't really used Lua before, nor have I modded any games so this may be a stupid question.

My goal here isn't to modify the game but to extract eg. production data from the game. I have already looked into mods which do that so getting the data part is not the question.

As I was looking into these mods and skimming through the API I discovered that I have the chance to write to files while the game is running so that's a way of persisting the data.
But I would like to be able to send the data to some other application through some kind of IPC (I would accept any kind of queue/message type of communication. But if it can be
accomplished on a higher level I would prefer TCP or maybe UDP or if it's possible HTTP but it doesn't matter that much).
In the end I would like to persist the data in a database and do some analytics on it but that's obviously not the topic of this post.

I already looked into luasocket but it seems like I can't use that. Though I don't really understand why or how should that work, so I may be missing something here.

Is something like this even possible or am I forced to use files to transfer my data to some other application?

Thanks for your answers in advance!

At best you can write files from a mod. There's no other way to export data from the game.

Well, that's sad for me but thank you for the rapid answer.

You want to extract data from a running save while it's live?

Hapcy wrote:Well, that's sad for me but thank you for the rapid answer.

Although writing to a file seems to be somewhat slow, it actually works fairly well. What I'd suggest is in to set up handlers on the events you want to consume and/or set up a sampling rate in on_tick to read/calculate data every X ticks. Store this data in memory. Then, have a separate rate at which you write the data out to a file, adding the current tick count to every line in the file. I would also recommend writing every batch of data out to a new unique file, by appending an incrementing number or the tick count to the file name.

Then, you can have an external process ingest the batched data by monitoring the directory for new files and it is free to do anything with that information - store it in a database, make a remote call, provide a web API to access it, etc. You can also translate the tick counts back to a time (more or less) by writing the tick count as the last line in the file and correlating this with the modification time of the file (which is updated when the file is written and closed), assuming that you are able to maintain a steady UPS of 60.

Since you are always writing to a new file, you do not have to worry about read/write conflicts between your Factorio mod and your external process. You can also tune how often you sample data, how often you write a file, and how often you consume a file in order to get an acceptable level of performance.