No, you just assumed/claimed a 300MB save would have half as many entities as your save, so just 15MB entities. Might be true, might be false, who knows. I picked one of the biggest megabases I could find. That means most other bases are smaller. I obviously can't speak for all bases, but statistic...

Ah, so my entire deleting was put into the queue. That makes sense, thanks!

Many thanks, this explains a lot!
I didn't see there was a recipe dump in the console, and undo queue, well, that makes sense.
Btw, is there a way to clear the undo queue when you see it gets too big?

Well, it's encoded in combinators - which are all deleted.

mrvn wrote: ↑

Wed Nov 10, 2021 9:54 am

I have no idea how big a save with actual terrain features would be for the factory you tested. You didn't even mention how big the save is, only that entities are 10/30 MB.

As I noted, it doesn't matter how big the save is. Didn't I just explain that?

Which is a useless value without something to compare it to. 10 MB in a 20MB save is a lot. 10MB in a 300MB save not. When the entities of a huge base barely exceed 30MB, you can assume that a small factory occupies proportionally less space. It follows that the rest of the save data is occupied by...

Because it doesn't allow comparing the size of the enmtity data to the rest of the save. Is the landscape below those entities 1MB or 100MB? That makes a big difference. You mean because of the uncertainties of compression? Apart from that, it allows to compare the "save with entities" to...

So, I took a map from the forum (the sandstorm movie player) and deleted everything in there. For some reason, the map is still huge though. My question now is, what makes this map so huge, and how do you figure that out? Map download here Might be a stupid question, but I really wonder how it can r...

Yes, a specific implementation would be better but short of encoding a chunk as a diff from when it was first generated (probably slower than sending the data), I think if you get the serialisation "right", rsync might be quite close. Not sure, I guess I'll run a few tests on the current ...

Separating entity main data and state might not yield that much. But separating them from the landscape and tiles should yield quite a lot. Yes, but wasn't that my idea from the start? As for your test base: That seems like a bad test. The map is all blank and should compress really well. So I'm no...

There are a lot of things in entities that generally remain constant. Like the inserter has a pickup and dropoff location. Belts have a direction. Assemblers have a recipe. Chests have a reserved space. Cargo wagons have item filters. Combinators have settings. All of those can change but generally...

As a result I think a worthwhile benefit may require either the data format to change to enable greater efficiency with a generic tool like rsync or (as was suggested previously) a Factorio-specific implementation using a similar concept. I think the former would have performance implications, at l...

BTW I found some old article https://forums.factorio.com/viewtopic.php?p=169194#p169194 , where I suggested to use delta to store the diffs between saves. This would help keep save sizes low, but wouldn't make saving/loading any faster. If anything, it would add overhead. Now the cool thing (and si...

More than 300 mb. And now when I think about it, it was just so slow when I updated the mods or a new version did came out - so that the game needs to filter every single tile. In normal case load took about 20 seconds, save perhaps 60. When updating it could take 2-10 minutes. And because I though...

It's churning through some GB of data and then transmitting num_chunks * 16 bytes or so. You would need a decent checksum because any collision would be fatal. I don't think a CRC32 would cut it. 128bit sounds good enough. Sure, it might be a few MB transferred for huge maps, but then you can be ce...

The game always has a CRC for the whole state so the client can compute the CRC and detect this problem and fall back to checksum chunks or download a full map or something. I would consider this a rare event and not one we should optimize for at the cost of the rest. Well, how high is the cost of ...

As noted, this might take even longer than downloading for big maps. Generating chunks is expensive. Time, cpu, bandwidth. Pick one. Time and CPU are essentially the same. Technically I only care about time, since I only want to save bandwidth for faster transmission :) If generating chunks is slow...

Na, single player also has this problem. I had saves where loading needed 2-3 minutes, saving 5. Mainly because of the large areas I discovered. May I ask how big that save was? For me, I can load even 200MB maps in a few seconds. Loading from a SSD? @ssilk Still wondering, would you mind telling m...

Uploading a save wold be easiest

Maybe like so: if there are only some minimal changes to a chunk (missing some trees, added some worms…) the changes can be applied as diff after the generation of the chunk. Possibly. But that doesn't sound too easy either. Especially not for a mod... Na, single player also has this problem. I had...