[MOD 0.16] Charge Transmission - Wireless Bot Charging

[Dustine]

Title: Charge Transmission
Short Description: Special entities that wirelessly charge bots around them. An idea by Mike C from the KatherineOfSkies community.
Name: ChargeTransmission
Factorio Version: 0.16
Mod State: Beta
Dependencies: Base
Optional Extras:
Author: Dustine
Downloads: Mod Portal
Licence: MIT

Bots waste a lot of time (and power bursts) having to recharge in the middle of their activities. But if you've got module effect transmission already, how hard would it be to beam electricity around as well...?

Enter the Bot Charger, the only entity added by this mod, which works exactly as it says on the tin.



Recipe

• 1 beacon
• 2x radars
• 10x processing units
• 20x batteries

To use, research Charge Transmission, place a Bot Charger near a functioning roboport and voilá, you're done. From now on, the charger will recharge any robots that attempt to recharge at said roboport before they reach them, with a satisfying electrical beam.



Chargers connected to the same roboport will share the load between themselves; this is important as each charger has a limited energy input amount and can only fulfill so many robots (see overtaxing below).

Power usage

Chargers use a lot of power because they have a x3 energy usage without modules, and robots aren't cheap to run to begin with (and get more power hungry the faster they are). There's a reason basegame bot bases usually have roboports as their number one power consumer.

A single charger feeds at worse between 174 to 49 robots over time (before space science), depending on a robot's speed. The number may seem small but chargers, even as power hungry as they are, diminish bot needs on a network over time so it balances out anyway.

Oh, and use efficiency modules, because you can up to double the performance of each roboport and even a single efficiency 1 module makes chargers reduce their power consumption to 255%. I recommend two efficiency 2 modules per each if you want to save on resources.



Overtaxing

A targeted roboport will display a custom alert over itself when it is overtaxed. It's hard to miss as it blinks like any other in-game alert, and also shows up on the minimap for extra visibility.



This means that the power needed to recharge robots wirelessly is more than what the chargers can take from the network. Now, this may mean two possibilities: either your power supply isn't sufficient or, most likely than not, there's just too many robots for each single charger to handle at once, as they have a max 10MW buffer recharge rate. For contrast, a vanilla roboport has a 5MW max recharge rate.

An overtaxed charger will slowly deplete its energy buffers, always charging as many robots as it can. This means that small hiccups (like when robots arrive from other networks, or need fluctuations) shouldn't completely stop wireless charging but eventually robots will start using the traditional charging methods, as shown on the animation above.

Easy solution though: chargers that point to the same roboport will share their work load, so just add more chargers! Or add modules if you haven't yet. Even if it is just a momentary hitch, if you see the alert, it won't hurt to construct additional chargers.

Maybe I should have called them energy pylons instead.

I found a bug, please fix it!

Sure! But I'll need a bug report for it. Describing what you were doing (+ steps to replicate the issue), what other mods you have installed, what version of this mod you have and if possible, sending me your bugged savegame. All these steps will make your issue much easier to diagnose and to be fixed.

DO NOT post your bug report on the mod portal, as it doesn't notify the mod maintainers about new posts or replies so it may take literal weeks for stuff to be noticed. Please, *please*, use this forum thread or github repo to make your bug report.

Credits

Special thanks to:

• Mike C for the concept and not giving up on it
• desseb for being key on the brainstorming and playtesting
• Nexela for general code help and workarounds
• KatherineOfSkies for her friendly and welcoming community

This mod contains graphics adapted from the icons made by Roundicons under a CC 3.0 BY license.

[Dustine]

Beta Status

As the post indicates above (and the mod's version not being 1.0, or it being on the WIP subforums (I'm sorry forum moderators for posting it on finished mods first ;o;)), this mod is on a beta status. This means that all core features are in and relatively stable and bug tested, but some extraneous ones may be upcoming, and that the code isn't very optimised performance wise. At all. It won't kill your PC but be careful with how many roboports you add chargers to, ok?

Upcoming features

• Adding a power armour version of the bot charger (personal bots still charge if they are in range of a fixed charger, but a portable one could be tasty...)
• Speed and processing optimisations
• Better model and component icons

[mraider94]

Nice gonna have to try this out. Had nearly this exact idea 9 months ago over on reddit.
https://www.reddit.com/r/factorio/comme ... echargers/

[Dustine]

Nice gonna have to try this out. Had nearly this exact idea 9 months ago over on reddit.
https://www.reddit.com/r/factorio/comme ... echargers/

thank you! the idea wasn't mine to begin with so to see more people with the same issue and proposal makes me happy!

i do agree with the answers that chargeless bots somewhat minimize the issue, but i like the concept more when it's a running cost than an initial investment.

[infinitysimplex]

Hi Dustine & community, thanks for this great and elaborated mod. In particular, thanks that it is so small! Great for people with slow internet connections. Some config-only-mods can be huge, but yours is just right.

However, with that large of a charging area, to me, the transmitter feels pretty overpowered (pun not intended... maybe ._. *cough*). Are there plans for an "independent" charger with a smaller charging area? Something like 7x7 tiles? That would feel a lot lees unfair, since it would mimic the beacon's limited range a (and that of inductive charging in general) slightly better. Such a charger would be restricted to predetermined locations with high robot traffic, like robot-operated train stations or heavily frequented roboports.

As an added, luxurious bonus, maybe you could (optionally) scale wasted energy with robot speed, or speed research? Thereby, the speed research would factor quadratically into power consumption. This would make for a great deterrent against transporting every small item with bots. (In-lore, it might just be harder to target bots the faster they move... ) I have some multiplayer colleagues who connect their ultra-compact factory set-ups to trains via robots, and robots only, and therefore spam their train stops with roboports for recharging. This kind of breaks the purpose of clever planning. Your non-blocking chargers seem like the perfect solution, but they also seem too easy and forgiving, planning- and energy-wise. But they're great for non-competitive single player set-ups.

Thanks for this great mod, and keep up the good work.

[Dustine]

Hi Dustine & community, thanks for this great and elaborated mod. In particular, thanks that it is so small! Great for people with slow internet connections. Some config-only-mods can be huge, but yours is just right.

However, with that large of a charging area, to me, the transmitter feels pretty overpowered (pun not intended... maybe ._. *cough*). Are there plans for an "independent" charger with a smaller charging area? Something like 7x7 tiles? That would feel a lot lees unfair, since it would mimic the beacon's limited range a (and that of inductive charging in general) slightly better. Such a charger would be restricted to predetermined locations with high robot traffic, like robot-operated train stations or heavily frequented roboports.

As an added, luxurious bonus, maybe you could (optionally) scale wasted energy with robot speed, or speed research? Thereby, the speed research would factor quadratically into power consumption. This would make for a great deterrent against transporting every small item with bots. (In-lore, it might just be harder to target bots the faster they move... ) I have some multiplayer colleagues who connect their ultra-compact factory set-ups to trains via robots, and robots only, and therefore spam their train stops with roboports for recharging. This kind of breaks the purpose of clever planning. Your non-blocking chargers seem like the perfect solution, but they also seem too easy and forgiving, planning- and energy-wise. But they're great for non-competitive single player set-ups.

Thanks for this great mod, and keep up the good work.

Oh wow, this made my day today! I've been... busy with my meatshell being horrible to code anything, so I'm still glad to see this getting use even after long on me adding well-needed updates to it.

The small size is actually incidental (and could be smaller if i removed some unnecessary libraries from it) because it doesn't have a lot of custom graphics, and the biggest one is reused from the main game (the animated antennae). I do want to have better graphics in the future but i'll be more conscious of the final mod size as that is a good feature to have!

Yes, it is overpowered, if not for ease. Initial plans were to make it basically a retextured beacon (including the beacon area) but the current system made both the coding and user usability of it a lot simpler. Like, if you make the area too small you end up needing to carpet the world with them to have any effect (i mean, you wouldn't but i know people would want to!), altho you lose some of the strategy regarding it, true (placing in critical spots vs plopping it on planned constructions). Having a intermediary tier could serve a purpose (and probably even be cpu-friendlier! huh.) but I'd need to balance and code it first. I already had a few ideas for to expand the mod (charge players!) but i'm not that excited about them so i was thinking on focusing on 1.0 first. I do like what you suggested a lot, i just can't promise it any time soon if i end up adding it, sorry.

And a quadratic increase in power versus linear? Honestly, if not for the energy cap (necessary code-wise, sadly. unless i did add the small-range chargers...) i would adore to do that ewe i do want to make them more expensive power-wise and that's a delightfully evil idea!

Thank you so much, will do so!

[infinitysimplex]

Great, the forum just logged me out again, while i was writing, and the text is gone. Seems like I take too long to write a post. -_-"

Oh wow, this made my day today! I've been... busy with my meatshell being horrible to code anything, so I'm still glad to see this getting use even after long on me adding well-needed updates to it.

I''m glad (and surprised) that you feel so positive about it. I was fearing for the worst, like burying you under even more work than you already have. Happy that it helps you.
And, don't be hard on yourself. Sometimes it just doesn't work out, and there might be nothing you could do about it. It won't "escape" you, you can still do it later.

The small size is actually incidental (and could be smaller if i removed some unnecessary libraries from it) because it doesn't have a lot of custom graphics, and the biggest one is reused from the main game (the animated antennae). I do want to have better graphics in the future but i'll be more conscious of the final mod size as that is a good feature to have!

Yes, it is overpowered, if not for ease. Initial plans were to make it basically a retextured beacon (including the beacon area) but the current system made both the coding and user usability of it a lot simpler. Like, if you make the area too small you end up needing to carpet the world with them to have any effect (i mean, you wouldn't but i know people would want to!), altho you lose some of the strategy regarding it, true (placing in critical spots vs plopping it on planned constructions). Having a intermediary tier could serve a purpose (and probably even be cpu-friendlier! huh.) but I'd need to balance and code it first. I already had a few ideas for to expand the mod (charge players!) but i'm not that excited about them so i was thinking on focusing on 1.0 first. I do like what you suggested a lot, i just can't promise it any time soon if i end up adding it, sorry.

No hurries! I'll barely be able to convince anyone to use a harder item in multiplayer, and I have some charger-mods for single player (that I haven't even had the time to try out). I'd offer to mod it myself, but I neither know Lua nor the factorio internals. ._. Anyway, there is no need for this, it was just an idea. Do whatever (and whenever) you feel like it.

And a quadratic increase in power versus linear? Honestly, if not for the energy cap (necessary code-wise, sadly. unless i did add the small-range chargers...) i would adore to do that ewe i do want to make them more expensive power-wise and that's a delightfully evil idea!

Thank you so much, will do so!

Well, to be honest, I was just calculating the energy consumption and accidentally factored in the speed too often. The results were way too large (1MW per construction bot). And then... I kinda liked the thought. Seems like I'm not the only one. But, please don't hurry, take your time, and do what you like best. Just had a stupid idea, you know.

[piterek2003]

Hey,

Awesome idea for the mod. I instantly wanted to use it, cause I thought:
1. More "air-time" = less bots needed = more UPS.
2. Charging stations are 2/2 in size and can be placed in so much better spots, therefore minimizing distance between train and assemblers. Again more UPS!

So I gave it go and in reality it's a...UPS killer :/ After placing only around 20 chargers my UPS dropped significantly. Any chance of optimizing it?

[Dustine]

Hey,

Awesome idea for the mod. I instantly wanted to use it, cause I thought:
1. More "air-time" = less bots needed = more UPS.
2. Charging stations are 2/2 in size and can be placed in so much better spots, therefore minimizing distance between train and assemblers. Again more UPS!

So I gave it go and in reality it's a...UPS killer :/ After placing only around 20 chargers my UPS dropped significantly. Any chance of optimizing it?

Weird, I never got notified of this post, sorry for the delay!

On the UPS weight... yep, I'm optimising it (and 0.5 should be even better on that regard, even with the new features I'm adding). The mod is on alpha, so I'm trimming UPS from its main cycle every new release in one way or another, and will only call it 1.0 once I'm sure I can't squeeze more performance off it. That and my mod dev computer isn't... the best, so I usually do catch it when I make a mistake or two and it gets a lot heavier.

That said, 20 chargers (specially if they're around the same spot) shouldn't lag the game this much, even if by your description you hadn't connected them to anything. Do you still have the savegame where this happened?

[Dustine]

Also it's been two months since I said I was working on new features. Whoops.

Due to performance hits I had to drop the idea of having individual-area chargers, which really annoys me because I do like the strategy element of it. On the non-linear power usage, there were also performance hits for the few test-runs I did of it but I could find a way if making the chargers overspend on charging individual robots as well was acceptable. I mean, I have a few ideas, I'd need to test them as well but for now I'm dropping that to the wayside.

But on good news, I've have been doing some actual progress! Here's a sneak peak of what's (potentially) to come for 0.5...

Picture sneak peak

[eradicator]

I've had a look at your on_tick function, it's a lot longer than what i would have expected for a mod like this. Especially you seem to be doing node maintenance, which is guess is related to the charging cells? If my interpretation of that is right don't really see a reason to do that in on_tick instead of using on_built/on_mined type events. While i kinda expected this to be a simple:
Loop through chargers -> find bots in chargers range -> chargs some bots until charger is depleted.

By the way what kind of target bot count do you have for this? Large factories can easily have several thousand bots which makes it not uncommon for one or two thousand bots to be in the same logistic cell. Especially with train stations.

Also btw i saw your interface request for reading beacon data directly (the main reason i came here ). And i was wondering why you need to iterate through beacon data so often that it becomes a performance hit?

[Dustine]

I've had a look at your on_tick function, it's a lot longer than what i would have expected for a mod like this. Especially you seem to be doing node maintenance, which is guess is related to the charging cells? If my interpretation of that is right don't really see a reason to do that in on_tick instead of using on_built/on_mined type events. While i kinda expected this to be a simple:
Loop through chargers -> find bots in chargers range -> chargs some bots until charger is depleted.

By the way what kind of target bot count do you have for this? Large factories can easily have several thousand bots which makes it not uncommon for one or two thousand bots to be in the same logistic cell. Especially with train stations.

Also btw i saw your interface request for reading beacon data directly (the main reason i came here ). And i was wondering why you need to iterate through beacon data so often that it becomes a performance hit?

It doesn't become a performance hit but it's there enough for me to get annoyed it exists I like having that chargers have no extra ups weight, and with modules, they do even if it's trivial (like 100 chargers added 0.005ms that weren't there before). Soooo, I don't want that.

And yeah, version 0.5 got rid of the node maintenance part on on_tick (it happens on place/removal yeah but also when iterating nodes, if it's an invalid one I remove it there and then), but to explain: a node is a collection of chargers connected to the same roboport, the mod treats all of them as a single entity. So when a player places or removes chargers, or roboports, you'd need to update them, but also when a player reassigns one with rotating. And adding items to a table while it's being iterated by next(), which is needed to smooth updates along several ticks, can cause desyncs which makes me need to have something on on_tick too to validate any new nodes and it's a headaaaache. My local git is full of branches of me rewriting and rewriting this system to be lighter or more streamlined, with some success-es and some nots.

Edit: I got a bit sidetracked there. Uh, as I was trying to say, nodes are an abstraction to save UPS, and adding new nodes are tricky because just inserting them directly causes desyncs (as I learned the hard way on version 0.3). But I'm still trying to move as much as I can out of on_tick so I appreciate a lot your comment here ^w^.

Target bot count, well my PC starts to lag at 20k but not from the mod itself; while it has a 1ms weight by then on my machine, the game *itself* starts going below 60UPS from that many bottage. And I call it a win when the game itself comes to a crawl before my mod has an impact on performance on my system.

[eradicator]

Ah. So the chargers are beacons now instead of EEI? (sorry, i didn't really read ALL of your code, just some bits ). As neither beacon efficiency nor module effects change during runtime this sounds like a good place to add some caching or even memoization. If that doesn't help enough then imho just reducing the frequency of the check is a good option. Realistically module configuration isn't going to change very often. (I have at least one slow-update cycle on one of my modded entities which i just explain to the player lorewise in the tooltip to get away with it. So they don't go "wtf" when updating isn't instant). Though if the impact of that is only 0.005ms then i'd probably aim to optimize the other parts as good as i can before that. What's the minimum time your mod takes with 100 chargers if there's no bots at all?

So if you're grouping chargers onto roboports does that mean that the list you're iterating over is basically an ordered list of roboports_with_charger and you're calling find_entities_filtered exactly once per roboport? If yes than i guess that's as low as you can get on the number of function calls there. So let's move on to the next: I noticed you had some "odd" distinction between logistic and construction bots which i didn't quite get. Is the charging range different for those two bot types? Because if it's just a way to determine maximum charge that's another thing that can be cached by bot type.

As to why you're causing desyncs with something as common as adding things to a table i'm not sure what you're doing there . Is the table not global maybe?

Regarding spreading of calculations i recommend you read my post on another thread.

Also looking at the documentation i noticed that find_entities_filtered supports a limit= parameter to reduce the number of entities returned. As this is probably the most important call in your on_tick i recommend you have a closer look there. The idea would be to estimate based on available charger energy how many bots you can at maximum charge, and then limit it down to that many.

[piterek2003]

Hey,

Awesome idea for the mod. I instantly wanted to use it, cause I thought:
1. More "air-time" = less bots needed = more UPS.
2. Charging stations are 2/2 in size and can be placed in so much better spots, therefore minimizing distance between train and assemblers. Again more UPS!

So I gave it go and in reality it's a...UPS killer :/ After placing only around 20 chargers my UPS dropped significantly. Any chance of optimizing it?

Weird, I never got notified of this post, sorry for the delay!

On the UPS weight... yep, I'm optimising it (and 0.5 should be even better on that regard, even with the new features I'm adding). The mod is on alpha, so I'm trimming UPS from its main cycle every new release in one way or another, and will only call it 1.0 once I'm sure I can't squeeze more performance off it. That and my mod dev computer isn't... the best, so I usually do catch it when I make a mistake or two and it gets a lot heavier.

That said, 20 chargers (specially if they're around the same spot) shouldn't lag the game this much, even if by your description you hadn't connected them to anything. Do you still have the savegame where this happened?

I'm happy to provide the save game (https://drive.google.com/file/d/0B8h8O2 ... sp=sharing), but keep in mind I use quite a few mods, some of which are needed to run the save (advanced nuclear, roboport recharge upgrade, robot batter research, ups-up). It's a mega base that has just dropped to 57 UPS (but I'll fix it soon, rebuilding atm ). Try putting chargers in the biggest bot networks (that need to be reworked by me ), namely train stations PGB Science or Wheels Monster.

[Dustine]

Hey,

Awesome idea for the mod. I instantly wanted to use it, cause I thought:
1. More "air-time" = less bots needed = more UPS.
2. Charging stations are 2/2 in size and can be placed in so much better spots, therefore minimizing distance between train and assemblers. Again more UPS!

So I gave it go and in reality it's a...UPS killer :/ After placing only around 20 chargers my UPS dropped significantly. Any chance of optimizing it?

Weird, I never got notified of this post, sorry for the delay!

On the UPS weight... yep, I'm optimising it (and 0.5 should be even better on that regard, even with the new features I'm adding). The mod is on alpha, so I'm trimming UPS from its main cycle every new release in one way or another, and will only call it 1.0 once I'm sure I can't squeeze more performance off it. That and my mod dev computer isn't... the best, so I usually do catch it when I make a mistake or two and it gets a lot heavier.

That said, 20 chargers (specially if they're around the same spot) shouldn't lag the game this much, even if by your description you hadn't connected them to anything. Do you still have the savegame where this happened?

I'm happy to provide the save game (https://drive.google.com/file/d/0B8h8O2 ... sp=sharing), but keep in mind I use quite a few mods, some of which are needed to run the save (advanced nuclear, roboport recharge upgrade, robot batter research, ups-up). It's a mega base that has just dropped to 57 UPS (but I'll fix it soon, rebuilding atm ). Try putting chargers in the biggest bot networks (that need to be reworked by me ), namely train stations PGB Science or Wheels Monster.

Will do! Will report later the performance both on 0.4 and on 0.5, but no promises on time as I do want to finish 0.5 as soon as I can. Thank you!

[Dustine]

Ah. So the chargers are beacons now instead of EEI? (sorry, i didn't really read ALL of your code, just some bits ). As neither beacon efficiency nor module effects change during runtime this sounds like a good place to add some caching or even memoization. If that doesn't help enough then imho just reducing the frequency of the check is a good option. Realistically module configuration isn't going to change very often. (I have at least one slow-update cycle on one of my modded entities which i just explain to the player lorewise in the tooltip to get away with it. So they don't go "wtf" when updating isn't instant). Though if the impact of that is only 0.005ms then i'd probably aim to optimize the other parts as good as i can before that. What's the minimum time your mod takes with 100 chargers if there's no bots at all?

So if you're grouping chargers onto roboports does that mean that the list you're iterating over is basically an ordered list of roboports_with_charger and you're calling find_entities_filtered exactly once per roboport? If yes than i guess that's as low as you can get on the number of function calls there. So let's move on to the next: I noticed you had some "odd" distinction between logistic and construction bots which i didn't quite get. Is the charging range different for those two bot types? Because if it's just a way to determine maximum charge that's another thing that can be cached by bot type.

As to why you're causing desyncs with something as common as adding things to a table i'm not sure what you're doing there . Is the table not global maybe?

Regarding spreading of calculations i recommend you read my post on another thread.

Also looking at the documentation i noticed that find_entities_filtered supports a limit= parameter to reduce the number of entities returned. As this is probably the most important call in your on_tick i recommend you have a closer look there. The idea would be to estimate based on available charger energy how many bots you can at maximum charge, and then limit it down to that many.

Ah, no, the EEI stays because I do want something with a variable energy usage (prettier energy graphs! without that much of a performance hit, i though?), but I suppose there are ways around that.

As of right now, 100 chargers without bots, on the same node, reaches 0.078~0.120ms on my machine (normal graphics, small test world). I can't tell if the fluctuations are from the mod or my computer because all mods running alongside (creative-mode specially) get the same ups spikes in the same intervals.

And true, true, I hadn't thought to cache module efficiency as I already do for bot's maximum energy (and other stats, should the need arise). I had planned to cache the module efficiency result at least but that should also be a good step, specially as I only care for one kind of module (efficiency) on chargers. So, thank you so much <3. Come 0.16 that won't be an issue anymore~

And you're right, it's better to aim elsewhere if there are bigger issues but I also like trimming off the easy parts too now than to never do them, delaying them over and over while chasing the miliseconds elsewhere. That and it avoids programming burnout from my part, having more than one task to focus on.

Yes, that's almost right. Chargers are grouped around roboports (nodes), and find_entities_filtered is called twice (i'll go over on why twice later, it's something i wish i didn't have to do) so to iterate over all the bots, of that force, in the considerable target area, and then assigned energy consumption as a batch. Now, I have to run the scan twice because find_entities_filtered only accept, as far as I know, as a filter one entity type at a time. And construction and logistic bots have different types, they're not both "robot". I already cache a robot's max energy on a game's start-up, all I'd need to do is cache it per force and update it every 30 seconds or so (?) because of some force-modifiers that directly touch robot battery size now. Just a simple multiplication.

On what's causing desyncs... I wish I knew! The table I'm referring to is on global (global.nodes), and it's iterated with next(...) inside on_tick over several ticks. I've read online that adding elements to a table while using next() is unadvised (removing is fine) but I've also read that in Factorio that'd be fine because of determinism. So, I didn't mind at first. And then the desync reports started coming. And then I did the blind leap of making a separate global table for newly created nodes and only add them to global.nodes before starting next() iteration (when the iterating index is equal to nil). Tested on the desync maps, no more desync, and never got any more reports from people using the mod on multiplayer. It could be a coincidence, heck it probably is, but the overview it adds seems to be minimal.

I haven't read the link you've sent me YET (wanted to write this... really big reply, whoops, first. wish i had pictures to cut it too) but the biggest issue with spreading calculations, as I've tried before, is that any mechanism that saves the bots for later causes a lot more ups consumption than not. Could be me doing anything wrong, I'm open for suggestions or anything that does work gladly so thank you for the stuff, I'll check it once I'm done writing this. I've had the idea of making smaller scans over an area and spread that over time, but unsure on how to implement it.

Oooh, didn't know of limit. I feel the math wouldn't be as easy as it seems at a first glance (how do you predict how X of energy you'll need when spread over several robots with different battery sizes (as in, how much of X to each type?), without doing a count_entities_filtered that while lighter on ups is still an extra weight on the critical code path) but that is DEFINITELY something I need to investigate and play with. Specially as I either already have or plan to have all the necessary variables cached or known before the scan!

[eradicator]

Darn, with that post size the forums input field is really too small to do proper editing . /me opens notepad.

Ah, no, the EEI stays because I do want something with a variable energy usage (prettier energy graphs! without that much of a performance hit, i though?), but I suppose there are ways around that.

Oh. Nono. EEI's are just fine. I was just asking for better comprehension.

As of right now, 100 chargers without bots, on the same node, reaches 0.078~0.120ms on my machine (normal graphics, small test world). I can't tell if the fluctuations are from the mod or my computer because all mods running alongside (creative-mode specially) get the same ups spikes in the same intervals.

That doesn't sound too bad at least @120ms. That every mod gets the same spikes is obvious. Far too many people use if game.tick % 60 == 0, so everybody does their heavy calcuations on the exact same tick. The post linked above (that you should've read by now ) mentions this and how to fix it.

And true, true, I hadn't thought to cache module efficiency as I already do for bot's maximum energy (and other stats, should the need arise). I had planned to cache the module efficiency result at least but that should also be a good step, specially as I only care for one kind of module (efficiency) on chargers. So, thank you so much <3. Come 0.16 that won't be an issue anymore~

Yea. I saw your request got implemented. Grats.

And you're right, it's better to aim elsewhere if there are bigger issues but I also like trimming off the easy parts too now than to never do them, delaying them over and over while chasing the miliseconds elsewhere. That and it avoids programming burnout from my part, having more than one task to focus on.

That's another way to see it i guess. I was just thinking that based on your description you wouldn't even be getting any milliseconds out of it ^^.

Yes, that's almost right. Chargers are grouped around roboports (nodes), and find_entities_filtered is called twice (i'll go over on why twice later, it's something i wish i didn't have to do) so to iterate over all the bots, of that force, in the considerable target area, and then assigned energy consumption as a batch. Now, I have to run the scan twice because find_entities_filtered only accept, as far as I know, as a filter one entity type at a time. And construction and logistic bots have different types, they're not both "robot". I already cache a robot's max energy on a game's start-up, all I'd need to do is cache it per force and update it every 30 seconds or so (?) because of some force-modifiers that directly touch robot battery size now. Just a simple multiplication.

Yea. Can only do one type per call. But the really expensive part is the number of objects returned (thus limit) not the number of calls. So it wouldn't be much cheaper even if you could batch them. Yea, every 30s should be fine. It's just reading number in a table after all. That way you're safe if any mod ever should increase it without triggering it by technology.

On what's causing desyncs... I wish I knew! The table I'm referring to is on global (global.nodes), and it's iterated with next(...) inside on_tick over several ticks. I've read online that adding elements to a table while using next() is unadvised (removing is fine) but I've also read that in Factorio that'd be fine because of determinism. So, I didn't mind at first. And then the desync reports started coming. And then I did the blind leap of making a separate global table for newly created nodes and only add them to global.nodes before starting next() iteration (when the iterating index is equal to nil). Tested on the desync maps, no more desync, and never got any more reports from people using the mod on multiplayer. It could be a coincidence, heck it probably is, but the overview it adds seems to be minimal.

Global stuff should be safe from desyncs. So no clue what happend there. Maybe you accidentially fixed some rogue local variable while changing the code.

I haven't read the link you've sent me YET (wanted to write this... really big reply, whoops, first. wish i had pictures to cut it too) but the biggest issue with spreading calculations, as I've tried before, is that any mechanism that saves the bots for later causes a lot more ups consumption than not. Could be me doing anything wrong, I'm open for suggestions or anything that does work gladly so thank you for the stuff, I'll check it once I'm done writing this. I've had the idea of making smaller scans over an area and spread that over time, but unsure on how to implement it.

The idea was to spread the cells/nodes not the bots. Spreading the bots would be wrong because they might not even exist or be in range a few ticks later.

Oooh, didn't know of limit. I feel the math wouldn't be as easy as it seems at a first glance (how do you predict how X of energy you'll need when spread over several robots with different battery sizes (as in, how much of X to each type?), without doing a count_entities_filtered that while lighter on ups is still an extra weight on the critical code path) but that is DEFINITELY something I need to investigate and play with. Specially as I either already have or plan to have all the necessary variables cached or known before the scan!

Trivial. Suppose you have two bot types, and a node that has 100 energy available:
botA, max_charge = 10
botB, max_charge = 20
So now take smallest max_charge, which is 10. Devide 100 by 10 -> You now know that you can at most charge 10 bots before running out of energy, so you can limit entities_filtered to 10. This would be just to limit the impact of getting back 2000 bots when you can only charge a small amount of them in megafactories. In normal factories it probably won't have a very large impact.

[Dustine]

So I won't be able to do anything code-wise or anything today because Opus Magnum came out and I already lost several hours to it. Many more to come, I'm sure.

That doesn't sound too bad at least @120ms. That every mod gets the same spikes is obvious. Far too many people use if game.tick % 60 == 0, so everybody does their heavy calcuations on the exact same tick. The post linked above (that you should've read by now ) mentions this and how to fix it.

Yep, read it. And I feel kinda silly because the thing you suggested... I'm already doing it. Only global.nodes is a sparse list so stuff is done a "biiiit" differently.

The following code, which is inside on_tick, around line 400 or so of control.lua, was changed so it makes more sense out of its context but the logic is the same:

Code: Select all

local next_node = next(nodes)

local iter = 0
local count = table_size(nodes)
local max

-- Damage control, does ALL remaining nodes in the end until you nil
if event.tick%60 == 59 then max = math.huge 
elseif event.tick%60 < count%60 then max = math.ceil(count/60) 
else max = math.floor(count/60) end

while next_node and iter < max do
  local node = nodes[next_node]
  iter = iter + 1
  -- (...)
  next_node = next(nodes, next_node)
end

The math is a bit... confusing (took me an afternoon of fumbling around and testing to make sure it worked as I wanted) but what it does is it spreads out all nodes over 60 ticks as evenly as possible, as just using math.ceil(count/60) as the max would clump up most nodes at the start if the number of nodes is close to a multiple of 60.

An example of what I mean by clumping, imagine if you have to iterate 6 nodes over 5 ticks. If you used math.ceil(...) as your max you'd get 2-2-2-0-0, while this method would give you 2-1-1-1-1. If that makes sense.

On the actual mod next_node is saved on global so it can be recalled on the next tick. global.nodes does somewhat need to be sparse...

Also I wasn't clear on the fluctuations, they're more that every few seconds every mod gets twice as heavy ups-wise, stays that way for a few seconds more and finally it slowly dwindles to the previous ups usage. This all over twenty seconds or so. It's a rolling kind of lag, I'm unsure if %0 is to blame here!

Yea. Can only do one type per call. But the really expensive part is the number of objects returned (thus limit) not the number of calls. So it wouldn't be much cheaper even if you could batch them. Yea, every 30s should be fine. It's just reading number in a table after all. That way you're safe if any mod ever should increase it without triggering it by technology.

Oh! That... makes a lot of sense. Seems limit=... is the way to go, that or that suggestion I had of splitting roboport areas into quadrants and iterate that instead.

Trivial. Suppose you have two bot types, and a node that has 100 energy available:
botA, max_charge = 10
botB, max_charge = 20
So now take smallest max_charge, which is 10. Devide 100 by 10 -> You now know that you can at most charge 10 bots before running out of energy, so you can limit entities_filtered to 10. This would be just to limit the impact of getting back 2000 bots when you can only charge a small amount of them in megafactories. In normal factories it probably won't have a very large impact.

Yeah but you'd be giving priority to botA, wouldn't you? Wouldn't it be fair to also save some for botB kind? But if priorities and spreading out isn't a concern, then yeah, it's trivial.

Only I'd use the estimated energy usage per 60 ticks (or whatever pooling interval i'd use) because that's the minimum energy you're estimated to recharge per bot, so you wouldn't get the case where you asked to charge 10 out of a 2000 and still had juice left over by the 10th.

Actually, what do you think would be heavier: for area A, doing 2 find_entities_filtered for all kinds of robots, or doing n searches but one per each (valid) robot prototype? Like, you said above the biggest ups usage is creating/storing the bot entity references, while i always thought the heaviest part was the scan itself which is why I went for 2 (hoping for 1). Homework for me too I suppose.

[eradicator]

So I won't be able to do anything code-wise or anything today because Opus Magnum came out and I already lost several hours to it. Many more to come, I'm sure.

I never knew they had a remake in the works. I was going to throw curses at you for telling me this. But instead i'm going to link you to the original: Magnum Opus. Lucky for me i'm largely immune to early-access so i'm gonna wait it out like i did with Infinifactory.

Only global.nodes is a sparse list so stuff is done a "biiiit" differently.

Is there any reason for it to be sparse (because i don't see why)? If you can make it proper that's going to make the other parts of the code easier (==shorter, faster). Even if the spareness is required for some reason, you could still construct a non-sparse indexing table above it to get the benefits.

The following code, which is inside on_tick, around line 400 or so of control.lua, was changed so it makes more sense out of its context but the logic is the same:
*snip code snap*
The math is a bit... confusing (took me an afternoon of fumbling around and testing to make sure it worked as I wanted) but what it does is it spreads out all nodes over 60 ticks as evenly as possible, as just using math.ceil(count/60) as the max would clump up most nodes at the start if the number of nodes is close to a multiple of 60.

An example of what I mean by clumping, imagine if you have to iterate 6 nodes over 5 ticks. If you used math.ceil(...) as your max you'd get 2-2-2-0-0, while this method would give you 2-1-1-1-1. If that makes sense.

'2-1-1-1-1' is exactly what my code does. Only without any of the complicated mathing you use. And less math is faster . Ofc mine only works for continuous lists. Btw if you externalize the adding and removing of list entries out of the loop you can actually make my version even shorter:

Code: Select all

for i = 1 + offset, #my_table_of_entities do stuff() end

-> removing the backwards-walking as that was only to prevent errors when removing entries inside the loop.

Also I wasn't clear on the fluctuations, they're more that every few seconds every mod gets twice as heavy ups-wise, stays that way for a few seconds more and finally it slowly dwindles to the previous ups usage. This all over twenty seconds or so. It's a rolling kind of lag, I'm unsure if %0 is to blame here!

Don't know what mods you're using (and even if i wouldn't know the internals of most of them ) but depending on how bad the lag is it might be possible that one singular mod is causing it and pulling everyone else down with it?

Yeah but you'd be giving priority to botA, wouldn't you? Wouldn't it be fair to also save some for botB kind?

No. You'd have to do this per type.

Only I'd use the estimated energy usage per 60 ticks (or whatever pooling interval i'd use) because that's the minimum energy you're estimated to recharge per bot, so you wouldn't get the case where you asked to charge 10 out of a 2000 and still had juice left over by the 10th.

Yes. So if you know you'll have no juice left for no.11 there's no point in getting that many from find_entities.

Actually, what do you think would be heavier: for area A, doing 2 find_entities_filtered for all kinds of robots, or doing n searches but one per each (valid) robot prototype? Like, you said above the biggest ups usage is creating/storing the bot entity references, while i always thought the heaviest part was the scan itself which is why I went for 2 (hoping for 1). Homework for me too I suppose.

If you don't want special treatment for any particular prototype i'd go with the least-calls-possible method. Chances are if you have 20 different prototypes that they'll never all be in the same area so you end up with empty calls most of the time. And while empty calls aren't expensive they're not free either.

So. Let me try to explain again what i meant (attention pseudocode, i don't know everything by heart ):

So first you cache the maximum charge for every prototype.
Then you find the smallest maximum charge for construction bots -> determine maximum count of Cbots that can be charged fully.
Then you find the smallest maximum charge for logistic bots. -> determine maximum count of Lbots that can be charged fully.
Scan area: local Cbots = find_entities(type='cbots',limit='cbotlimit')
Scan area: local Lbots = find_entities(type='cbots',limit='lbotlimit')

Ok. So now we have two tables that contain no more bots per type that we can charge. Keep in mind that it's highly likey that Cbots is empty most of the time. But if theoretically both tables were full we wouldn't have enough energy to charge all bots in both tables right? Because we limited it in a way that we get enough bots even if one list is empty. So we need to distribute the energy evenly. So if we have

Cbots = {C1,C2}
Lbots = {L1,L2,L3}

we can just distribute energy in a simple loop

Code: Select all

local available_energy = get_energy() * 0.7 --your stuff costs more right? so make energy less valuable for calculation purposes
local i = 1
local e,m
repeat
  e = Cbots[i].energy
  m = Cbots[i].max_energy
  t = m-e --transfered energy
  if available_energy >= t then
    Cbots[i].energy = e+t
    available_energy = available_energy - t
  else 
    Cbots[i].energy = e+available_energy
    available_energy = 0
    end
  i = i+1
  until available_energy == 0
  
  --repeat for Lbots

  end

Bear in mind that the charge() function now has to handle 'nil' as an input (because Cbots has no third entry) but that's trivial to add. Writing the code above i noticed that if you do one call per prototype you chould skip having to ask every bot for its max_energy value. But wheather that it's worth it kinda depends on how many bots you charge vs how many prototypes you have. As with infinite speed/cargo research there's not too much reason anymore to add tons of new prototypes but you tend to have thousands of bots i'd tend to say that per-prototype-call _might_ be more performance after all.

[Dustine]

Agh, I hate the quoting system.

- Opus Magnum (well, Codex, it is only a level pack) were my late childhood and a big part on getting me interested on programming so no waiting for me. No regrets either, it's all I would have wanted and more so far. And nobody out the company knew they were working on it, it was announced on this Monday to be released on Thursday. It was a really good surprise!

- It's sparse because there's a lot of times I need to quickly find a node either by the roboport/cell it targets, or by the chargers it includes (altho that second part is me not trusting circular references). Easily solved with a separate index for the on_tick iterating and leaving global.nodes unit_number indexed still, bigger savefile size or not. (so yeah i'm changing this, thank you <3) Altho that "complex" math is one extra if and modulo, I forgot I simplified the math since 0.3 (it was... much worse back then), so it's not thaaaat bad.

- I am bad at names and need better terminology for all of this.

- So it is 2 1 1 1 1, I wrongly assumed and didn't check it before accusing it of being clumping. Sorry.

- I already cache the maximum charge per bot kind so yeah, it'd simplify the code a lot if I did do a scan per robot prototype. Just still need to test if, for a modded game with tons of (active) robots of several kinds, if n prototype scans compares to 2 type scans. I will, I promise, just... Opus Magnum.

The limit issue

Okay, I feel I didn't make myself clear by what I meant by "estimated energy usage" last time, because, if I get what you're saying and I'm sorry if I didn't, using the maximum charge a robot can carry as the limit seems like a bad idea.

So, let's say every charger has 200MJ of internal storage and 24MW of maximum energy input (aka how much it can recharge its internal battery). That's the stats on version 0.4, they're being nerfed on 0.5 because they're a biiiit too high. Just a tad.

Anyway, vanilla construction robot (the more expensive of the two energy wise, because it's faster than logistic bots by 0.6 m/s and energy consumption is directly linked to the distance one travels, and both have the same internal storage) has these stats on its prototype:

Code: Select all

speed = 0.06,
max_energy = "1.5MJ",
energy_per_tick = "0.05kJ",
energy_per_move = "5kJ",
min_to_charge = 0.2,
max_to_charge = 0.95,

Note that 1.5MJ? That would mean, assuming no consumption penalties, a single charger would only have the ability to charge ~133 robots, if by internal storage alone. We're not counting the 24MW, which we probably should be, but for predictions you can't trust that refilling potential as you don't know how much the electrical network is able to give back, so for now we'll ignore it.

But, in general bots don't require to be fully charged, as they already attempt to charge if they're nearing 0.2 of their internal storage (so that increases it to ~167) and, if they were already charged before, or just came out of a roboport (with a full charge), they only need to top off whatever they used. So that would be 200/t(0.00035) robots, t being the number of ticks since the last charge (for t = 60, that's ~9500 robots). Ignoring robot speed research, that'd bring the latter number way down, at pre-space research with a +240% speed bonus that drops to ~3100 bots.

So why am I saying all of this? Because ~133 robots, your proposed search limit, pales in comparison to the average number of robots (3100-9500), so a search around the maximum number of robots that can be charged fully would reach the end of the list still with tons of energy remaining and uncharged bots. Not that using 3k-9k as smart limits is much better, not only are the limits humongous (and this is for a *single* charger! every additional charger would double robot capacity no matter what metric you're using), it of course doesn't account for newly-entered robots in a network requiring a top-up, or even doesn't count for unmoving bots (rare case, but it can happen, as when storage runs full and robots just stand in place waiting for the player to make more space for their load), which would cause the same issue as if I had used 133 robots and cut the charging up too soon.

And I mean, the "best" solution is to just run more limit-less scans if we have energy remaining by the end of the iteration but now we've doubled the number of iterations! Sorry for being hard here but I want to try to make it as good as I can without being dismissive, which I feel I'm being...

---

In the mod as it stands, all calculations are done around the 24MW limit instead, altho I now realize that may not be quite right. Still, the 24MW recharge rate is important for one thing: if any particular charger ever is using more energy than 24MW, it has to display the overtaxed warning.

Also don't take all my feet dragging as me refuting all you've been kindly suggesting and advising about, I am implementing everything I can from the past few days and I couldn't be more thankful for this. I just... don't want to break the mod functionality in the name of UPS. I'm not the best coder either so... I'm sorry for my fumbling about on that.