LuaGameScript.get_data_raw()

[Earendel]

LuaGameScript.get_data_raw() would return a copy of data.raw. This would be a copy of the same data that is available in data-final-fixes.lua, but modifying any thing in the table would not affect the game. The purpose is for control.lua to be able to access the prototype data it requires in order to do other things.

I know you can access prototype classes via LuaGameScript.entity_prototypes, LuaGameScript.item_prototypes, etc, however you can't use LuaEntityPrototype or LuaItemPrototype to get all of the information that would be available in the raw prototype definition tables.

Example: I'm trying to fire projectiles and beams from control.lua. It is going well providing I manually enter data for the projectile spawning in control.lua but it would be better to access that information from the gun, ammo, projectile, and explosion prototype definitions. Trying to find the damage that machine gun magazines do is not possible from LuaItemPrototype (even though the original definition has that data). I could start importing the various prototype/entity.lua and prototype/item.lua files into control.lua, however, that would mean that the mod would not automatically be able to support weapons or ammo from other mods. If I could load a copy of the final data.raw from LuaGameScript in the first part of control.lua's data lifecycle then I could search through the final prototype structures and get things like damage values burred 6 levels deep in tables, the type of shell particles to spawn, etc.

My immediate requirements are quite specific, but I imagine that there are many many other situations where getting an accurate copy of the raw data would be extremely useful to other mods.

[aubergine18]

If you need access to data.raw from within control.lua, you can serpent.block data.raw to the `order` property of a custom item and then loadstring it from within control.lua, like this: https://mods.factorio.com/mods/sparr/expose-data-raw

[Rseding91]

LuaItemPrototype does have every bit of information about every item. I made sure of that a few months ago. LuaEntityPrototype on the other hand does have a lot lacking.

The copying of data.raw is not going to happen. We process and change parts of it such that it isn't accurate after the game has started. Using it can work in some instances but others it can easily be wrong.

[Earendel]

If you need access to data.raw from within control.lua, you can serpent.block data.raw to the `order` property of a custom item and then loadstring it from within control.lua, like this: https://mods.factorio.com/mods/sparr/expose-data-raw

Thanks for posting that. I will probably use it but I wish there was a better way.

LuaItemPrototype does have every bit of information about every item. I made sure of that a few months ago. LuaEntityPrototype on the other hand does have a lot lacking.

If you load the item "piercing-rounds-magazine" as a LuaItemPrototype, how would you find that source_effect is "explosion-gunshot" or that the damage amount is 5?

Code: Select all

{
    type = "ammo",
    name = "piercing-rounds-magazine",
    icon = "__base__/graphics/icons/piercing-rounds-magazine.png",
    flags = {"goes-to-main-inventory"},
    ammo_type =
    {
      category = "bullet",
      action =
      {
        type = "direct",
        action_delivery =
        {
          type = "instant",
          source_effects =
          {
              type = "create-explosion",
              entity_name = "explosion-gunshot"
          },
          target_effects =
          {
            {
              type = "create-entity",
              entity_name = "explosion-hit"
            },
            {
              type = "damage",
              damage = { amount = 5 , type = "physical"}
            }
          }
        }
      }
    },
    magazine_size = 10,
    subgroup = "ammo",
    order = "a[basic-clips]-b[piercing-rounds-magazine]",
    stack_size = 100
  }

The copying of data.raw is not going to happen. We process and change parts of it such that it isn't accurate after the game has started. Using it can work in some instances but others it can easily be wrong.

Can you save a copy after all the data-final-fixes have been applied and load it if get_data_raw() is called? I'm not trying to be awkward about it, it's just that the information is so valuable.

[Rseding91]

If you load the item "piercing-rounds-magazine" as a LuaItemPrototype, how would you find that source_effect is "explosion-gunshot" or that the damage amount is 5?

http://lua-api.factorio.com/latest/LuaI ... .ammo_type and the additional property that the docs seem to miss: "action" - the action.

Can you save a copy after all the data-final-fixes have been applied and load it if get_data_raw() is called? I'm not trying to be awkward about it, it's just that the information is so valuable.

We do additional processing of the data when the prototypes are created and after every prototype is created - not related to the final version of "data.raw" so that wouldn't work.

[aubergine18]

Is it possible to find max_underground_distance value for underground pipes? Or the base character reach/build/etc distances?

[Rseding91]

Is it possible to find max_underground_distance value for underground pipes? Or the base character reach/build/etc distances?

Not currently. Entity prototypes are still messing a large amount of the properties.

[aubergine18]

entity tile_width and tile_height are another two that would be very useful to access via control.lua

[Earendel]

LuaItemPrototype does have every bit of information about every item. I made sure of that a few months ago. LuaEntityPrototype on the other hand does have a lot lacking.

The copying of data.raw is not going to happen. We process and change parts of it such that it isn't accurate after the game has started. Using it can work in some instances but others it can easily be wrong.

Hello, I'm asking for this again. I'd like to explain what happened between the last time I made this request and now, and why I think it's still a good idea.

So after hearing that copying of data.raw is not going to happen, finding that LuaItemPrototype does not contain every bit of information about every item, and finding more an more cases where it's just impossible to find entity data that I need, I took aubergine18's suggestion and started storing data in the 'order' string of dummy entities.

I ended up making the data-raw-prototypes mod: https://mods.factorio.com/mods/Earendel ... prototypes which has been unbelievably useful in accessing the necessary data. It's been used by other mod makers too, who have found it the only way to access the data they need - things that I never would have thought to ask for if I had to ask for each and every thing as it is found to be missing.

It really doesn't matter so much if the structure isn't neat, isn't processed, or doesn't have the defaults built in, because if we really needed that then we could find that stuff using the game.entity_prototypes methods (like locale). The key thing is that everything that is defined in the data.raw stage is there, regardless of whether anyone decided ahead of time if it was useful enough to include in the API.
Things like:
Car braking force
Car turning speed
Turret rotation speed
Resource infinite depletion amount
The details of layered item icons
Combinator connection distance
Finding a layered animation's component's tint or animation speed
and so on...

For a while everything was working out just fine, my mod did the job with no problems. Fast forward to 0.15 being released and suddenly the order string can only support 200 characters. That's a huge problem for data-raw-prototypes, but completely understandable change, presumably for optimisation purposes. In the rush of trying to get my mods working for 0.15 I had to split the strings into 200 long chunks and get the data in that way, which is a terrible solution, but I have not been able to find a way around yet. The bigger problem is that on heavily modded games it can cause the game to exceed the 65535 instances of type limit.

So hopefully you can see the value in providing access to data.raw from the LuaGameScript object.
Perhaps an access method like: game.get_data_raw("type", "name") would be better than my original suggestion?

Anyway I hope you'll consider this request again. Even if 95% of the missing parts were added to game.entity_prototypes and other methods, I still believe providing access to raw is worth it just for the other 5% that might get deemed too niche, or too difficult to be provided after processing, or might be delayed for weeks after being reported missing.

[Nexela]

I whole hardheartedly agree.

[Rseding91]

The details of layered item icons
Finding a layered animation's component's tint or animation speed

Curious: what would you ever use these for?

[Rseding91]

Exposing the raw Lua table is most likely never going to happen. Not only does it lack information but it will frequently be wrong compared to the actual game because of game settings and post-processing we do at load time.

I'm more than willing to add any properties to read anything the game has available simply It's not an efficient use of time to "add them all" when the vast majority would never be used.

If you asked for things or made a list I'll work on adding them.

[Earendel]

One example of getting layered icon data being useful is that I can get around the issue of an image button not being able to render a layered icon, by making a button and adding each layer as a sprite (an image button with sprite="item/crude-oil-barrel" won't work).

Knowing animation speed is useful because on car type-vehicles the animation is tied to the vehicle speed, so if you need to force the vehicle to move in order to keep the animation playing at a minimum rate you need to know the animation speed... unless there's a different way to force the animation to play? (This is for https://mods.factorio.com/mods/Earendel ... cles-miner mining animations.)

Reading the tint is to appropriately match a coloured particle/projectile effect to ammo based on the colour of the tint mask used on the ammo (if found).

We'll try and put together a list of properties that are missing and would be useful.

[Rseding91]

That icon layering problem is just a bug that I've yet to fix

Mostly because... well go look at the bug reports section. It will be fixed. Probably this weekend.

[Earendel]

Here is the list of properties on prototypes that are currently being accessed through data-raw-prototypes by my mods.

resource.normal
resource.infinite_depletion_amount
resource.minable.required_fluid
resource.minable.fluid_amount

gun.attack_parameters.sound
gun.attack_parameters.shell_particle
gun.attack_parameters.shell_particle.name
gun.attack_parameters.shell_particle.speed
gun.attack_parameters.shell_particle.starting_frame_speed
gun.attack_parameters.projectile_center
gun.attack_parameters.gun_center_shift
gun.attack_parameters.projectile_creation_distance

ammo.ammo_type -- I'm not sure how much (if any) is missing here, the API format is different and there are a lot of possibilities to check due to nesting.

car.effectivity
car.consumption
car.weight
car.friction
car.braking_power
car.tank_driving
car.rotation_speed
car.turret_rotation_speed
car.turret_animation -- specifically, car.turret_animation.layers is the best way I have found to determine that the vehicle definitely has a turret and that the turret is probably visible, since turret_rotation_speed might be set to something but have no separate turret layers. entity_prototype.turret.turret_specific_properties would be ideal.
car.guns
car.turret_animation
car.animation
car.animation.layers
car.animation.layers[].animation_speed

*.icon
*.icon[] -- getting layers of icons

Also from Picker Extended:

*.maximum_wire_distance
*.circuit_wire_max_distance

I'm not sure what other mods are using.

[sparr]

If you asked for things or made a list I'll work on adding them.

Perhaps a sticky post in the modding interface requests forum, specifically to ask for data.raw information to be added to the lua prototype classes? Could be seeded with Earendel's comment above.

[Rseding91]

gun.attack_parameters.sound
gun.attack_parameters.shell_particle
gun.attack_parameters.shell_particle.name
gun.attack_parameters.shell_particle.speed
gun.attack_parameters.shell_particle.starting_frame_speed
gun.attack_parameters.projectile_center
gun.attack_parameters.gun_center_shift
gun.attack_parameters.projectile_creation_distance

All of these already exists through http://lua-api.factorio.com/latest/LuaI ... parameters

ammo.ammo_type -- I'm not sure how much (if any) is missing here, the API format is different and there are a lot of possibilities to check due to nesting.

I mirrored the input format as far as I remember so there should be no confusion.

car.turret_animation
car.animation
car.animation.layers
car.animation.layers[].animation_speed
*.icon
*.icon[] -- getting layers of icons

What use could those be? We don't support rendering specific images so I'm having a hard time trying to figure out what they would ever be useful for.

[sparr]

car.turret_animation
car.animation
car.animation.layers
car.animation.layers[].animation_speed
*.icon
*.icon[] -- getting layers of icons

What use could those be? We don't support rendering specific images so I'm having a hard time trying to figure out what they would ever be useful for.

Knowing animation speed is useful because on car type-vehicles the animation is tied to the vehicle speed, so if you need to force the vehicle to move in order to keep the animation playing at a minimum rate you need to know the animation speed... unless there's a different way to force the animation to play? (This is for https://mods.factorio.com/mods/Earendel ... cles-miner mining animations.)

[Rseding91]

If you asked for things or made a list I'll work on adding them.

Perhaps a sticky post in the modding interface requests forum, specifically to ask for data.raw information to be added to the lua prototype classes? Could be seeded with Earendel's comment above.

I like it. I'll make one shortly.

[sparr]

If you asked for things or made a list I'll work on adding them.

Perhaps a sticky post in the modding interface requests forum, specifically to ask for data.raw information to be added to the lua prototype classes? Could be seeded with Earendel's comment above.

I like it. I'll make one shortly.

Cool! Be sure to let ssilk know about it, as a move/merge target for future such one-off posts.