factorio-recipe-analyzer/README.org

376 lines
14 KiB
Org Mode
Raw Normal View History

2023-08-22 21:39:47 +02:00
#+title: Factorio Recipe Analyzer
#+author: Phil Bajsicki
2023-08-23 12:02:02 +02:00
#+PROPERTY: header-args :tangle fra.py
2023-08-22 21:39:47 +02:00
* Intro
2023-08-23 12:02:02 +02:00
This is a simple Python script which generates an analysis of each recipe into its component parts, and delivers insight into the balance and progression of a mod.
2023-08-22 21:39:47 +02:00
The .csv files used as input are generated in Factorio, by loading a new game with only ~base~ and your chosen mod enabled, and running the following Lua code:
#+name: Lua code generating a .csv file containing all recipes using the items in the whitelist.
2023-08-22 21:39:47 +02:00
#+begin_src lua :tangle no
/c
local whitelist = {}
for _, k in pairs({
"iron-plate",
"copper-plate",
"steel-plate",
"copper-wire",
"iron-gear-wheel",
"iron-stick",
"pipe",
}) do whitelist[k] = true end
local parts = {}
for name, recipe in pairs(game.recipe_prototypes) do
local history = script.get_prototype_history("recipe", name)
if history.created == "base" then
local add = false
local ingredients = {}
for _, ingredient in pairs(recipe.ingredients) do
if whitelist[ingredient.name] then add = true end
ingredients[#ingredients+1] = ingredient.amount .. "," .. ingredient.name
2023-08-22 21:39:47 +02:00
end
if add then
local item = game.item_prototypes[name] or game.fluid_prototypes[name]
parts[#parts+1] = "," .. name .. "," .. item.subgroup.name .. "," .. table.concat(ingredients, ",")
2023-08-22 21:39:47 +02:00
end
end
end
game.write_file("recipes.csv", table.concat(parts, "\n"), false)
#+end_src
On the other hand, this script outputs ALL available recipes in an org-readable format, for easy overview, sorting, and insight.
#+name: lua-to-org: Lua script generating an org-mode file containing all available recipes.
#+begin_src lua :tangle no
/c
local recipes = {}
for name, recipe in pairs(game.recipe_prototypes) do
local ingredients = {}
for _, ingredient in pairs(recipe.ingredients) do
ingredients[#ingredients+1] = ingredient.name .. "," .. ingredient.amount
end
local products = {}
for _, product in pairs(recipe.products) do
local amount = product.amount or product.amount_min .. "-" .. product.amount_max
products[#products+1] = product.name .. "," .. amount
end
recipes[#recipes+1] = "\n* " .. recipe.subgroup.name .. "\n** " .. name .. "\n*** products" .. "\n- " .. table.concat(products, "\n- ") .. "\n*** ingredients" .. "\n- " .. table.concat(ingredients, "\n- ")
end
game.write_file("recipes.org", table.concat(recipes, "\n"), false)
#+end_src
The output of this is an org-mode file in the following pattern:
#+name: Example output of the lua-to-org script.
#+begin_example
,* recipe.subgroup
,** recipe.name
,*** products
- product1
- product2
- ...
,*** ingredients
- ingredient1
- ingredient2
- ...
#+end_example
2023-08-23 16:01:53 +02:00
* On the script
2023-08-22 21:39:47 +02:00
2023-08-23 12:02:02 +02:00
This is a literate script. The source code is embedded in these code blocks, and tangled into the Python script using org-babel. This allows me to write a description of what I want to do, and comment on it without resorting to ~// /* */~ ugly comments.
2023-08-22 21:39:47 +02:00
Better yet, I can include [[Lua code generating a .csv file containing all recipes using the items in the whitelist.][links directly to places within the file]] and easily cross-reference what I'm doing with the documentation/ design.
2023-08-22 21:39:47 +02:00
If you open this org file raw (e.g. by clicking [[https://git.bajsicki.com/phil/factorio-recipe-analyzer/raw/branch/main/README.org][here]]), you will see that there are a number of different blocks.
There is a rough overview of the structure:
#+begin_example
,#+name: Name of the following source block.
2023-08-23 12:02:02 +02:00
,#+begin_src python :tangle file.name
[code goes here]
,#+end_src
#+end_example
~:tangle~ defines the file into which the code block will be passed to ~org-babel-tangle~. To disable, pass ~no~.
In the begging of the file, we can set global (in the scope of the file) properties, such as the default target for ~:tangle~:
2023-08-23 12:02:02 +02:00
~#+PROPERTY: header-args :tangle fra.py~
The easiest way to generate outputs from here is to open this file in Emacs, and run ~M-x org-babel-tangle~. If you're using [[https://github.com/doomemacs/doomemacs][Doom Emacs]], the default key binding is ~C-c C-v C-t~.
*Important note:* the files in ~./mod-recipes~ are working files. They're not intrinsic parts of the software - they're there primarily for testing and development.
2023-08-22 21:39:47 +02:00
* License
2023-08-23 12:02:02 +02:00
I don't own the source csv files generated by Factorio, nor the mods the script is pulling from. The Python script itself is GPLv3, with the exception of third-party libraries licensed otherwise.
2023-08-22 21:39:47 +02:00
** TODO: include GPLv3 in the repo
** Credits:
The following are mods from which .csv files have been generated. The recipe .csv files are included in the ~mod-recipes~ directory.
* The Code1
** Idea:
Create an org-mode file with all the calculations included, and complete, for a clear overview of recipe progression, subgroups, item inputs and outputs, and total cost.
2023-08-22 21:39:47 +02:00
** Setup
** Open csv file
** Open (create if needed) output csv
pseudocode is in Lisp./
2023-08-22 21:39:47 +02:00
For this, we'll likely want to include some metadata, like creation date, mod name, number of ingredients, maybe the total amount of raw mats needed to make one of everything?
** Make a list of products
That's the first column in the .csv file.
Read first column of the csv file and insert it into the .org (output) file.
*** For each product, create a templated section
Ideally we'd end up with Something like:
#+name: Example structure of a section of the output file.
2023-08-22 21:39:47 +02:00
#+begin_example
,* product
,** direct inputs
- input and amount
- input and amount
,** raw ingredients
+ raw ingredient and amount
+ raw ingredient and amount
#+end_example
**** Parse the csv file:
For each line:
1. First column becomes the top header
2. Insert second header
3. Insert each ingredient and its amount as a separate item
**** Parse the output.org file, filling it out recursively
1. Open .org (output file)
2. Loop over output.org:
1. Find *product section.*
2. Find (next) ingredient lines in this product section.
3. Pass the product and each direct input item and its number to ~raw-ingredients~. /(We can distinguish direct inputs from raw ingredients easily because org-mode supports multiple characters for defining lists. So we can just look for lines beginning with ~-~ and not really think about anything else.)/
This would look something like:
#+name: Example call to the raw-ingredients function.
#+begin_example
(raw-ingredients iron-gear-wheel iron-plate 2)
(raw-ingredients yellow-belt iron-plate 1)
#+end_example
2023-08-22 21:39:47 +02:00
3. ~(raw-ingredients (product item number))~:
1. Store in variables for clarity:
- product
- item
- number
2. Read .csv file:
1. If a recipe for this item exists in the csv file:
1. Go to the line with the recipe (first column.
2. For each ~(item number)~ pair, call ~(raw-ingredients (product new-item (* new-number old-number)))~.
3. If a recipe does not exist:
1. Find ~* product~ section in the .org file.
2. If the ingredient item already exists:
1. Add new number we just got to the existing number.
3. Else: write new raw ingredient line and number in this section.
* The Code 2
** Idea/ outline
This is not for analysis as much as helping Galdoc out with creating compat layers for [[https://github.com/Orion351/galdocs_manufacturing][Galdoc's Manufacturing.]]
Take above csv data dump from Factorio, then:
1. Figure out appropriate categories. This is the starting point we have:
- Telescoping, (inserters, belts, things that reach)
- Metalworking,
- Plastic,
- Wood,
- Stone,
- Glass,
- Electronics.
- And possibly in the future:
- Motors,
- Agriculture,
- Chemicals,
- Small Arms / Equipment.
This has to be done manually. The csv file already includes an empty first column, which lets us manually go over it and add the tags to each item.
2. Then pull the output template CSV file, which should have the following structure:
#+name: Example csv template for code-2
#+begin_src csv :tangle no
item-category, item-name, amt1, catitem1, amt2, catitem2, amt3, catitem3, amt4, catitem4, ...
#+end_src
In this example, we create an arbitrary number of columns, based on the largest number of ingredients a recipe requires from each category.
For example, stack inserters would look like so:
#+begin_src csv :tangle no
item-category, item-name, amt1, telescoping1, amt2, metal2, amt3, electronics3, amt4, electronics4
telescoping, stack-inserter, 1, fast-inserter, 15, iron-gear-wheel, 15, electronic-circuit, 1, advanced-circuit
#+end_src
This structure allows for unambiguous selection of the relevant data from the csv file, for the following reasons:
1. Amounts and categories are paired by the matching number at the end of the column name.
2. These cannot be confused with the amounts themselves, because the column names include [a-zA-Z] characters.
3. The first column makes it easy to find whether an item belongs to a particular column or not.
4. Additionally, this way we can automate creating the output .csv template, since we can check what number of columns we need for each ingredient category.
2023-08-23 16:01:53 +02:00
** Testing process 1:
*** Get csv from Factorio
This needs to be run manually rn, will figure out an automatic way later? Maybe?
#+name: lua-to-org: Lua script generating a csv file containing all available recipes and their ingredients.
#+begin_src lua :tangle no
/c
local recipes = {}
for name, recipe in pairs(game.recipe_prototypes) do
local ingredients = {}
for _, ingredient in pairs(recipe.ingredients) do
ingredients[#ingredients+1] = ingredient.amount .. "," .. ingredient.name
end
recipes[#recipes+1] = "," .. name .. "," .. recipe.subgroup.name .. "," .. table.concat(ingredients, ",")
end
game.write_file("recipes-all.csv", table.concat(recipes, "\n"), false)
#+end_src
*** Move csv to the right directory
/Note:/ If you have this open in Emacs, move your cursor into the code block and hit ~C-c C-c~ to execute the code. It's very handy if you have commands that you use often.
#+name: move the output from the above command to the mod-recipes directory for easy access.
#+begin_src sh :results none
mv ~/.factorio/script-output/recipes-all.csv ./mod-recipes
#+end_src
*** Sort for easier human readability
**** Sort the csv file by recipe subgroups
This allows us to find groups of related processes easier.
#+begin_src sh :results none
cat ./mod-recipes/recipes-all.csv | sort -k3 -t, | column --table -s, -o, > ./mod-recipes/recipes-all-sorted-subgroups.csv
#+end_src
**** Sort the csv file by product name
This allows us to find similarly named items easier
#+begin_src sh :results none
cat ./mod-recipes/recipes-all.csv | sort -k2 -t, | column --table -s, -o, > ./mod-recipes/recipes-all-sorted-product.csv
#+end_src
This is the file we'll be using for further testing, because it groups up a lot of items by material, which is what we want.
For this reason, let's make ourselves a working copy;
#+begin_src sh :results none
cp ./mod-recipes/recipes-all-sorted-product.csv ./mod-recipes/intermediate.csv
#+end_src
From here on, all changes will take place on the basis of the ~intermediate.csv~ file, and outputs will be directed to ~output.csv~. This will prevent time loss in case of a mistyped command.
*** Define raw ingredients by their category
2023-08-23 17:02:15 +02:00
**** We have the following categories we'll be assigning:
2023-08-23 16:01:53 +02:00
- Telescoping, (inserters, belts, things that reach)
- Metalworking,
- Plastic,
- Wood,
- Stone,
- Glass,
- Electronics.
At the same time, we have 2843 entries to deal with. That's a lot, so let's start with the raw ingredients, and assign those using the following rough rules:
For this reason, it is important that we first cover the base materials.
1. If the item exists in vanilla as a raw ingredient, it should be treated as such in the compatibility layer as well. If it is made craftable by a mod, its ingredients must also be tagged as raw materials.
2. The ingredient in question may not have a recipe of its own. E.g. iron ore in vanilla Factorio, or stiratite in AngelBob. In this case, it is a raw ingredient.
**** Metals
The following code should:
1. Find all the items which stem from iron ore *only*.
2. Add 'metal' to the first field of their line in the CSV.
To do this, we'll start with something like this:
#+begin_src perl :tangle fra.pl
open(my $in, "<", "./mod-recipes/intermediate.csv") or die "Can't open intermediates.csv";
open(my $out, ">", "./mod-recipes/output.csv") or die "Can't open output.csv";
2023-08-23 17:02:15 +02:00
my @patterns = (
# 0 metal
"m/[\-?,?s*]
(bronze|iron|steel|copper|\
zinc| tungsten|titanium|tin|\
nickel|silver|platinum|manganese|\
lead|gold|aluminium|aluminum)\
[\-?,?\s*]/",
# 1 wood
"m/[\-?,?s*]
(bronze|iron|steel|copper|\
zinc| tungsten|titanium|tin|\
nickel|silver|platinum|manganese|\
lead|gold|aluminium|aluminum)\
[\-?,?\s*]/",
# 2 plastic
"m/[\-?,?s*]
(bronze|iron|steel|copper|\
zinc| tungsten|titanium|tin|\
nickel|silver|platinum|manganese|\
lead|gold|aluminium|aluminum)\
[\-?,?\s*]/",
# 3 electronics
"m/[\-?,?s*]
(bronze|iron|steel|copper|\
zinc| tungsten|titanium|tin|\
nickel|silver|platinum|manganese|\
lead|gold|aluminium|aluminum)\
[\-?,?\s*]/",
# 4 glass
"m/[\-?,?s*]
(bronze|iron|steel|copper|\
zinc| tungsten|titanium|tin|\
nickel|silver|platinum|manganese|\
lead|gold|aluminium|aluminum)\
[\-?,?\s*]/",
# 5 stone
"m/[\-?,?s*]
(bronze|iron|steel|copper|\
zinc| tungsten|titanium|tin|\
nickel|silver|platinum|manganese|\
lead|gold|aluminium|aluminum)\
[\-?,?\s*]/");
2023-08-23 16:01:53 +02:00
while (<$in>) {
@line = split(",", $_);
2023-08-23 17:02:15 +02:00
if ($line[1] =~ "$patterns[0]") {
print $out "metal @line";
}
if ($line[1] =~ "$patterns[1]") {
print $out "wood @line";
}
if ($line[1] =~ "$patterns[2]") {
print $out "plastic @line";
2023-08-23 16:01:53 +02:00
}
2023-08-23 17:02:15 +02:00
if ($line[1] =~ "$patterns[3]") {
print $out "electronics @line";
}
if ($line[1] =~ "$patterns[4]") {
print $out "glass @line";
}
if ($line[1] =~ "$patterns[5]") {
print $out "stone @line";
}
else {
print $out "@line";
}
}
2023-08-23 16:01:53 +02:00
}
close $in or die "$in: $!";
close $out or die "$out: $!";
#+end_src
#+RESULTS: