453 lines
19 KiB
Org Mode
453 lines
19 KiB
Org Mode
#+title: Factorio Recipe Analyzer
|
|
#+author: Phil Bajsicki
|
|
#+PROPERTY: header-args :tangle fra.py
|
|
* Intro
|
|
This is a simple Python script which generates an analysis of each recipe into its component parts, and delivers insight into the balance and progression of a mod.
|
|
|
|
The .csv files used as input are generated in Factorio, by loading a new game with only ~base~ and your chosen mod enabled, and running the following Lua code:
|
|
|
|
#+name: Lua code generating a .csv file containing all recipes using the items in the whitelist.
|
|
#+begin_src lua :tangle no
|
|
/c
|
|
local whitelist = {}
|
|
for _, k in pairs({
|
|
"iron-plate",
|
|
"copper-plate",
|
|
"steel-plate",
|
|
"copper-wire",
|
|
"iron-gear-wheel",
|
|
"iron-stick",
|
|
"pipe",
|
|
}) do whitelist[k] = true end
|
|
|
|
local parts = {}
|
|
for name, recipe in pairs(game.recipe_prototypes) do
|
|
local history = script.get_prototype_history("recipe", name)
|
|
if history.created == "base" then
|
|
local add = false
|
|
local ingredients = {}
|
|
for _, ingredient in pairs(recipe.ingredients) do
|
|
if whitelist[ingredient.name] then add = true end
|
|
ingredients[#ingredients+1] = ingredient.amount .. "," .. ingredient.name
|
|
end
|
|
if add then
|
|
local item = game.item_prototypes[name] or game.fluid_prototypes[name]
|
|
parts[#parts+1] = "," .. name .. "," .. item.subgroup.name .. "," .. table.concat(ingredients, ",")
|
|
end
|
|
end
|
|
end
|
|
game.write_file("recipes.csv", table.concat(parts, "\n"), false)
|
|
#+end_src
|
|
|
|
On the other hand, this script outputs ALL available recipes in an org-readable format, for easy overview, sorting, and insight.
|
|
|
|
#+name: lua-to-org: Lua script generating an org-mode file containing all available recipes.
|
|
#+begin_src lua :tangle no
|
|
/c
|
|
local recipes = {}
|
|
for name, recipe in pairs(game.recipe_prototypes) do
|
|
local ingredients = {}
|
|
for _, ingredient in pairs(recipe.ingredients) do
|
|
ingredients[#ingredients+1] = ingredient.name .. "," .. ingredient.amount
|
|
end
|
|
local products = {}
|
|
for _, product in pairs(recipe.products) do
|
|
local amount = product.amount or product.amount_min .. "-" .. product.amount_max
|
|
products[#products+1] = product.name .. "," .. amount
|
|
end
|
|
recipes[#recipes+1] = "\n* " .. recipe.subgroup.name .. "\n** " .. name .. "\n*** products" .. "\n- " .. table.concat(products, "\n- ") .. "\n*** ingredients" .. "\n- " .. table.concat(ingredients, "\n- ")
|
|
end
|
|
game.write_file("recipes.org", table.concat(recipes, "\n"), false)
|
|
#+end_src
|
|
|
|
The output of this is an org-mode file in the following pattern:
|
|
#+name: Example output of the lua-to-org script.
|
|
#+begin_example
|
|
,* recipe.subgroup
|
|
,** recipe.name
|
|
,*** products
|
|
- product1
|
|
- product2
|
|
- ...
|
|
,*** ingredients
|
|
- ingredient1
|
|
- ingredient2
|
|
- ...
|
|
#+end_example
|
|
|
|
|
|
* On the script
|
|
|
|
This is a literate script. The source code is embedded in these code blocks, and tangled into the Python script using org-babel. This allows me to write a description of what I want to do, and comment on it without resorting to ~// /* */~ ugly comments.
|
|
|
|
Better yet, I can include [[Lua code generating a .csv file containing all recipes using the items in the whitelist.][links directly to places within the file]] and easily cross-reference what I'm doing with the documentation/ design.
|
|
|
|
If you open this org file raw (e.g. by clicking [[https://git.bajsicki.com/phil/factorio-recipe-analyzer/raw/branch/main/README.org][here]]), you will see that there are a number of different blocks.
|
|
|
|
There is a rough overview of the structure:
|
|
|
|
#+begin_example
|
|
,#+name: Name of the following source block.
|
|
,#+begin_src python :tangle file.name
|
|
[code goes here]
|
|
,#+end_src
|
|
#+end_example
|
|
|
|
~:tangle~ defines the file into which the code block will be passed to ~org-babel-tangle~. To disable, pass ~no~.
|
|
|
|
In the begging of the file, we can set global (in the scope of the file) properties, such as the default target for ~:tangle~:
|
|
|
|
~#+PROPERTY: header-args :tangle fra.py~
|
|
|
|
The easiest way to generate outputs from here is to open this file in Emacs, and run ~M-x org-babel-tangle~. If you're using [[https://github.com/doomemacs/doomemacs][Doom Emacs]], the default key binding is ~C-c C-v C-t~.
|
|
|
|
*Important note:* the files in ~./mod-recipes~ are working files. They're not intrinsic parts of the software - they're there primarily for testing and development.
|
|
|
|
* License
|
|
I don't own the source csv files generated by Factorio, nor the mods the script is pulling from. The Python script itself is GPLv3, with the exception of third-party libraries licensed otherwise.
|
|
** TODO: include GPLv3 in the repo
|
|
** Credits:
|
|
The following are mods from which .csv files have been generated. The recipe .csv files are included in the ~mod-recipes~ directory.
|
|
|
|
* The Code1
|
|
** Idea:
|
|
Create an org-mode file with all the calculations included, and complete, for a clear overview of recipe progression, subgroups, item inputs and outputs, and total cost.
|
|
** Setup
|
|
** Open csv file
|
|
** Open (create if needed) output csv
|
|
pseudocode is in Lisp./
|
|
|
|
For this, we'll likely want to include some metadata, like creation date, mod name, number of ingredients, maybe the total amount of raw mats needed to make one of everything?
|
|
** Make a list of products
|
|
That's the first column in the .csv file.
|
|
Read first column of the csv file and insert it into the .org (output) file.
|
|
*** For each product, create a templated section
|
|
Ideally we'd end up with Something like:
|
|
#+name: Example structure of a section of the output file.
|
|
#+begin_example
|
|
,* product
|
|
,** direct inputs
|
|
- input and amount
|
|
- input and amount
|
|
,** raw ingredients
|
|
+ raw ingredient and amount
|
|
+ raw ingredient and amount
|
|
#+end_example
|
|
|
|
|
|
**** Parse the csv file:
|
|
For each line:
|
|
1. First column becomes the top header
|
|
2. Insert second header
|
|
3. Insert each ingredient and its amount as a separate item
|
|
**** Parse the output.org file, filling it out recursively
|
|
/Note: copied over to another section, this stays here for later./
|
|
1. Open .org (output file)
|
|
2. Loop over output.org:
|
|
1. Find *product section.*
|
|
2. Find (next) ingredient lines in this product section.
|
|
3. Pass the product and each direct input item and its number to ~raw-ingredients~. /(We can distinguish direct inputs from raw ingredients easily because org-mode supports multiple characters for defining lists. So we can just look for lines beginning with ~-~ and not really think about anything else.)/
|
|
|
|
This would look something like:
|
|
|
|
#+name: Example call to the raw-ingredients function.
|
|
#+begin_example
|
|
(raw-ingredients iron-gear-wheel iron-plate 2)
|
|
(raw-ingredients yellow-belt iron-plate 1)
|
|
#+end_example
|
|
|
|
3. ~(raw-ingredients (product item number))~:
|
|
1. Store in variables for clarity:
|
|
- product
|
|
- item
|
|
- number
|
|
2. Read .csv file:
|
|
1. If a recipe for this item exists in the csv file:
|
|
1. Go to the line with the recipe (first column.
|
|
2. For each ~(item number)~ pair, call ~(raw-ingredients (product new-item (* new-number old-number)))~.
|
|
3. If a recipe does not exist:
|
|
1. Find ~* product~ section in the .org file.
|
|
2. If the ingredient item already exists:
|
|
1. Add new number we just got to the existing number.
|
|
3. Else: write new raw ingredient line and number in this section.
|
|
|
|
* The Code 2
|
|
** Idea/ outline
|
|
This is not for analysis as much as helping Galdoc out with creating compat layers for [[https://github.com/Orion351/galdocs_manufacturing][Galdoc's Manufacturing.]]
|
|
|
|
Take above csv data dump from Factorio, then:
|
|
1. Figure out appropriate categories. This is the starting point we have:
|
|
- Telescoping, (inserters, belts, things that reach)
|
|
- Metalworking,
|
|
- Plastic,
|
|
- Wood,
|
|
- Stone,
|
|
- Glass,
|
|
- Electronics.
|
|
- And possibly in the future:
|
|
- Motors,
|
|
- Agriculture,
|
|
- Chemicals,
|
|
- Small Arms / Equipment.
|
|
|
|
This has to be done manually. The csv file already includes an empty first column, which lets us manually go over it and add the tags to each item.
|
|
|
|
2. Then pull the output template CSV file, which should have the following structure:
|
|
#+name: Example csv template for code-2
|
|
#+begin_src csv :tangle no
|
|
item-category, item-name, amt1, catitem1, amt2, catitem2, amt3, catitem3, amt4, catitem4, ...
|
|
#+end_src
|
|
|
|
In this example, we create an arbitrary number of columns, based on the largest number of ingredients a recipe requires from each category.
|
|
|
|
For example, stack inserters would look like so:
|
|
|
|
#+begin_src csv :tangle no
|
|
item-category, item-name, amt1, telescoping1, amt2, metal2, amt3, electronics3, amt4, electronics4
|
|
telescoping, stack-inserter, 1, fast-inserter, 15, iron-gear-wheel, 15, electronic-circuit, 1, advanced-circuit
|
|
#+end_src
|
|
|
|
This structure allows for unambiguous selection of the relevant data from the csv file, for the following reasons:
|
|
1. Amounts and categories are paired by the matching number at the end of the column name.
|
|
2. These cannot be confused with the amounts themselves, because the column names include [a-zA-Z] characters.
|
|
3. The first column makes it easy to find whether an item belongs to a particular column or not.
|
|
4. Additionally, this way we can automate creating the output .csv template, since we can check what number of columns we need for each ingredient category.
|
|
** Testing process 1:
|
|
*** Get csv from Factorio
|
|
This needs to be run manually rn, will figure out an automatic way later? Maybe?
|
|
|
|
#+name: lua-to-org: Lua script generating a csv file containing all available recipes and their ingredients.
|
|
#+begin_src lua :tangle no
|
|
/c
|
|
local recipes = {}
|
|
for name, recipe in pairs(game.recipe_prototypes) do
|
|
local products = {}
|
|
local ingredients = {}
|
|
for _, product in pairs(recipe.products) do
|
|
local amount = product.amount or product.amount_min .. "-" .. product.amount_max
|
|
products[#products+1] = product.name .. ">" .. amount
|
|
end
|
|
for _, ingredient in pairs(recipe.ingredients) do
|
|
ingredients[#ingredients+1] = ingredient.name .. "<" .. ingredient.amount
|
|
end
|
|
recipes[#recipes+1] = "," .. table.concat(products, "+") .. "," .. name .. "," .. recipe.subgroup.name .. "," .. table.concat(ingredients, "+")
|
|
end
|
|
game.write_file("products-all.csv", table.concat(recipes, "\n"), false)
|
|
#+end_src
|
|
|
|
This outputs the following format of csvfile:
|
|
|
|
#+begin_example
|
|
prod1>amt1+prod2>amt2, recipe.name, recipe.subgroup, ingr1<amt1+ingr2<amt2...
|
|
#+end_example
|
|
|
|
|
|
#+RESULTS: lua-to-org: Lua script generating a csv file containing all available recipes and their ingredients.
|
|
*** Example outputs that we could use:
|
|
**** Sorted by recipes:
|
|
| Galdoc's groups | recipe-name | subgroup | prod1 | prod2 | prod... | ingr1 | ingr2 | ingr... | | | |
|
|
|-----------------+-------------+----------+--------------+-------+---------+------------+-------+---------+---+---+---|
|
|
| metal | iron-plate | | iron-plate:1 | | | iron-ore:1 | | | | | |
|
|
| wood | | | | | | | | | | | |
|
|
| plastic | | | | | | | | | | | |
|
|
| glass | | | | | | | | | | | |
|
|
| stone | | | | | | | | | | | |
|
|
**** Sorted by products:
|
|
- One line per product, each product getting its own copy of all the recipes that make it (gg slag).
|
|
- Each product gets assigned a clear category. No issues, no conflicts.
|
|
- Problem: differing amounts between the recipes, so amounts need to be stated in a sep. column.
|
|
- If we use an unusual separator symbol for fields with more than one value (e.g. long lists of ingredients), we can then parse them very easily and remove ambiguity with a simple [whitespace, comma or +] test. Then situations like 'tin' being found in 'tinned wire' won't be a problem.
|
|
|
|
|
|
| Galdoc's groups | product | amount | subgroup | recipe | required building | | ingr1 | ingr2 | ingr... | | | |
|
|
|-----------------+------------+--------+----------+------------+----------------------------------------------+---+------------+-------+---------+---+---+---|
|
|
| metal | iron-plate | 1 | | iron-plate | stone-furnace+steel-furnace+electric-furnace | | iron-ore:1 | | | | | |
|
|
| wood | | | | | | | | | | | | |
|
|
| plastic | | | | | | | | | | | | |
|
|
| glass | | | | | | | | | | | | |
|
|
| stone | | | | | | | | | | | | |
|
|
|
|
|
|
*** Move csv to the right directory
|
|
/Note:/ If you have this open in Emacs, move your cursor into the code block and hit ~C-c C-c~ to execute the code. It's very handy if you have commands that you use often.
|
|
#+name: move the output from the above command to the mod-recipes directory for easy access.
|
|
#+begin_src sh :results none
|
|
mv ~/.factorio/script-output/products-all.csv ./mod-recipes
|
|
#+end_src
|
|
*** Sort for easier human readability
|
|
**** Sort the csv file by recipe subgroups
|
|
This allows us to find groups of related processes easier.
|
|
#+begin_src sh :results none
|
|
cat ./mod-recipes/recipes-all.csv | sort -k3 -t, | column --table -s, -o, > ./mod-recipes/recipes-all-sorted-subgroups.csv
|
|
#+end_src
|
|
|
|
**** Sort the csv file by product name
|
|
This allows us to find similarly named items easier
|
|
#+begin_src sh :results none
|
|
cat ./mod-recipes/products-all.csv | sort -k2 -t, | column --table -s, -o, > ./mod-recipes/recipes-all-sorted-product.csv
|
|
#+end_src
|
|
|
|
This is the file we'll be using for further testing, because it groups up a lot of items by material, which is what we want.
|
|
|
|
For this reason, let's make ourselves a working copy;
|
|
|
|
#+begin_src sh :results none
|
|
cp ./mod-recipes/recipes-all-sorted-product.csv ./mod-recipes/intermediate.csv
|
|
#+end_src
|
|
|
|
From here on, all changes will take place on the basis of the ~intermediate.csv~ file, and outputs will be directed to ~output.csv~. This will prevent time loss in case of a mistyped command.
|
|
*** Parse the intermetiate.csv file, filling it out recursively
|
|
|
|
|
|
#+begin_src perl :tangle raw.pl
|
|
open(my $in, "<", "./mod-recipes/intermediate.csv") or die "Can't open intermediates.csv";
|
|
open(my $out, ">", "./mod-recipes/raw.csv") or die "Can't open raw.csv";
|
|
local $| = 1;
|
|
raw_ingredients();
|
|
|
|
sub raw_ingredients () {
|
|
$product = $ARGV[1];
|
|
while (<$in>){
|
|
@line = split(",", $_);
|
|
if ($line[1] == $product) {
|
|
print "ingredients $line[4]\n";
|
|
@ingredients = split(/\+/, $line[4]);
|
|
print "\t\@ingredients: @ingredients \n";
|
|
foreach my $ingredient (@ingredients) {
|
|
print "\t\t\$ingredient: $ingredient\n";
|
|
raw_ingredients($ingredient);
|
|
print "\t\t\trecurring! \$ingredient = $ingredient\n";
|
|
}
|
|
}
|
|
else {
|
|
continue;
|
|
}
|
|
}
|
|
}
|
|
|
|
#+end_src
|
|
|
|
*** Define raw ingredients by their category
|
|
|
|
**** We have the following categories we'll be assigning:
|
|
- Telescoping, (inserters, belts, things that reach)
|
|
- Metalworking,
|
|
- Plastic,
|
|
- Wood,
|
|
- Stone,
|
|
- Glass,
|
|
- Electronics,
|
|
|
|
At the same time, we have 2843 entries to deal with. That's a lot, so let's start with the raw ingredients, and assign those using the following rough rules:
|
|
For this reason, it is important that we first cover the base materials.
|
|
|
|
1. If the item exists in vanilla as a raw ingredient, it should be treated as such in the compatibility layer as well. If it is made craftable by a mod, its ingredients must also be tagged as raw materials.
|
|
2. The ingredient in question may not have a recipe of its own. E.g. iron ore in vanilla Factorio, or stiratite in AngelBob. In this case, it is a raw ingredient.
|
|
|
|
**** Temp code:
|
|
|
|
|
|
#+begin_src perl :tangle fra.pl
|
|
|
|
our @patterns = (
|
|
|
|
# 0 metal
|
|
|
|
"m/[\-?,?s*]
|
|
(bronze|iron|steel|copper|
|
|
zinc|lithium|tungsten|titanium|tin|
|
|
nickel|silver|platinum|manganese|
|
|
lead|gold|aluminium|aluminum)
|
|
[\-?,?\s*]/",
|
|
|
|
# 1 wood
|
|
|
|
"m/[\-?,?s*]
|
|
(seedlings|wood|wooden)
|
|
[\-?,?\s*]/",
|
|
|
|
# 2 plastic
|
|
|
|
"m/[\-?,?s*]
|
|
(plastic)
|
|
[\-?,?\s*]/",
|
|
|
|
# 3 electronics
|
|
|
|
"m/[\-?,?s*]
|
|
(electronic)
|
|
[\-?,?\s*]/",
|
|
|
|
# 4 glass
|
|
|
|
"m/[\-?,?s*]
|
|
(glass)
|
|
[\-?,?\s*]/",
|
|
|
|
# 5 stone
|
|
|
|
"m/[\-?,?s*]
|
|
(limestone|stone|slag|concrete|sand)
|
|
[\-?,?\s*]/");
|
|
|
|
open(my $in, "<", "./mod-recipes/intermediate.csv") or die "Can't open intermediates.csv";
|
|
open(my $out, ">", "./mod-recipes/output.csv") or die "Can't open output.csv";
|
|
|
|
|
|
while (<$in>) {
|
|
my @line = split(",", $_);
|
|
|
|
if ($line[1] =~ "$patterns[0]") {
|
|
print $out "metal @line";
|
|
print "$patterns[0]";
|
|
print "metal @line";
|
|
}
|
|
elsif ($line[1] =~ "$patterns[1]") {
|
|
print $out "wood @line";
|
|
}
|
|
elsif ($line[1] =~ "$patterns[2]") {
|
|
print $out "plastic @line";
|
|
}
|
|
elsif ($line[1] =~ "$patterns[3]") {
|
|
print $out "electronics @line";
|
|
}
|
|
elsif ($line[1] =~ "$patterns[4]") {
|
|
print $out "glass @line";
|
|
}
|
|
elsif ($line[1] =~ "$patterns[5]") {
|
|
print $out "stone @line";
|
|
}
|
|
else {
|
|
print $out "@line";
|
|
print "@line";
|
|
}
|
|
}
|
|
close $in or die "$in: $!";
|
|
close $out or die "$out: $!";
|
|
#+end_src
|
|
|
|
#+RESULTS:
|
|
|
|
* New Idea (parsing lua files directly):
|
|
https://wiki.factorio.com/Console#Access_a_mod's_data
|
|
|
|
- It may be possible to access lua directly and pull data from there.
|
|
- It may be more effective and accurate than pulling data from an exported CSV.
|
|
- Further investigation needed.
|
|
- *The key problem:* it would be very challenging for a human/ person to make modifications to this export as-is. It's then necessary to create another layer of processing to make the output human-readable.
|
|
- ON the other hand, we don't yet have a system for importing recipes, items, and tech levels from a csv into Factorio mods...
|
|
- That may be something to explore to better define the constraints on this project.
|
|
|
|
E.g. If it were possible to build the Lua files programmatically from user-created CSVs... Factorio modding would become significantly more accessible and easy for a ton of people. Of course, this would not be as powerful as diving into the code, however... I do see potential here with regards to large overhaul mods.
|
|
|
|
Imagine:
|
|
1. Export factorio base to csv.
|
|
2. Change csv however you like, adding items, changing values, adding recipes, technologies, etc.
|
|
3. Run a script which compiles your csv into a mod
|
|
4. Have your mod running.
|
|
|
|
There are several issues which bear addressing:
|
|
1. Icons - in the case of a lack of icons, the script should default to something, so the GUI isn't broken.
|
|
2. The script would have to dynamically generate a lot of lua code, or at least wrap the contents of the CSV in lua code. This may create licensing issues? Maybe?
|
|
3. The item descriptions could be generated (partially) automatically, but they would still require someone to write large parts of them.
|
|
4. Localization; it may be possible to also do with the script (eg hooking into google translate, deepl or sth), at least for the item names.
|