Perl in the end, because ez.

README.org

@@ -76,7 +76,7 @@ The output of this is an org-mode file in the following pattern:
 #+end_example
 
 
-* This script
+* On the script
 
 This is a literate script. The source code is embedded in these code blocks, and tangled into the Python script using org-babel. This allows me to write a description of what I want to do, and comment on it without resorting to ~// /* */~ ugly comments.
 
@@ -211,3 +211,90 @@ This structure allows for unambiguous selection of the relevant data from the cs
 2. These cannot be confused with the amounts themselves, because the column names include [a-zA-Z] characters.
 3. The first column makes it easy to find whether an item belongs to a particular column or not.
 4. Additionally, this way we can automate creating the output .csv template, since we can check what number of columns we need for each ingredient category.
+** Testing process 1:
+*** Get csv from Factorio
+This needs to be run manually rn, will figure out an automatic way later? Maybe?
+
+#+name: lua-to-org: Lua script generating a csv file containing all available recipes and their ingredients.
+#+begin_src lua :tangle no
+/c
+local recipes = {}
+for name, recipe in pairs(game.recipe_prototypes) do
+    local ingredients = {}
+    for _, ingredient in pairs(recipe.ingredients) do
+        ingredients[#ingredients+1] = ingredient.amount .. "," .. ingredient.name
+    end
+    recipes[#recipes+1] = "," .. name .. "," .. recipe.subgroup.name .. "," ..  table.concat(ingredients, ",")
+end
+game.write_file("recipes-all.csv", table.concat(recipes, "\n"), false)
+
+#+end_src
+
+*** Move csv to the right directory
+/Note:/ If you have this open in Emacs, move your cursor into the code block and hit ~C-c C-c~ to execute the code. It's very handy if you have commands that you use often.
+#+name: move the output from the above command to the mod-recipes directory for easy access.
+#+begin_src sh :results none
+mv ~/.factorio/script-output/recipes-all.csv ./mod-recipes
+#+end_src
+*** Sort for easier human readability
+**** Sort the csv file by recipe subgroups
+This allows us to find groups of related processes easier.
+#+begin_src sh :results none
+cat ./mod-recipes/recipes-all.csv | sort -k3 -t, | column --table -s, -o, > ./mod-recipes/recipes-all-sorted-subgroups.csv
+#+end_src
+
+**** Sort the csv file by product name
+This allows us to find similarly named items easier
+#+begin_src sh :results none
+cat ./mod-recipes/recipes-all.csv | sort -k2 -t, | column --table -s, -o, > ./mod-recipes/recipes-all-sorted-product.csv
+#+end_src
+
+This is the file we'll be using for further testing, because it groups up a lot of items by material, which is what we want.
+
+For this reason, let's make ourselves a working copy;
+
+#+begin_src sh :results none
+cp ./mod-recipes/recipes-all-sorted-product.csv ./mod-recipes/intermediate.csv
+#+end_src
+
+From here on, all changes will take place on the basis of the ~intermediate.csv~ file, and outputs will be directed to ~output.csv~. This will prevent time loss in case of a mistyped command.
+
+*** Define raw ingredients by their category
+We have the following categories we'll be assigning:
+- Telescoping, (inserters, belts, things that reach)
+- Metalworking,
+- Plastic,
+- Wood,
+- Stone,
+- Glass,
+- Electronics.
+
+At the same time, we have 2843 entries to deal with. That's a lot, so let's start with the raw ingredients, and assign those using the following rough rules:
+For this reason, it is important that we first cover the base materials.
+
+1. If the item exists in vanilla as a raw ingredient, it should be treated as such in the compatibility layer as well. If it is made craftable by a mod, its ingredients must also be tagged as raw materials.
+2. The ingredient in question may not have a recipe of its own. E.g. iron ore in vanilla Factorio, or stiratite in AngelBob. In this case, it is a raw ingredient.
+
+**** Metals
+The following code should:
+1. Find all the items which stem from iron ore *only*.
+2. Add 'metal' to the first field of their line in the CSV.
+
+To do this, we'll start with something like this:
+#+begin_src perl :tangle fra.pl
+open(my $in, "<", "./mod-recipes/intermediate.csv") or die "Can't open intermediates.csv";
+open(my $out, ">", "./mod-recipes/output.csv") or die "Can't open output.csv";
+
+while (<$in>) {
+    @line = split(",", $_);
+    if ($line[1] =~ /(iron|steel|copper|zinc|tungsten|titanium|tin|nickel|silver|platinum|manganese|lead|gold|aluminium)/) {
+        $line[0] = 'metal'
+    }
+    my $lineout = join (',', @line);
+    print $out "$lineout";
+}
+close $in or die "$in: $!";
+close $out or die "$out: $!";
+#+end_src
+
+#+RESULTS:

fra.pl

@@ -0,0 +1,13 @@
+open(my $in, "<", "./mod-recipes/intermediate.csv") or die "Can't open intermediates.csv";
+open(my $out, ">", "./mod-recipes/output.csv") or die "Can't open output.csv";
+
+while (<$in>) {
+    @line = split(",", $_);
+    if ($line[1] =~ /(iron|steel|copper|zinc|tungsten|titanium|tin|nickel|silver|platinum|manganese|lead|gold|aluminium)/) {
+        $line[0] = 'metal'
+    }
+    my $lineout = join (',', @line);
+    print $out "$lineout";
+}
+close $in or die "$in: $!";
+close $out or die "$out: $!";

mod-recipes/recipes-sorted-product.csv

@@ -0,0 +1,99 @@
+,accumulator                     ,angels-power-accumulator                      ,2  ,iron-plate          ,10 ,battery             ,2  ,electronic-circuit           ,   ,                    ,  ,
+,artillery-turret                ,angels-artillery-a                            ,60 ,concrete            ,60 ,steel-plate         ,40 ,iron-gear-wheel              ,20 ,advanced-circuit    ,  ,
+,artillery-wagon                 ,angels-artillery                              ,40 ,invar-alloy         ,64 ,engine-unit         ,10 ,iron-gear-wheel              ,20 ,advanced-circuit    ,16,pipe
+,assembling-machine-1            ,bob-assembly-machine                          ,9  ,iron-plate          ,5  ,iron-gear-wheel     ,3  ,basic-circuit-board          ,   ,                    ,  ,
+,assembling-machine-2            ,bob-assembly-machine                          ,1  ,assembling-machine-1,2  ,steel-plate         ,5  ,iron-gear-wheel              ,3  ,electronic-circuit  ,  ,
+,assembling-machine-3            ,bob-assembly-machine                          ,1  ,assembling-machine-2,9  ,steel-plate         ,5  ,steel-gear-wheel             ,3  ,advanced-circuit    ,  ,
+,automation-science-pack         ,science-pack                                  ,1  ,copper-plate        ,1  ,iron-gear-wheel     ,   ,                             ,   ,                    ,  ,
+,battery-equipment               ,angels-personal-equipment-power-c             ,10 ,steel-plate         ,5  ,battery             ,   ,                             ,   ,                    ,  ,
+,beacon                          ,module-beacon                                 ,10 ,copper-cable        ,10 ,steel-plate         ,20 ,electronic-circuit           ,20 ,advanced-circuit    ,1 ,crystal-splinter-harmonic
+,belt-immunity-equipment         ,angels-personal-equipment-power-d             ,10 ,steel-plate         ,5  ,advanced-circuit    ,   ,                             ,   ,                    ,  ,
+,big-electric-pole               ,angels-big-power-poles                        ,5  ,copper-plate        ,5  ,steel-plate         ,8  ,iron-stick                   ,   ,                    ,  ,
+,boiler                          ,angels-power-steam-boiler                     ,1  ,stone-furnace       ,4  ,pipe                ,   ,                             ,   ,                    ,  ,
+,burner-inserter                 ,bob-logistic-tier-0                           ,1  ,iron-plate          ,1  ,iron-gear-wheel     ,   ,                             ,   ,                    ,  ,
+,burner-mining-drill             ,extraction-machine                            ,1  ,stone-furnace       ,3  ,iron-plate          ,3  ,iron-gear-wheel              ,   ,                    ,  ,
+,cannon-shell                    ,angels-exploration-tank-a                     ,2  ,steel-plate         ,2  ,plastic-bar         ,1  ,explosives                   ,   ,                    ,  ,
+,car                             ,angels-vehicle-car                            ,20 ,iron-plate          ,5  ,steel-plate         ,8  ,engine-unit                  ,   ,                    ,  ,
+,cargo-wagon                     ,bob-cargo-wagon                               ,20 ,iron-plate          ,20 ,steel-plate         ,10 ,iron-gear-wheel              ,   ,                    ,  ,
+,centrifuge                      ,production-machine                            ,100,concrete            ,50 ,steel-plate         ,100,iron-gear-wheel              ,100,advanced-circuit    ,  ,
+,chemical-plant                  ,petrochem-buildings-chemical-plant            ,5  ,steel-plate         ,5  ,iron-gear-wheel     ,5  ,electronic-circuit           ,5  ,pipe                ,  ,
+,cluster-grenade                 ,angels-capsules-a                             ,5  ,steel-plate         ,5  ,explosives          ,7  ,grenade                      ,   ,                    ,  ,
+,combat-shotgun                  ,angels-physical-ba                            ,15 ,steel-plate         ,5  ,wood                ,1  ,shotgun                      ,   ,                    ,  ,
+,copper-cable                    ,angels-copper-casting                         ,1  ,copper-plate        ,   ,                    ,   ,                             ,   ,                    ,  ,
+,discharge-defense-equipment     ,angels-personal-equipment-combat-c            ,20 ,steel-plate         ,5  ,processing-unit     ,10 ,laser-turret                 ,   ,                    ,  ,
+,electric-energy-interface       ,other                                         ,2  ,iron-plate          ,5  ,electronic-circuit  ,   ,                             ,   ,                    ,  ,
+,electric-furnace                ,smelting-machine                              ,10 ,stone-brick         ,10 ,steel-plate         ,5  ,advanced-circuit             ,   ,                    ,  ,
+,electric-mining-drill           ,extraction-machine                            ,10 ,iron-plate          ,5  ,iron-gear-wheel     ,3  ,basic-circuit-board          ,   ,                    ,  ,
+,empty-barrel                    ,angels-fluid-control                          ,1  ,steel-plate         ,   ,                    ,   ,                             ,   ,                    ,  ,
+,energy-shield-equipment         ,angels-personal-equipment-combat-a            ,10 ,steel-plate         ,5  ,advanced-circuit    ,   ,                             ,   ,                    ,  ,
+,engine-unit                     ,angels-basic-intermediate                     ,1  ,steel-plate         ,1  ,iron-gear-wheel     ,2  ,pipe                         ,   ,                    ,  ,
+,exoskeleton-equipment           ,angels-personal-equipment-combat-c            ,20 ,steel-plate         ,30 ,electric-engine-unit,10 ,advanced-circuit             ,   ,                    ,  ,
+,explosive-cannon-shell          ,angels-exploration-tank-a                     ,2  ,steel-plate         ,2  ,plastic-bar         ,2  ,explosives                   ,   ,                    ,  ,
+,firearm-magazine                ,angels-physical-aa                            ,4  ,iron-plate          ,   ,                    ,   ,                             ,   ,                    ,  ,
+,flamethrower-ammo               ,angels-fire                                   ,5  ,steel-plate         ,50 ,liquid-fuel-oil     ,50 ,liquid-naphtha               ,   ,                    ,  ,
+,flamethrower                    ,angels-fire                                   ,5  ,steel-plate         ,5  ,iron-gear-wheel     ,1  ,submachine-gun               ,   ,                    ,  ,
+,flamethrower-turret             ,angels-fire                                   ,20 ,steel-plate         ,5  ,engine-unit         ,1  ,flamethrower                 ,10 ,pipe                ,  ,
+,fluid-wagon                     ,bob-fluid-wagon                               ,16 ,steel-plate         ,1  ,storage-tank        ,10 ,iron-gear-wheel              ,8  ,pipe                ,  ,
+,flying-robot-frame              ,angels-robot-a                                ,1  ,steel-plate         ,1  ,electric-engine-unit,2  ,battery                      ,3  ,electronic-circuit  ,  ,
+,gate                            ,angels-exploration-walls                      ,2  ,steel-plate         ,2  ,electronic-circuit  ,1  ,stone-wall                   ,   ,                    ,  ,
+,grenade                         ,angels-capsules-a                             ,5  ,iron-plate          ,10 ,coal                ,   ,                             ,   ,                    ,  ,
+,gun-turret                      ,angels-physical-aa                            ,5  ,copper-plate        ,10 ,iron-plate          ,1  ,submachine-gun               ,   ,                    ,  ,
+,heat-pipe                       ,angels-power-nuclear-heat-pipe                ,10 ,copper-plate        ,5  ,steel-plate         ,   ,                             ,   ,                    ,  ,
+,heavy-armor                     ,angels-personal-equipment-armor               ,100,copper-plate        ,50 ,steel-plate         ,   ,                             ,   ,                    ,  ,
+,inserter                        ,bob-logistic-tier-1                           ,1  ,iron-plate          ,1  ,iron-gear-wheel     ,1  ,basic-circuit-board          ,   ,                    ,  ,
+,iron-chest                      ,angels-chests-small-a                         ,8  ,iron-plate          ,   ,                    ,   ,                             ,   ,                    ,  ,
+,iron-gear-wheel                 ,bob-gears                                     ,2  ,iron-plate          ,   ,                    ,   ,                             ,   ,                    ,  ,
+,iron-stick                      ,angels-basic-intermediate                     ,1  ,iron-plate          ,   ,                    ,   ,                             ,   ,                    ,  ,
+,lab                             ,production-machine                            ,10 ,iron-gear-wheel     ,10 ,basic-circuit-board ,4  ,basic-transport-belt         ,   ,                    ,  ,
+,land-mine                       ,angels-capsules-a                             ,1  ,steel-plate         ,2  ,explosives          ,   ,                             ,   ,                    ,  ,
+,laser-turret                    ,angels-electric-a                             ,20 ,steel-plate         ,12 ,battery             ,20 ,electronic-circuit           ,   ,                    ,  ,
+,light-armor                     ,angels-personal-equipment-armor               ,40 ,iron-plate          ,   ,                    ,   ,                             ,   ,                    ,  ,
+,loader                          ,belt                                          ,5  ,iron-plate          ,5  ,iron-gear-wheel     ,5  ,electronic-circuit           ,5  ,transport-belt      ,5 ,inserter
+,locomotive                      ,bob-locomotive                                ,30 ,steel-plate         ,20 ,engine-unit         ,10 ,electronic-circuit           ,   ,                    ,  ,
+,medium-electric-pole            ,angels-medium-power-poles                     ,2  ,copper-plate        ,2  ,steel-plate         ,4  ,iron-stick                   ,   ,                    ,  ,
+,modular-armor                   ,angels-personal-equipment-armor               ,50 ,steel-plate         ,30 ,advanced-circuit    ,   ,                             ,   ,                    ,  ,
+,night-vision-equipment          ,angels-personal-equipment-power-d             ,10 ,steel-plate         ,5  ,advanced-circuit    ,   ,                             ,   ,                    ,  ,
+,nuclear-reactor                 ,angels-power-nuclear-reactor-b                ,500,concrete            ,250,steel-plate         ,500,advanced-circuit             ,25 ,heat-pipe-2         ,  ,
+,offshore-pump                   ,washing-building                              ,1  ,iron-gear-wheel     ,2  ,basic-circuit-board ,1  ,pipe                         ,   ,                    ,  ,
+,oil-refinery                    ,petrochem-buildings-oil-refinery              ,6  ,steel-plate         ,12 ,steel-gear-wheel    ,4  ,electronic-circuit           ,36 ,steel-pipe          ,  ,
+,personal-laser-defense-equipment,angels-personal-equipment-combat-b            ,5  ,steel-plate         ,3  ,battery             ,5  ,electronic-circuit           ,   ,                    ,  ,
+,personal-roboport-equipment     ,angels-personal-equipment-robot-d             ,5  ,steel-plate         ,2  ,roboport-antenna-1  ,1  ,roboport-door-1              ,2  ,roboport-chargepad-1,  ,
+,piercing-rounds-magazine        ,angels-physical-aa                            ,5  ,copper-plate        ,1  ,steel-plate         ,1  ,firearm-magazine             ,   ,                    ,  ,
+,piercing-shotgun-shell          ,angels-physical-ba                            ,5  ,copper-plate        ,2  ,steel-plate         ,2  ,shotgun-shell                ,   ,                    ,  ,
+,pipe                            ,pipe                                          ,1  ,iron-plate          ,   ,                    ,   ,                             ,   ,                    ,  ,
+,pipe-to-ground                  ,pipe-to-ground                                ,5  ,iron-plate          ,10 ,pipe                ,   ,                             ,   ,                    ,  ,
+,pistol                          ,angels-physical-aa                            ,5  ,copper-plate        ,5  ,iron-plate          ,   ,                             ,   ,                    ,  ,
+,poison-capsule                  ,angels-capsules-a                             ,3  ,steel-plate         ,10 ,coal                ,3  ,electronic-circuit           ,   ,                    ,  ,
+,power-armor                     ,angels-personal-equipment-armor               ,40 ,steel-plate         ,20 ,electric-engine-unit,40 ,processing-unit              ,   ,                    ,  ,
+,power-switch                    ,circuit-network                               ,5  ,copper-cable        ,5  ,iron-plate          ,2  ,electronic-circuit           ,   ,                    ,  ,
+,programmable-speaker            ,circuit-network                               ,5  ,copper-cable        ,3  ,iron-plate          ,4  ,iron-stick                   ,4  ,electronic-circuit  ,  ,
+,pump                            ,angels-fluid-control                          ,1  ,steel-plate         ,1  ,engine-unit         ,1  ,copper-pipe                  ,   ,                    ,  ,
+,pumpjack                        ,extraction-machine                            ,5  ,steel-plate         ,10 ,iron-gear-wheel     ,5  ,electronic-circuit           ,10 ,pipe                ,  ,
+,radar                           ,angels-artillery-a                            ,10 ,iron-plate          ,5  ,iron-gear-wheel     ,5  ,basic-circuit-board          ,   ,                    ,  ,
+,rail-chain-signal               ,train-transport                               ,5  ,iron-plate          ,1  ,electronic-circuit  ,   ,                             ,   ,                    ,  ,
+,rail-signal                     ,train-transport                               ,5  ,iron-plate          ,1  ,electronic-circuit  ,   ,                             ,   ,                    ,  ,
+,rail                            ,train-transport                               ,1  ,stone               ,1  ,steel-plate         ,1  ,iron-stick                   ,   ,                    ,  ,
+,refined-concrete                ,angels-stone-casting                          ,20 ,concrete            ,1  ,steel-plate         ,8  ,iron-stick                   ,100,water               ,  ,
+,repair-pack                     ,angels-chests-small-a                         ,2  ,iron-gear-wheel     ,2  ,basic-circuit-board ,   ,                             ,   ,                    ,  ,
+,roboport                        ,angels-roboport-a                             ,15 ,steel-plate         ,5  ,roboport-antenna-1  ,1  ,roboport-door-1              ,4  ,roboport-chargepad-1,  ,
+,rocket-launcher                 ,angels-explosion-a                            ,5  ,steel-plate         ,5  ,electronic-circuit  ,2  ,shotgun                      ,   ,                    ,  ,
+,shotgun                         ,angels-physical-ba                            ,5  ,iron-plate          ,5  ,wood                ,5  ,iron-gear-wheel              ,2  ,pistol              ,  ,
+,shotgun-shell                   ,angels-physical-ba                            ,2  ,copper-plate        ,2  ,iron-plate          ,   ,                             ,   ,                    ,  ,
+,slowdown-capsule                ,angels-capsules-a                             ,2  ,steel-plate         ,5  ,coal                ,2  ,electronic-circuit           ,   ,                    ,  ,
+,small-lamp                      ,circuit-network                               ,3  ,copper-cable        ,1  ,iron-plate          ,1  ,basic-circuit-board          ,   ,                    ,  ,
+,solar-panel                     ,angels-power-solar-panel                      ,4  ,copper-plate        ,4  ,steel-plate         ,14 ,electronic-circuit           ,   ,                    ,  ,
+,solar-panel-equipment           ,angels-personal-equipment-power-a             ,4  ,copper-cable        ,2  ,steel-plate         ,5  ,electronic-circuit           ,   ,                    ,  ,
+,splitter                        ,bob-logistic-tier-1                           ,8  ,tin-plate           ,14 ,iron-gear-wheel     ,5  ,basic-circuit-board          ,1  ,basic-splitter      ,  ,
+,steam-engine                    ,angels-power-steam-generator                  ,10 ,iron-plate          ,8  ,iron-gear-wheel     ,5  ,pipe                         ,   ,                    ,  ,
+,steel-chest                     ,angels-chests-small-b                         ,8  ,steel-plate         ,   ,                    ,   ,                             ,   ,                    ,  ,
+,steel-furnace                   ,smelting-machine                              ,10 ,stone-brick         ,6  ,steel-plate         ,   ,                             ,   ,                    ,  ,
+,steel-plate                     ,angels-steel-casting                          ,2  ,iron-plate          ,10 ,gas-oxygen          ,   ,                             ,   ,                    ,  ,
+,storage-tank                    ,angels-fluid-tanks                            ,20 ,iron-plate          ,5  ,steel-plate         ,1  ,bob-small-inline-storage-tank,   ,                    ,  ,
+,submachine-gun                  ,angels-physical-aa                            ,5  ,iron-plate          ,10 ,iron-gear-wheel     ,1  ,pistol                       ,   ,                    ,  ,
+,substation                      ,angels-sub-power-poles                        ,5  ,copper-plate        ,10 ,steel-plate         ,5  ,advanced-circuit             ,   ,                    ,  ,
+,sulfuric-acid                   ,angels-fluid-converter-petrochem-sulfer-fluids,1  ,iron-plate          ,5  ,sulfur              ,100,water-purified               ,   ,                    ,  ,
+,tank                            ,angels-exploration-tank-a                     ,50 ,steel-plate         ,32 ,engine-unit         ,15 ,iron-gear-wheel              ,10 ,advanced-circuit    ,  ,
+,train-stop                      ,train-transport                               ,6  ,iron-plate          ,3  ,steel-plate         ,6  ,iron-stick                   ,5  ,electronic-circuit  ,  ,
+,transport-belt                  ,bob-logistic-tier-1                           ,2  ,tin-plate           ,2  ,iron-gear-wheel     ,1  ,basic-transport-belt         ,   ,                    ,  ,
+,underground-belt                ,bob-logistic-tier-1                           ,14 ,tin-plate           ,20 ,iron-gear-wheel     ,2  ,basic-underground-belt       ,   ,                    ,  ,
+,uranium-fuel-cell               ,angels-power-nuclear-fuel-cell                ,15 ,iron-plate          ,1  ,uranium-235         ,29 ,uranium-238                  ,   ,                    ,  ,

mod-recipes/recipes.org

@@ -24,6 +24,8 @@ game.write_file("recipes.org", table.concat(recipes, "\n"), false)
 The purpose of the file is to give an overview of the data structures used in the mods.
 
 
+
+
 * angels-alloys
 ** angels-solder-mixture
 *** products