Generalize material loss, initial cost, learn rate and any other blade-specific factors in cost graph

[eberlea]

Generalize material loss, initial cost, learn rate and any other blade-specific factors in cost graph

[rjhanes]

I'll work on wrapping up some of the existing issue branches and likely wait until Release 1.0.0 is complete before creating a branch to work on this issue. In the meantime, I'm storing some thoughts here. Feedback is welcome but not required.

Cost parameters in config

A lot can be accomplished by taking advantage of the config file's default dictionary structure. We can separate out the items currently in costgraph_parameters with case-study-specific names (fine grinding, etc) and move those items to a separate variable group. Then, we can read in that entire variable group as a dictionary with a default name (cost_params or similar - or maybe case_study_params) and pass that dictionary around instead of the individual cost parameters with names that change depending on the technology.

Material loss, which must also be accessed within the discrete event simulation, can be its own subdictionary within the case_study_params dictionary, with key:value pairs for every processing step that involves loss. The keys will need to be internally consistent with the processing steps defined in the input datasets. Then we can apply material loss calculations by default to every processing step, and if there's no material loss value for that step then it defaults to zero (no loss). In the discrete event simulation, perhaps trigger closest landfill and associated transportation incrementing if the loss is non-zero.

CostMethods will still need to be written specially for each case study, but main.py and CostGraph.py will be completely reusable.

Before:

costgraph_parameters:
    sc_begin: manufacturing
    sc_end: 
    - landfilling
    - cement co-processing
    - next use
    cg_verbose: 1
    save_cg_csv: True
    finegrind_cumul_initial: 1.0
    finegrind_initial_cost: 165.38
    finegrind_revenue: 242.56
    finegrind_learnrate: -0.05
    finegrind_material_loss: 0.3
    coarsegrind_cumul_initial: 1.0
    coarsegrind_initial_cost: 121.28
    coarsegrind_learnrate: -0.05
    cg_update_timesteps: 12

After:

costgraph_parameters:
    component: blade
    sc_begin: manufacturing
    sc_end: 
    - landfilling
    - cement co-processing
    - next use
    cg_verbose: 1
    save_cg_csv: True
    cg_update_timesteps: 12

# These parameters are specific for the cost methods,
# which must be written in Python for each 
# case study.
# This set of parameters is passed into CostGraph as
# a dictionary, allowing the user to avoid changing
# parameters for the CostGraph instantiation in main.py.
case_study_parameters:
    finegrind_cumul_initial: 1.0
    finegrind_initial_cost: 165.38
    finegrind_revenue: 242.56
    finegrind_learnrate: -0.05
    coarsegrind_cumul_initial: 1.0
    coarsegrind_initial_cost: 121.28
    coarsegrind_learnrate: -0.05
    material_loss:
        finegrind : 0.3

CostGraph instantiation in main.py

Reading in an entire dictionary of cost parameters instead of individual parameters takes care of most of the generalization during instantiation. The component mass parameter (necessary for some internal CostGraph calculations) can be re-named and calculated by component from the mass input file.

Before:

    netw = CostGraph(
        step_costs_file=step_costs_filename,
        fac_edges_file=fac_edges_filename,
        transpo_edges_file=transpo_edges_filename,
        locations_file=locations_computed_filename,
        routes_file=routes,
        sc_begin=cg_params.get('sc_begin'),
        sc_end=cg_params.get('sc_end'),
        year=scenario_params.get('start_year'),
        max_dist=scenario_params.get('max_dist'),
        verbose=cg_params.get('cg_verbose'),
        save_copy=cg_params.get('save_cg_csv'),
        save_name=costgraph_csv_filename,
        pathway_cost_history_filename = pathway_cost_history_filename,
        blade_mass=avgblade.loc[avgblade.year==scenario_params.get('start_year'),
                                'total'].values[0],
        finegrind_cumul_initial=cg_params.get('finegrind_cumul_initial'),
        coarsegrind_cumul_initial=cg_params.get('coarsegrind_cumul_initial'),
        finegrind_initial_cost=cg_params.get('finegrind_initial_cost'),
        finegrind_revenue=cg_params.get('finegrind_revenue'),
        coarsegrind_initial_cost=cg_params.get('coarsegrind_initial_cost'),
        finegrind_learnrate=cg_params.get('finegrind_learnrate'),
        coarsegrind_learnrate=cg_params.get('coarsegrind_learnrate'),
        finegrind_material_loss=cg_params.get('finegrind_material_loss')
    )

After:

    netw = CostGraph(
        step_costs_file=step_costs_filename,
        fac_edges_file=fac_edges_filename,
        transpo_edges_file=transpo_edges_filename,
        locations_file=locations_computed_filename,
        routes_file=routes,
        sc_begin=cg_params.get('sc_begin'),
        sc_end=cg_params.get('sc_end'),
        year=scenario_params.get('start_year'),
        max_dist=scenario_params.get('max_dist'),
        verbose=cg_params.get('cg_verbose'),
        save_copy=cg_params.get('save_cg_csv'),
        save_name=costgraph_csv_filename,
        pathway_cost_history_filename = pathway_cost_history_filename,
        component_mass= ...,
        costmethod_params = cost_params
    )

Assigning cost parameters to self. variables in CostGraph.__init__

The individual parameters are replaced with the dictionary of parameters. Error-handling for the initial cumulative processing values can be moved to the relevant cost method instead of being done here (basically, those values can't be zero).

Before:

        if kwargs['finegrind_cumul_initial']==0:
            self.finegrind_cumul_initial=1.0
        else:
            self.finegrind_cumul_initial=kwargs['finegrind_cumul_initial']
        if kwargs['coarsegrind_cumul_initial']==0:
            self.coarsegrind_cumul_initial=1.0
        else:
            self.coarsegrind_cumul_initial=kwargs['coarsegrind_cumul_initial']

        self.finegrind_initial_cost = kwargs['finegrind_initial_cost']
        self.coarsegrind_initial_cost = kwargs['coarsegrind_initial_cost']

        self.finegrind_revenue = kwargs['finegrind_revenue']

        self.finegrind_learnrate = kwargs['finegrind_learnrate']
        self.coarsegrind_learnrate = kwargs['finegrind_learnrate']

        self.finegrind_material_loss = kwargs['finegrind_material_loss']

After:

    self.cost_params = cost_params

Passing cost parameters and info from DES into the cost methods in CostGraph.update_costs

To make the cost calculations as streamlined as they are, all of the CostMethod functions need to have the same arguments. This is currently accomplished using **kwargs, but could instead be done using the same dictionary of parameters. Once supplied to the cost functions, the relevant values are extracted and used in the calculation.

update_costs itself would need to have a specific year argument rather than just **kwargs.

Before:

        for edge in self.supply_chain.edges():
            self.supply_chain.edges[edge]['cost'] = sum(
                [f(vkmt=self.supply_chain.edges[edge]['dist'],
                   year=kwargs['year'],
                   blade_mass=kwargs['blade_mass'],
                   finegrind_cumul=kwargs['finegrind_cumul'],
                   coarsegrind_cumul=kwargs['coarsegrind_cumul'],
                   finegrind_cumul_initial=self.finegrind_cumul_initial,
                   coarsegrind_cumul_initial=self.coarsegrind_cumul_initial,
                   finegrind_initial_cost=self.finegrind_initial_cost,
                   finegrind_revenue=self.finegrind_revenue,
                   coarsegrind_initial_cost=self.coarsegrind_initial_cost,
                   finegrind_learnrate=self.finegrind_learnrate,
                   coarsegrind_learnrate=self.coarsegrind_learnrate,
                   finegrind_material_loss=self.finegrind_material_loss)
                 for f in self.supply_chain.edges[edge]['cost_method']]
            )

After:

        for edge in self.supply_chain.edges():
            self.supply_chain.edges[edge]['cost'] = sum(
                [f(vkmt=self.supply_chain.edges[edge]['dist'],
                   year=model_year,
                   cost_params = self.cost_params)
                 for f in self.supply_chain.edges[edge]['cost_method']]
            )

[akey7]

Thanks @rjhanes for putting this together. This looks like a good path forward.

[eberlea]

@rjhanes I like this approach and I think the changes to CostGraph.init and CostGraph.update_costs look good. We may want to think a little bit more about the parameter naming convention in the config. Maybe something like:

pathway names
pathway1: finegrind
pathway2: coarsegrind

pathway parameters
pathway1:
cumul_intial: 1.0
initial_cost: 165.38
revenue: 242.56
learnrate: -0.05
material_loss: 0.3
pathway2:
cumul_intial: 1.0
initial_cost: 121.28
revenue: 0
learnrate: -0.05
material_loss: 0

[rjhanes]

Issue #81 now covers this aspect of code generalization, so I'm closing this issue as a duplicate after moving comments over to 81