Calibration of USA fesob fails

[remindmodel]

Calibration of USA fesob fails

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

The calibration of fesob in the USA region does not reach the target, even after many iterations.

In specific periods, or across the time horizon? How big is the difference?

Removing the smoothing would help for fesos USA

Have you tried this, or are you guessing?

[…] but it would cause problems elsewhere.

I think you could justify special treatment for fesos price smoothing.

In addition, I have tried to fix fesob and fesoi to their 2025 target quantity: the idea is to provoke an infeasibility that could point to the original problem (for early periods it is usually related to capacities). There was no infeasibility.

What I did a while back was fixing quantities to the target value and slacken the fixing over calibration iterations. The idea was to force prices into the right range early on and help the calibration figuring it out this way. My results where unconvincing, but you could try if you are desperate. ;)

vm_cesIO.lo(t,regi,in) 
  = pm_cesdata(t,regi,in,"quantity") 
  * max(0, (1 - max(0, %c_CES_calibration_iteration% - 2) / 8));

vm_cesIO.up(t,regi,in) 
  = pm_cesdata(t,regi,in,"quantity") 
  * (1 + max(0, %c_CES_calibration_iteration% - 2) / 8);

[piklev]

In specific periods, or across the time horizon? How big is the difference?

The problem occurs in different settings. I have seen two so far: around 2050 (SSP2) on the one hand and around 2025 (SDP). The most recent runs are from @Loisel (snapshots).

/p/tmp/aloisdir/remind-trunk/output/SSP2-calibrate_2020-04-09_10.26.22
/p/tmp/aloisdir/remind-trunk/output/SDP-calibrate_2020-04-09_10.26.20

The price profiles are slightly different. In SSP2, the price falls drastically both around 2025 and 2050, while in SDP, the huge drop is mostly around 2025, with prices that near 0.

Have you tried this, or are you guessing?

I have tried this, and this was my first quick fix to the issue... before the problem popped up again.

I think you could justify special treatment for fesos price smoothing.

I think this is actually a supply-side issue, more than a calibration issue. Applying a special treatment for fesos, or for prices that fall or rise abruptly might do the trick, but it would not solve the problem, which could happen elsewhere. Though that could be an option, I am afraid that this option might in the end hide problems in the model.

What I did a while back was fixing quantities to the target value and slacken the fixing over calibration iterations. The idea was to force prices into the right range early on and help the calibration figuring it out this way. My results where unconvincing, but you could try if you are desperate. ;)

vm_cesIO.lo(t,regi,in) 
  = pm_cesdata(t,regi,in,"quantity") 
  * max(0, (1 - max(0, %c_CES_calibration_iteration% - 2) / 8));

vm_cesIO.up(t,regi,in) 
  = pm_cesdata(t,regi,in,"quantity") 
  * (1 + max(0, %c_CES_calibration_iteration% - 2) / 8);

I have tried something similar (changing prices to a reasonable value in the beginning). But the model came back to normal after the prices were set free. I do not think this is an initial point issue.

[MariannaR]

Following our discussion, I made the following tests for SSP2:

• I commented out the constraints on vm_deltaCap.fx("biotr") in core/bounds.gms
• I smoothed the phase out of p21_tau_fe_sub_bit_st in USA so that it reached 0 more gradually

both tests did not work out, the quantity value still spikes in 2055.

[Loisel]

Removing biotr bounds on vm_deltaCap in the core/bounds.gms file (switch cm_biotr_bounds) did not solve the issues neither for SSP2 nor for SDP. Checked both for testOneRegi and nash runs.

Paths:

/p/tmp/aloisdir/remind-trunk/output/testNoBounds_SSP2
/p/tmp/aloisdir/remind-trunk/output/testNoBounds_SDP

[piklev]

If I remember correctly, biotr = 0 in the USA anyways. So the constraints on capacity are not used anyways. I am expecting more from the test fixing the demand to the target for early periods in the SDP scenario. If both issues are related, solving the SDP issue (which is more extreme) would put us on a good track

[Loisel]

Calibration with fixings just seem to work (!):

SSP2 (fixed 2055)
/p/tmp/aloisdir/remind-trunk/output/SSP2-calibrate_2020-04-16_14.52.59

SDP
/p/tmp/aloisdir/remind-trunk/output/SDP-calibrate_2020-04-16_14.52.58

The switch is cm_feso_fixing. Prices seem to just adapt (going up for SDP, down for SSP2). Does this mean that a different starting point for prices might help @piklev ?

[piklev]

This is actually bad news. I was hoping the model to hit a constraint that could explain why the prices fall to 0 in SDP. I still think understanding the reason behind the dynamic leading to prices going 0 is the right track to follow. One would probably have to look into the details of the loop prices-(quantities)-efficiencies across all iterations to get some clues, and have an idea for new tests.
@Loisel I do not think changing the initial prices helps, but you can have a try

[Loisel]

A change in initial prices does not solve the issue:

SDP (with initial prices from the fixed version above)
/p/tmp/aloisdir/remind-trunk/output/SDP-calibrate-initialPrice_2020-04-17_13.44.46
The solid red line (original prices) is hidden behind the black line (target price):

SSP2
/p/tmp/aloisdir/remind-trunk/output/SSP2-calibrate-initialPrice_2020-04-17_14.47.49

[Loisel]

What seems to do as a workaround is to fix the trajectories for fesob and fesoi for US, switch cm_feso_full_fixing.

SDP

SSP2

However, for SSP2 the backwards calculated quantities are still somewhat off (which would mean the efficiencies and prices are not able to fully adapt @piklev ?).

[piklev]

Actually if you fix the quantity to the target quantity, there should be no difference at all. So both in SSP2 and SDP, there are differences between the actual pathway taken and the target pathway. From the GDX, I read that the fixing only works for the upper bound. The fixing of the lower bound seems overwritten. It probably comes from ./modules/01_macro/singleSectorGr/bounds.gms. If this is fixed, then the trajectories will be exactly the same.

[LaviniaBaumstark]

Antoine, do you thing that the fix a good intermediate bug fix?

[LaviniaBaumstark]

I wonder whether we want to have it in our next release. If so we would need the new input data based on this calibration. If not I would prefer a separate branch.

[piklev]

@LaviniaBaumstark I think this fix is not something we would like to have in the model. It would mean that the solids energy demand both in buildings and industry would be fixed to the Baseline levels. The model could only decide to substitute biomass for coal, but not electricity/gas for solids. I do not think it would be helpful.
It could have been helpful to spot an infeasibility, but even that did not work out

[Loisel]

@piklev @LaviniaBaumstark For the record: The PR is only about the calibration data. I do not suggest to merge the patch. I fully agree with @piklev in that regard. This would be just to save some time if we would require new FE demand trajectories after checking the results in normal runs.

[Loisel]

Fixing fesob and fesoi for USA in the calibration does not solve the issue, as expected by @piklev
The dashed outlier line is the last iteration where quantities were not fixed.

SDP:

SSP2:

Quantities will just jump back to the unwanted behavior as soon as left alone. This again points towards the prices being the problem.

[piklev]

I have made some tests. The issue with prices seems to be related with coal essentially. I did not yet identify why however.

Switches:

• nosm/sm : without or with price smoothing
• 1st/exp: cost of residue biomass. 1st is the default, exp gives a very high value that pushes the model to avoid it (the problem with residue biomass is that it has a flat cost curve, which can cause problems for the calibration)
• nml/acl: deals with the slope of the curve for purpose-grown biomass. nml is the default and acl has a faster increasing curbe
• notech/biotrmod/coaltr: technologies which are not invested into after 2020. When notech is chosen, there is no constraint (in addition to the default ones)
• nolimit/limit: q_limitBiotrmod off (nolimit) or on (limit). It has to be off for the runs without coaltr to be feasible (pre-triangular)

Scenarios with constraints on coaltr do not show the huge price drop for fesob, which is most likely the root cause for this issue

CES calibration report_feso_nosm_1st_nml_biotrmod_nolimit.pdf
CES calibration report_feso_nosm_1st_nml_coaltr_nolimit.pdf
CES calibration report_feso_nosm_1st_nml_notech_limit.pdf
CES calibration report_feso_nosm_1st_nml_notech_nolimit.pdf
CES calibration report_feso_nosm_exp_nml_biotrmod_nolimit.pdf
CES calibration report_feso_nosm_exp_nml_coaltr_nolimit.pdf
CES calibration report_feso_nosm_exp_nml_notech_limit.pdf
CES calibration report_feso_nosm_exp_nml_notech_nolimit.pdf

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

@robinhasse, is this still relevant?

[robinhasse]

@robinhasse, is this still relevant?

No, I will close it.

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

Nice, this way @Loisel leaves no issues behind :p