A central question of training problem 1b addressed at the working group was how to compare the different penalties. Following up on those discussions, here are a few notes on how far I’ve gotten in the analysis regarding the best way to make these comparisons.
simulate_npv_curves Nice job of showing the performance of the variance ratio, clear example for L2. Correlation test should probably be applied to differences, pattern is much less clear. As for NPV comparison, looks too noisy, should calculate exactly from the SDP policy matrix in the way Michael suggested.
exact_npv. Much clearer plot than the stochastic case, not sure why asymmetric shows no penalty. Note that fixed costs keep increasing, getting policy-cost to exceed NPV0, implying that the profit margin has gone to zero. Probably not worth computing the cases that go beyond this. Surprised how much a difference discretization makes too.
updated policycost_L1. I’ve updated the simulation function to keep track of policy costs and total costs on the fly, so I no longer need to calculate these post-hoc, but this breaks the earlier version calls to ForwardSim. Likewise I’ve updated the policy cost dynamic programming solution to allow for arbitrary cost penalty, and to treat the quadratic effort penalty as a cost (for accounting purposes) which also breaks the prior version of these calls. Note that we’re also in Dan’s values for default parameters now. This file has been updated to reflect these changes, the examples of other penalties should be one-line chances to this code.
Example figures (more figures and details in the analyses linked above)
