Support for discounted properties in MDPs and DTMCs #621
Conversation
|
This is absolutely great. I have two questions:
Best, |
|
Yes, convergence is detected using the Bellman residual, see The error messages are not implemented, I will take care of it! I'm happy about any pointers where such problems might occur. |
|
|
||
| namespace storm { | ||
| namespace logic { | ||
| class DiscountedCumulativeRewardFormula : public PathFormula { |
There was a problem hiding this comment.
Would the amount of code duplication shrink if you instead use another base class?
There was a problem hiding this comment.
Yes, that makes things a lot more compact. Thanks for the hint
| viOperator = std::make_shared<solver::helper::ValueIterationOperator<ValueType, TrivialRowGrouping>>(); | ||
| viOperator->setMatrixBackwards(this->discountedA); | ||
| } | ||
| /*if (this->choiceFixedForRowGroup) { |
There was a problem hiding this comment.
Ah, copy artifacts :D removed it now
| DiscountedVIOperatorBackend(ValueType const& precision, ValueType const& discountFactor, ValueType const& maximalAbsoluteReward) | ||
| : precision{precision}, | ||
| discountFactor{discountFactor}, | ||
| bound{(((storm::utility::one<ValueType>() - discountFactor) * precision) / (2 * discountFactor))} { |
There was a problem hiding this comment.
I am slightly confused by the factor two here? Can we add a comment that clarifies this bound variable? :-)
There was a problem hiding this comment.
I checked and the factor 2 is indeed incorrect, thanks for spotting that! I added a reference to the Russell/Norvig book
|
Thanks @sjunges for the review! I incorporated your comments now. |
This PR adds support for checking cumulative and total reward properties with a given discount factor on MDPs and DTMCs.
R=? [Cdiscount=_factor_]