-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for discounted properties in MDPs and DTMCs #621
base: master
Are you sure you want to change the base?
Conversation
This is absolutely great. I have two questions:
Best, |
Yes, convergence is detected using the Bellman residual, see The error messages are not implemented, I will take care of it! I'm happy about any pointers where such problems might occur. |
|
||
namespace storm { | ||
namespace logic { | ||
class DiscountedCumulativeRewardFormula : public PathFormula { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would the amount of code duplication shrink if you instead use another base class?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that makes things a lot more compact. Thanks for the hint
viOperator = std::make_shared<solver::helper::ValueIterationOperator<ValueType, TrivialRowGrouping>>(); | ||
viOperator->setMatrixBackwards(this->discountedA); | ||
} | ||
/*if (this->choiceFixedForRowGroup) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
....? :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, copy artifacts :D removed it now
DiscountedVIOperatorBackend(ValueType const& precision, ValueType const& discountFactor, ValueType const& maximalAbsoluteReward) | ||
: precision{precision}, | ||
discountFactor{discountFactor}, | ||
bound{(((storm::utility::one<ValueType>() - discountFactor) * precision) / (2 * discountFactor))} { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am slightly confused by the factor two here? Can we add a comment that clarifies this bound variable? :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I checked and the factor 2 is indeed incorrect, thanks for spotting that! I added a reference to the Russell/Norvig book
Thanks @sjunges for the review! I incorporated your comments now. |
This PR adds support for checking cumulative and total reward properties with a given discount factor on MDPs and DTMCs.
R=? [Cdiscount=_factor_]