docs: draft HardForkCombinatorRollbackMonotonicityProposal.md

[nfrisby]

This an attempt at the first step of Issue #389.

[nfrisby]

This is a Draft PR. But I'd appreciate your thoughts already.

Rendered: https://github.com/input-output-hk/ouroboros-consensus/blob/nfrisby/hfc-proposal/docs/website/contents/for-developers/HardForkCombinatorRollbackMonotonicityProposal.md

[dnadales]

It's not clear to me what the worst case is here, and why this is true in that case.

[nfrisby]

Yes, this parenthetical is admittedly handwaivy.

C can be better than D in two ways: it can be the same length and have a better tiebreaker or it can be longer. When it's longer requires a deeper rollback from us. That extra depth is the "worst case" aspect here.

[dnadales]

Perhaps the "Otherwise" bullet can be merged with the previous one, or indented one level.

[dnadales]

Why k+1 in total? Are we counting the header of the block that led to L?

[nfrisby]

Suppose Y is the first block after L on C. The forecasting allows for k blocks after Y on C. So that's k+1 total.

[dnadales]

What constitutes a complete answer in this context? (and how do we convince ourselves this is one of them?)

[nfrisby]

What constitutes a complete answer in this context?

The next sentence was intended to clarify that.

how do we convince ourselves this is one of them?

Conversation, property testing, etc :D

[dnadales]

I think a concrete example would be great here.

[dnadales]

Could we define, explain and exemplify "weak monotonicity" in this context?

[nfrisby]

It's a slightly more complicated general version of "non-descending". I think it'd actually be a non-trivial aside. But I have no doubt a reference exists.

[nfrisby]

Just a point of reference: the node CLI query for the leader schedule is perhaps also in the same gray area as time translations. Should that query refuse to answer until a rollback by <= k blocks could not change it?

Edit: it was discussed, and yes it should refuse. See Issue #390.

Nick would like me to fully understand the document before approving

[nfrisby]

I pushed up an Introduction; hopefully that makes the document easier to approach.

[nfrisby]

I also pushed up some adjustments for the smaller comments you made.

[dnadales]

About DST: I: I would have expected that it is sensitive to ticks since ticking might cause votes to be counted and a hardfork could be decided as a consequence of this. Would it be possible to explain why this is not the case?

[nfrisby]

Yeah. Extremely high-level summary: it's because of PIL=Possible. The ticking itself does make the vote-counting actually happen, but if there hasn't been enough blocks, the Cardano HFC logic will "veto" the decision to end the era and simply not transition to the next era.

[dnadales]

The ticking itself does make the vote-counting actually happen

Oh, I thought this was the case. Do you know how we currently determine if we should transition to the next era based on the ledger info? I guess that we do not rely on this rule on epoch change. Do we roll out our own logic (that totally ignores the ledger?).

[nfrisby]

The ticking itself does make the vote-counting actually happen

Oh, I thought this was the case.

I sounds like maybe you read "doesn't" because I wrote "does", which is a bit unusual.

Do we roll out our own logic (that totally ignores the ledger?)

Here's the entrypoint to the Consensus-side logic https://github.com/input-output-hk/ouroboros-consensus/blob/51da3876c01edc2eec250fdc998f6cb33cdc4367/ouroboros-consensus-cardano/src/shelley/Ouroboros/Consensus/Shelley/ShelleyHFC.hs#L130-L164

There's probably plenty of redundancy that could be removed from this Consensus-side logic (in the "Inspect" module), but it's only exactly this one line that "ignores the ledger" https://github.com/input-output-hk/ouroboros-consensus/blob/51da3876c01edc2eec250fdc998f6cb33cdc4367/ouroboros-consensus-cardano/src/shelley/Ouroboros/Consensus/Shelley/ShelleyHFC.hs#L160 shelleyAfterVoting is from the ledger state; it's incremented (here) whenever we apply a block.

[dnadales]

I sounds like maybe you read "doesn't" because I wrote "does", which is a bit unusual.

I can't read 🤦 Sorry about that.

[dnadales]

Thanks for the links to the relevant snippets 🙏

but it's only exactly this one line that "ignores the ledger"

"Ignores the ledger when there aren't k blocks after a transition to an era was voted on the ledger, right?

[dnadales]

Could we elaborate an example of this?

[nfrisby]

We could. It should be pretty simple, but it will require a lot of words, just because of the context. If it doesn't require a lot of works, then it would remain pretty abstract, which kind of defeats the purpose of the example.

I left it out for now because it seems like a lot of words. Maybe we could consolidate all of the examples into an "example" section that shares the necessary context amongst a few pointed examples.

[dnadales]

Yeah, I think that would help the readers greatly. But I acknowledge this is not intended to be a document used for mass consumption, right? So perhaps this is also out of scope.

[dnadales]

I wonder how that could even be possible. 🤔 Do you have an example?

[nfrisby]

I think @amesgen would more easily be able to explain it than I can.

[amesgen]

Right now, the criterion for singleEraTransition to report the era transition for Shelley-based eras is:

Were there at least $k$ blocks after the voting deadline ($6k/f$ before an epoch boundary)?

We could change that to

Were there at least $k$ blocks after the voting deadline or was the last applied block within the last $3k/f$ slots of the epoch?

This choice would have DST=Sensitive, PIL=Impossible and DSR=Insensitive (apart from chain growth violations in the $3k/f$ slots after the voting deadline).

[dnadales]

Were there at least $k$ blocks after the voting deadline ($6k/f$ before an epoch boundary)?

But don't we require 2k blocks after the voting deadline to actually transition to the next era?

[dnadales]

This choice would have DST=Sensitive

Why? Because we would be counting slots?

PIL=Impossible

Because we would be counting slots, like the ledger does?

[dnadales]

was the last applied block within the last $3k/f$ slots of the epoch?

and I don't get why this will ensure that a rollback won't cause a different era-transition decision (even if there are no CG violations).

(sorry for the many questions 🙏 )

[dnadales]

I fail to see the relation between chain growth violations and k + 1 blocks.

[nfrisby]

Right now governance actions are enacted if they both 1) get enough votes and 2) get those votes before a slot-based deadline. The specified choice of slot for the deadline has so far ensured that the final Yes-vs-No status of the governance action would be determined (ie predictable) more than k blocks before the change was actually made (aka "enacted")---unless there was a Chain Growth violation (since Chain Growth ensures at least k blocks of growth per a fixed number of slots).

[dnadales]

Let me see if I get this: if $b^d_1$ is the block at the voting deadline, where the votes are counted, then in the following chain:

$b^d_1 \rightarrow b^d_2 \rightarrow \ldots \rightarrow b^d_k \rightarrow b^d_{k+1}$

The vote counting is not stable until we reach/add? block $b^d_{k+1}$, right?

And does the insensitivity to ticks stem from the fact that we're counting blocks instead of slots?

[nfrisby]

The deadline is actually a boundary between slots. There's a slot that's the last slot you could sneak in a vote, and the next slot you can't. The boundary between them is the deadline.

If $b^d_1$ is the first block that is either in the slot immediately before that deadline or anywhere after it, then I think that diagram is correct. Because rolling back up to k blocks would still not allow the votes to change.

---

The vote counting is not stable until we reach/add? block $b^d_{k+1}$, right?

Hmm. The word "stable" is tricky, since it is often used to refer to something that's one stability window old. But in the presence of CG violations, that could still be subject to rollback.

So it's "actually stable" only once block k+1 arrives, yes.

And does the insensitivity to ticks stem from the fact that we're counting blocks instead of slots?

Yep.

[dnadales]

Now I realize that I'm confused about the mention of "youngest block", doesn't "block at the intersection" suffice?

[nfrisby]

Yeah; see our previous convo #346 (comment)

[dnadales]

Not related to this PR, but why do we require k+1 instead of k? Because we won't be able to switch to C otherwise?

[nfrisby]

Yep, that's why. Praos promises we'll never need to rollback more than k. And it's possible that (the tiebreakers align such that) we'd need to rollback exactly k in order to a longer chain, ie a chain with at least k+1 blocks after the intersection.

[dnadales]

Why do we require that L can judge the validity of at least k + 1 blocks on any chain that forks from it, but judge the validity of 3k / f slots only? CG ensures we have at least k blocks, but we could have exactly k. Does that mean that we can only switch to chains that have > k blocks in 3k/f slots?

[dnadales]

Hmmm, I think this is described below:

and hence k+1 blocks in total, as necessary.

Are we counting the intersection block as well? If so I'm still confused about the requirement that L can validate k + 1 blocks after the intersection 😬

[nfrisby]

We can tick L up to the next block on their chain. And then we can forecast 3k/f from there.

I think that's actually exactly one slot short 🤔 of ensuring L can validate k+1 headers on a chain that does not violate without having to apply any more blocks to L, unless the sparsity of the k headers after the first violate Chain Growth.

[dnadales]

I've drawn some pictures. I find they help me greatly in following the arguments presented here. Maybe we can add them to our documentation (but might not be needed for this PR).

And perhaps we should unify the notation for these diagrams 😄

[dnadales]

Here is a bit of context I'm missing: do we assume a certain duration of the epochs in terms of 3k/f (or k/f)?

[nfrisby]

Yes: Byron was 10k and every era after has been 10k/f slots per epoch. It could vary to other values in later eras, but no one within IOG has yet suggested it should.

[dnadales]

Without a drawing, it's quite hard for me to follow what L and C are. (I had to make one myself and I go to it back and forth, maybe we can spare the reader having to make her own drawing).

[dnadales]

We would also need some more context/background info on when nonces are determined in relation to the slot numbers or division of epoch into chunks of k/f.

[nfrisby]

It's currently acquired 3k/f before the end of the epoch. IOG Researchers want us to move it slightly earlier, ie 4k/f before the end.

[dnadales]

Why ticking would make this true?

[nfrisby]

Because you can only forecast so far, but you can tick indefinitely (in theory, at least). EG if there's a block in the slot 0 of an epoch, but there no block in the next 3k/f slots, then today's node wouldn't be able to lead even if it were elected in the next slot (ie slot 3k/f+1 of the epoch), since it couldn't forecast far enough to see that. On the other hand, it can tick to that slot.

(In theory ticking has no limit, but I think the Cardano ledger currently assumes you can't tick across more than one epoch boundary.)

[dnadales]

I don't get wahy this is the case. Would it be possible to illustrate this?

[nfrisby]

Claim: "No other behavior of the simplified node will change due to a Chain Growth violation."

It's claiming that the intersection of A and B is empty, where A is "behaviors that only arise in the presence of CG violations" and B is "behaviors that haven't already been discussed".

• If we change the leadership check to use ticking, then CG violations wouldn't affect it, and so the above claim would actually be true, since the leadership check is no longer member of A.

• If we instead also explained that (very extreme) CG violations can affect the leadership check, then the above claim would actually be true, since the leadership check is no longer a member of B.

[dnadales]

This section makes more sense after I read your answers to my questions. Also the second draft helped me a lot.

[dnadales]

What is the first trigger constraint? 😬

[nfrisby]

That the ledger governance decided the protocol major version would change.

[dnadales]

Does this mean that, when checking era transitions we'd be counting slots instead?

[dnadales]

I don't follow this. Even if we rely on the ledger's decision, we'd still need a double stability to ensure time-translations are sound. If not, a rollback could give a different answer to a time-translation.

[nfrisby]

We suspect we should rename the .md file, as of this second draft