-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
aes-gcm: fix decryption bug, add properties #46 #75
Conversation
- Adds APIs to match the NIST spec - Adds properties of API equivalence - Adds properties that test decryption bug - Adds a batch file
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The version from
weaversa
had a requirement that P, IV, and A are all finite. Is that redundant if we have size upper bounds on each of them?
I assume you mean bounds like P <= 2^^39 - 256
? If so, then yes, that implies fin P
, so it would be redundant to say (fin P, P <= 2^^39 - 256)
.
Does it still allow negative infinity? Is the fact that it's used as a length enforce that it's non-negative?
All type-level numbers are non-negative, so you don't need to worry about negative infinity.
Style question: Taking / returning a named record vs a set of parameters. Why would we do one vs the other?
This is ultimately a matter of taste, so there aren't any hard guidelines. That being said, I think that if you start using tuples that are sufficiently large (say, four or more elements), then I would generally prefer using records at that point. Naming the fields makes it easier to tell at a glance what each field represents, and it confers the advantage that you can construct the record without needing to remember the exact order in which the fields were defined.
How can we make two type variables and constraint them to be the same, so that things look more like the human spec? (P = len_C)
I'm not quite sure what you mean here. Can you give an example of what you are trying to do?
Guidelines on when to use module-level parameters vs function-level?
In general, you'd make something a module-level parameter if it is shared among multiple functions in the module (e.g., the key size K
). Again, I think this comes down to a matter of taste.
How do I go about doing something useful with the equivalence properties in gcm.cry? Should I prove these in the batch file for some typical parameter sets? Is there a way to say for all?
You could try :check
'ing some of the properties. For certain properties, you might even be able to cover all possible inputs using :prove
, but I imagine that some properties will simply be too complicated to :prove
in a reasonable amount of time (this is where you'd need SAW to do better). But even just :check
would be better than nothing.
- switches from gross coerceSize function to nice numeric constraints - removes Option due to limited SAW support - adjusts some naming to more closely match spec
I was missing the type keyword. The method uses parameter |
Here are some follow-up notes:
|
I've attempted to answer the questions that I feel qualified to answer:
I am generally a fan of more tests, although I don't have a suggestion for how to make it more compelling than what you currently have.
Yes, I agree that that is a pretty compelling property to test, even if we only test it on specific inputs.
I can provide at least a partial explanation for this, since I recently modified the code in We could, in principle, enable the tests at other key sizes as well, but I think we'd need to parameterize the code in
Yes, we may as well test this.
We can certainly test properties over private functions. Note that Cryptol doesn't yet have a clean way to mark properties as private (see GaloisInc/cryptol#1621), but that needn't be a blocker.
I don't think there is any direct way to prove that, other than perhaps checking if the particular values of |
Thank you! Appreciate the context on the AES parameterization. I think there's a more general question here about whether we want to support smaller AES key sizes and if so, whether we're satisfied with the given structure -- 128 is mostly not recommended in practice but it's still part of Suite B so we might want to have easier access to it. I was looking at the Kyber implementation which is generic over parameters and then has a few modules that just define the standardized parameter sets. This is definitely out of scope for this PR but we might consider it if we come at this piece in the future.
I was wondering more if there's a way to say, like, a type is a 128-bit string drawn uniformly at random but that seems very hard to mechanically reason about. |
- separates private methods in GCM module - tries to make naming more canonical across properties - adds additional properties for AES-GCM - expands batch file and documentation for AES-GCM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is now ready for review!
I welcome nit-picky feedback on naming, spacing, and style from those who are more familiar with Cryptol standards than I am (e.g. everyone here).
Should I try to prove the 'check`ed properties before merging this? Do we have a standard setup on an external machine for doing so or would I just let it run overnight on my laptop?
Before I merge this I am going to make a separate issue collecting the concerns I noted in a previous comment so they don't get lost.
:check decryptionApisAreEquivalent `{K=128, IV=96, AAD=0, T=128, P=256} {E=AES::encrypt} | ||
:check encryptionApisAreEquivalent `{K=128, IV=96, AAD=0, T=128, P=256} {E=AES::encrypt} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style question: Should I parameterize these in the AES-GCM module with AES so we don't have to do it manually here? I did that for aesGcmIsSymmetric
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, that seems reasonable.
// This property is independent of the type parameters but we have to specify | ||
// them anyway. | ||
// It takes more than 25 minutes to prove. | ||
:check dotAndMultAreEquivalent `{K=128, IV=96, AAD=0, T=128} {E=AES::encrypt} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note: None of these parameters are relevant because this is just a function on 128-bit blocks. Should I make a separate batch file for it? Is there a way to reorganize the GCM module so that this property and the two relevant functions are not bound by the module parameters?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could imagine splitting up the module hierarchy more to make dotAndMultAreEquivalent
only parameterized over K
. That being said, it's unclear that doing so would pay its weight. Personally, I think it's fine to just invoke dotAndMultAreEquivalent `{K=128, IV=96, AAD=0, T=128}
like you do here.
// Set names to match the spec | ||
type s = len_IV %^ 128 // Equivalently: 128 * len_IV /^ 128 - len_IV | ||
|
||
/** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be nice to credit Sean in the copyright info above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed on this comment.
|
||
property AES_GCM_invalid_test_vector = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note: This is just a single negative test made by borking the tag from the fourth test vector. It's not exactly superseded by the symmetric property because that doesn't guarantee that there's only one correct tag for a given (key, iv, ct)
tuple. I'm going to make a note of this in the subsequent clean-up issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mccleeary-galois We talked a few days ago about proving properties that I thought would be fast but were not actually fast in practice. Not sure how much of a priority it is to dive into the SMT statements for these, but I'll tag them below for ya.
/** | ||
* Property demonstrating equivalence between `gcmDec` and `GCM_AD`. | ||
*/ | ||
decryptionApisAreEquivalent : {P} ( fin P, P <= 2^^39 - 256 ) | ||
=> [K] -> [IV] -> [P] -> [AAD] -> [T] -> Bool | ||
property decryptionApisAreEquivalent key iv ct aad tag = | ||
gcmDec { key=key, iv=iv, ct=ct, aad=aad, tag=tag} | ||
== GCM_AD key iv ct aad tag | ||
|
||
/** | ||
* Property demonstrating equivalence between `gcmEnc` and `GCM_AE`. | ||
*/ | ||
encryptionApisAreEquivalent : {P} ( fin P, P <= 2^^39 - 256 ) | ||
=> [K] -> [IV] -> [P] -> [AAD] -> Bool | ||
property encryptionApisAreEquivalent key iv pt aad = gcm_ae_output == (gcmEnc_output.ct, gcmEnc_output.tag) | ||
where | ||
input = { key=key, iv=iv, pt=pt, aad=aad } | ||
gcmEnc_output = gcmEnc input | ||
gcm_ae_output = GCM_AE key iv pt aad |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here are the definitions of the equivalence properties. They're parameterized and called in the batch file.
I expected these to be fast to prove because gcmEnc
calls GCM_AE
, but they are not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How long are these proofs taking at the moment?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have not run them to completion, but more than 25 minutes. I can set them spinning on my laptop and see if they end by the end of the day.
gcmIsSymmetric : {P} ( fin P, P < 2^^39 - 256 ) | ||
=> [K] -> [IV] -> [P] -> [AAD] -> Bool | ||
property gcmIsSymmetric key iv pt aad = dec.valid && (dec.pt == pt) | ||
where | ||
(ct, tag) = GCM_AE key iv pt aad | ||
dec = GCM_AD key iv ct aad tag |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here's the symmetric property. It's parameterized in AES_GCM.cry
and called in the batch file. I expected this to be fast-ish to prove because most of the steps of GCM-AE
are duplicated in GCM-AD
but it is not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should settle for a :check in the batch file at the moment.
// Set names to match the spec | ||
type s = len_IV %^ 128 // Equivalently: 128 * len_IV /^ 128 - len_IV | ||
|
||
/** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be nice to credit Sean in the copyright info above.
|
||
// This property is independent of the type parameters but we have to specify | ||
// them anyway. | ||
// It takes more than 25 minutes to prove. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you have a sense for what is causing it to be so slow?
In any case, it may be wise to reduce the number of random tests that we do in Cryptol to make this complete in a more reasonable amount of time. You can do this in a Cryptol batch file by running:
:set tests=<N>
The default value of <N>
is 100, so we could certainly stand to lower it if these tests take an excessively long time to complete.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, this isn't clear. The :check
here is fairly fast, but calling it with :prove
instead is very slow. I think if we want to claim that this is formally verified we'd need to eventually :prove
it. I don't know why :prove
is so slow, but I have no intuition for how Cryptol works or how to form properties that might prove faster or slower.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My apologies, I interpreted the comment above as applying to :check
rather than :prove
. In that case, the call to :check
here is fine as-is. (I am not surprised that it takes so long to prove, for reasons mentioned in #75 (comment).)
// This property is independent of the type parameters but we have to specify | ||
// them anyway. | ||
// It takes more than 25 minutes to prove. | ||
:check dotAndMultAreEquivalent `{K=128, IV=96, AAD=0, T=128} {E=AES::encrypt} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could imagine splitting up the module hierarchy more to make dotAndMultAreEquivalent
only parameterized over K
. That being said, it's unclear that doing so would pay its weight. Personally, I think it's fine to just invoke dotAndMultAreEquivalent `{K=128, IV=96, AAD=0, T=128}
like you do here.
:check decryptionApisAreEquivalent `{K=128, IV=96, AAD=0, T=128, P=256} {E=AES::encrypt} | ||
:check encryptionApisAreEquivalent `{K=128, IV=96, AAD=0, T=128, P=256} {E=AES::encrypt} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, that seems reasonable.
You can certainly try, but I wouldn't invest too much time in doing so. We've had to spend a considerable amount of effort to prove similar properties about AES-GCM in SAW, and this required manually rewriting proof goals. I would be very surprised if Cryptol could prove properties of a similar level of complexity without any assistance from a proof engineer. |
Agreed with @RyanGlScott here, I took a peak at this a bit and it would almost certainly require some proof engineering... I think leaving things as :check is good enough for now. |
This is a great step in the right direction for this! Generally LGTM for what the scope of this ticket is. |
// Set names to match the spec | ||
type s = len_IV %^ 128 // Equivalently: 128 * len_IV /^ 128 - len_IV | ||
|
||
/** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed on this comment.
/** | ||
* Property demonstrating equivalence between `gcmDec` and `GCM_AD`. | ||
*/ | ||
decryptionApisAreEquivalent : {P} ( fin P, P <= 2^^39 - 256 ) | ||
=> [K] -> [IV] -> [P] -> [AAD] -> [T] -> Bool | ||
property decryptionApisAreEquivalent key iv ct aad tag = | ||
gcmDec { key=key, iv=iv, ct=ct, aad=aad, tag=tag} | ||
== GCM_AD key iv ct aad tag | ||
|
||
/** | ||
* Property demonstrating equivalence between `gcmEnc` and `GCM_AE`. | ||
*/ | ||
encryptionApisAreEquivalent : {P} ( fin P, P <= 2^^39 - 256 ) | ||
=> [K] -> [IV] -> [P] -> [AAD] -> Bool | ||
property encryptionApisAreEquivalent key iv pt aad = gcm_ae_output == (gcmEnc_output.ct, gcmEnc_output.tag) | ||
where | ||
input = { key=key, iv=iv, pt=pt, aad=aad } | ||
gcmEnc_output = gcmEnc input | ||
gcm_ae_output = GCM_AE key iv pt aad |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How long are these proofs taking at the moment?
gcmIsSymmetric : {P} ( fin P, P < 2^^39 - 256 ) | ||
=> [K] -> [IV] -> [P] -> [AAD] -> Bool | ||
property gcmIsSymmetric key iv pt aad = dec.valid && (dec.pt == pt) | ||
where | ||
(ct, tag) = GCM_AE key iv pt aad | ||
dec = GCM_AD key iv ct aad tag |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should settle for a :check in the batch file at the moment.
- improves some documentation - moves parameterization of several properties into the aes-gcm module, instead of doing it on the fly in the batch file
Closes #46. Closes #74. Closes #47.
Here's an update to the AES-GCM spec that fixes the decryption bug (decryption was just implemented incorrectly, and it was never tested / proven to work anyway).
The existing version of AES is based on what I think is a submission document for the GCM algorithm. The format, algorithm names, and parameter names were changed at some point, and the official NIST standard uses some slightly different notation. I kept the old APIs in because it seems like they're being used, but changed the references and everything else to adhere to the NIST spec.
Conveniently, there were several correct implementations floating around:
GCM
decryption #47) adding a flag to this version to make decryption correctThis PR:
:check
ed in the file; I added docs with a note of how long I waited.This is now ready for review!
These questions have been answered:
weaversa
had a requirement that P, IV, and A are all finite. Is that redundant if we have size upper bounds on each of them? Does it still allow negative infinity? Is the fact that it's used as a length enforce that it's non-negative?I want to look into the following items: