Implement `verifySignature` #671

vmchale · 2019-03-01T20:40:32Z

This PR fixes the private/public key situation in wallet-api. The main changes are

Wallets are identified by private keys instead of Ints.
Signatures are attached to transactions (as per the spec), not the inputs of transactions.
Transactions have to be signed before submitting them to the blockchain.
verifySignature now has the correct number of arguments and works as expected.
plutus-tx has a SizedByteString n type to distinguish bytestrings of different lengths in PLC.

TO DOs:

j-mueller · 2019-03-27T11:43:25Z

note that the crowdfunding tests currently fail because of a bytestring size mismatch - verifySignature expectes the sig. to be 64 bytes but the default size for ByteStrings assigned by the plugin is 32 bytes.

To fix that I'm adding some newtypes for ByteStrings of length 32 and 64.

j-mueller · 2019-03-27T13:25:04Z

@michaelpj can you take alook at the last commit please -

the verify test fails with

        Exception: Type mismatch at () in term
          'arg_95'.
        Expected type
          '[(con bytestring) (con 32)]',
        found type
          '[(con bytestring) (con 64)]'

(See https://hydra.iohk.io/build/742161/nixlog/1) but I can't find the place where it would get the wrong size.

michaelpj · 2019-03-27T16:37:38Z

You haven't changed what the builtins machinery in language-plutus-core thinks the type of verifySignature is, I think.

michaelpj · 2019-03-27T16:38:10Z

We might need to ask @effectfully for help on what to change there.

effectfully · 2019-03-27T16:54:55Z

It doesn't look to me that the problem is in the builtins machinery, because verifySignature is fully polymorphic there:

typedVerifySignature :: TypedBuiltinName (BSL.ByteString -> BSL.ByteString -> BSL.ByteString -> EvaluationResult Bool) (EvaluationResult Bool)
typedVerifySignature =
    TypedBuiltinName VerifySignature $
        TypeSchemeAllSize $ \s0 -> TypeSchemeAllSize $ \s1 -> TypeSchemeAllSize $ \s2 ->
            TypeSchemeBuiltin (TypedBuiltinSized (SizeBound s0) TypedBuiltinSizedBS) `TypeSchemeArrow`
            TypeSchemeBuiltin (TypedBuiltinSized (SizeBound s1) TypedBuiltinSizedBS) `TypeSchemeArrow`
            TypeSchemeBuiltin (TypedBuiltinSized (SizeBound s2) TypedBuiltinSizedBS) `TypeSchemeArrow`
            TypeSchemeBuiltin TypedBuiltinDyn

Here is my random guess: from the definition of instSize I kinda expect the following

term <- wrapSizedBsrel [BS32, BS32, BS64] $ instSize haskellBS32Size $ instSize haskellBS32Size $ instSize haskellBS64Size $ mkBuiltin PLC.VerifySignature

to elaborate to

verifySignature {64} {32} {32}

rather than

verifySignature {32} {32} {64}

Is my guess correct?

effectfully · 2019-03-27T17:01:17Z

By the way,

-- | A 'BSL.ByteString' of 32 bytes.
newtype ByteString32 = ByteString32 { unByteString32 :: BSL.ByteString }
        deriving (Eq, Ord, Show, IsString)

-- | A 'BSL.ByteString' of 64 bytes.
newtype ByteString64 = ByteString64 { unByteString64 :: BSL.ByteString }
        deriving (Eq, Ord, Show, IsString)

this is terrible. Can we have

newtype Sized a (s :: k) = Sized a

and use Sized ByteString 32 and Sized ByteString 64 instead, so that we don't need to redefine all the functions (take, drop, etc) for distinctly sized bytestrings/integers?

If you decide to add this single newtype to handle everything, then please add it somewhere to language-plutus-core (PlutusPrelude.hs might be a fine place), so that I can add relevant KnownDynamicBuiltinType instances later.

michaelpj · 2019-03-27T17:07:32Z

I think you hit the nail on the head there.

Yes, this is terrible, but we're not brave enough to do the Sized thing yet. I'm going to look into it, but I think it's more important to get something working asap.

effectfully · 2019-03-27T18:06:40Z

I think you hit the nail on the head there.

Then I suggest to write it like that:

mkBuiltin PLC.VerifySignature `instSize` haskellBS32Size `instSize` haskellBS32Size `instSize` haskellBS64Size

Yes, this is terrible, but we're not brave enough to do the Sized thing yet. I'm going to look into it, but I think it's more important to get something working asap.

No problem, but I do not expect Sized to cause troubles here, it should provide a direct replacement.

michaelpj · 2019-03-27T18:41:27Z

Well, it won't work directly since the way the compiler plugin directly identifies type constructors with types. So it's not able to have special cases for Sized ByteString 64 and Sized ByteString 32 without some work.

data SizedByteString (s::k) = SizedBytestring ByteString

would probably work, but I would still need to work out how to get the sizes in the type system etc. So - some work.

effectfully · 2019-03-27T18:45:56Z

Ah, I see, thanks.

j-mueller · 2019-03-28T07:07:20Z

@effectfully

Is my guess correct?

Yes it is, thanks.

I thought about a type parameter for the sizes too but I decided to go with the solution that had a smaller risk of failure :)

j-mueller · 2019-03-29T14:33:48Z

@vmchale I updated plutus-use-cases so you can use that for profiling (the crowdfunding contract)

looks like everything is working, so I think now is a good time to rebase

michaelpj · 2019-03-29T19:36:33Z

I pushed a commit that adds type-level sizes on the Haskell side, and converts them over properly. So we don't need anything special, and a bunch of size-polymorphic stuff Just Works.

Once this is merged I'll do the same for integers, but I don't want to hold this up.

michaelpj

Broadly good. We may want to merge and address my comments later.

michaelpj · 2019-04-01T08:43:54Z

wallet-api/src/Ledger/Crypto.hs

 makeLift ''PubKey

+-- | A cryptographic private key.
+newtype PrivateKey = PrivateKey { getPrivateKey :: KeyBytes }


Both this and the public key should be 32-byte segments of the key bytes, no?

I changed it to use the PublicKey and PrivateKey types everywhere so KeyBytes type isn't actually necessary anymore. I think that's better than having a 64-byte ByteString with two dedicated segments.

Okay, but the PublicKey should definitely not contain the private key in case we serialize it.

Hm, I just realised something. We do actually serialise private keys now. (To identify wallets in the Playground). But I think I'll change it so that wallets are still Ints and to get a wallet's private key we look up the index in the list of known private keys.

I think that's benign, since it's just in the playground which isn't meant to be secure.

Yeah but keeping wallets as Ints has the advantage that I don't need to update the playground code ;)

michaelpj · 2019-04-01T08:45:44Z

wallet-api/src/Ledger/Crypto.hs

-signedBy (Signature k) (PubKey s) = k == s
+signedBy :: Signature -> PubKey -> TxId -> Bool
+signedBy (Signature s) (PubKey k) txId =
+    let k' = ED25519.publicKey $ BSL.toStrict $ Builtins.unSizedByteString $ KB.getKeyBytes k


i.e. I'd probably do this stuff in the construction of PublicKey. Note that the current version is potentially insecure, since we might put the PublicKey somewhere public even though it actually contains the private key too!

michaelpj · 2019-04-01T08:48:08Z

wallet-api/src/Ledger/Crypto.hs

+signedBy (Signature s) (PubKey k) txId =
+    let k' = ED25519.publicKey $ BSL.toStrict $ Builtins.unSizedByteString $ KB.getKeyBytes k
+        s' = ED25519.signature $ BSL.toStrict $ Builtins.unSizedByteString s
+    in throwCryptoError $ ED25519.verify <$> k' <*> pure (getTxId txId) <*> s' -- TODO: is this what we want


Good question. I think ideally we'd handle these errors explicitly. This isn't a very exceptional error, and if we throw a bunch of exceptions then we have to write a robust harness in the executable, since we still want to send a response in case of an error, so it can't kill the whole executable.

michaelpj · 2019-04-01T08:51:53Z

wallet-api/src/Ledger/Crypto.hs

+    let k  = ED25519.secretKey $ BSL.toStrict $ Builtins.unSizedByteString $ KB.getKeyBytes privKey
+        pk = ED25519.toPublic <$> k
+        salt :: BS.ByteString
+        salt = "" -- TODO: do we need better salt?


I think this doesn't actually matter too much. AIUI, there are two reasons we might want a salt:

To get a different signature for each message, to make it harder to reuse signatures. This isn't so much of a problem for us since the transaction itself is only valid once.

To prevent replay attacks. Again, this isn't so much of a problem for us since transactions are only valid once, and in fact having replayability is actually helpful for us.

We could get a salt that's replayable by us by having it be pseudo-randomly generated with a seed that's passed in by the caller. Then the contract backend can still replay everything deterministically.

wallet-api/src/Ledger/Crypto.hs

wallet-api/src/Ledger/Index.hs

wallet-api/src/Wallet/Generators.hs

michaelpj · 2019-04-01T09:08:32Z

wallet-api/src/Ledger/Tx.hs

    -- ^ The 'SlotRange' during which this transaction may be validated.
+    txSignatures :: Map PubKey Signature


I think we hash the transaction by using the derived Serialise instance. But we should be signing something that doesn't include the signatures, which that does. In particular, I'm not sure how we ever validate any signatures, since I'd expect us to recalculate the hash later and get a different thing...

Also, we don't want to just rely on the map being empty, since that way we can add more signatures later without having to contort ourselves.

The hash is of a TxStripped which excludes the signatures.

https://github.com/input-output-hk/plutus/blob/7ab2473b9b1856d0d8307916a85774810562a48a/wallet-api/src/Ledger/Tx.hs#L194

Although it might be better to just add a type parameter for the signatures to Tx...

Or have TxBody and Tx, which is the terminology that the specifications use, I think?

michaelpj · 2019-04-04T09:04:28Z

I think marlowe just needs to have its compile warnings fixed.

j-mueller · 2019-04-04T09:13:09Z

Right, I'll do that

* Add verifySignature * Add swagger bits * Use KeyBytes pervasively * Fix up wallet-api library * Test suite builds + fails * Add fromHex * Fix four test cases * More sensible API * Start signing mechanisms * Remove ownSignature * Remove unneeded functions * rearrange by secret key * Patch up types * Remove bit we don't need * Use PubKey * Use signature in pubKeyTxIn * Use bottom * cardano-crypto and http-api-data * transactions have signatures * Show KeyBytes * PublicKeySize invalid * wallet-api tests pass * fix plutus-use-cases

fix plutus-use-cases fix warnings keybytes exports game contract types

* PS bridge / JSON * Marlowe

j-mueller · 2019-04-04T12:45:27Z

marlow test failures are odd, I was able to build it without problems:

michaelpj · 2019-04-04T12:46:58Z

Ah - the marlowe tests are split out into another derivation, so that won't run the tests: you need localPackges.marlowe.testrun. I think the failure is genuine.

michaelpj · 2019-04-04T12:48:07Z

It's something weird where we're not producing a transaction when we think we should. Also I guess this means the timing out is also real.

michaelpj · 2019-04-04T12:52:08Z

@nau this looks suspicious: https://github.com/input-output-hk/plutus/pull/671/files#diff-f1f2d09538c116215822ad173fa89638R40

It looks like the ids are coming apart from the person's public key, is that right?

michaelpj · 2019-04-04T12:54:17Z

I wonder whether something in the test might be causing all the contracts to fail by timeouts? Which might be very slow in the property test.

michaelpj · 2019-04-04T14:21:35Z

Compile warnings for the stuff you commented out 🙈

j-mueller · 2019-04-04T14:22:37Z

it should be OK with the latest commit (cec125e)

michaelpj · 2019-04-04T14:24:19Z

👍

michaelpj · 2019-04-04T14:24:46Z

Stylish haskell - last thing!

michaelpj · 2019-04-04T14:29:09Z

\o/

michaelpj · 2019-04-04T14:29:28Z

What a slog, great work all. Will merge shortly.

j-mueller self-assigned this Mar 26, 2019

j-mueller force-pushed the plutus-core/signature-implementation branch from 1a32d9d to bc4f38d Compare March 26, 2019 14:51

vmchale self-assigned this Mar 26, 2019

j-mueller force-pushed the plutus-core/signature-implementation branch 4 times, most recently from 3125161 to 7ab2473 Compare March 31, 2019 18:17

michaelpj marked this pull request as ready for review April 1, 2019 08:39

michaelpj reviewed Apr 1, 2019

View reviewed changes

j-mueller force-pushed the plutus-core/signature-implementation branch 4 times, most recently from 901658f to c34069c Compare April 2, 2019 07:50

Vanessa McHale and others added 2 commits April 4, 2019 11:43

plutus-tx: sized bytestrings

3dabf5e

fix plutus-use-cases fix warnings keybytes exports game contract types

j-mueller and others added 2 commits April 4, 2019 11:52

plyaground related fixes

948328a

Migrate Marlowe to new signature API. Tests fail though.

98f20f7

j-mueller force-pushed the plutus-core/signature-implementation branch 2 times, most recently from 050357c to b2f3963 Compare April 4, 2019 10:05

j-mueller changed the title ~~Partial signature implementation~~ Implement verifySignature Apr 4, 2019

fix remaining issues

f56e681

* PS bridge / JSON * Marlowe

j-mueller force-pushed the plutus-core/signature-implementation branch from b2f3963 to f56e681 Compare April 4, 2019 10:20

plutus-tx: implement hash and crypto primitives

197546a

j-mueller force-pushed the plutus-core/signature-implementation branch 2 times, most recently from f83740a to 235da5f Compare April 4, 2019 14:16

Take marlowe tests out for now

cec125e

j-mueller force-pushed the plutus-core/signature-implementation branch from 235da5f to cec125e Compare April 4, 2019 14:20

michaelpj merged commit 5c4a4dd into master Apr 4, 2019

j-mueller deleted the plutus-core/signature-implementation branch April 4, 2019 15:14

		-- ^ The 'SlotRange' during which this transaction may be validated.
		txSignatures :: Map PubKey Signature

Implement verifySignature #671

Implement verifySignature #671

Conversation

vmchale commented Mar 1, 2019 • edited by j-mueller Loading

j-mueller commented Mar 27, 2019

j-mueller commented Mar 27, 2019

michaelpj commented Mar 27, 2019

michaelpj commented Mar 27, 2019

effectfully commented Mar 27, 2019 • edited Loading

effectfully commented Mar 27, 2019

michaelpj commented Mar 27, 2019

effectfully commented Mar 27, 2019

michaelpj commented Mar 27, 2019

effectfully commented Mar 27, 2019

j-mueller commented Mar 28, 2019

j-mueller commented Mar 29, 2019

michaelpj commented Mar 29, 2019

michaelpj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michaelpj commented Apr 4, 2019

j-mueller commented Apr 4, 2019

j-mueller commented Apr 4, 2019

michaelpj commented Apr 4, 2019

michaelpj commented Apr 4, 2019

michaelpj commented Apr 4, 2019

michaelpj commented Apr 4, 2019

michaelpj commented Apr 4, 2019

j-mueller commented Apr 4, 2019

michaelpj commented Apr 4, 2019

michaelpj commented Apr 4, 2019

michaelpj commented Apr 4, 2019

michaelpj commented Apr 4, 2019

Implement `verifySignature` #671

Implement `verifySignature` #671

vmchale commented Mar 1, 2019 •

edited by j-mueller

Loading

effectfully commented Mar 27, 2019 •

edited

Loading