Conversation Filtering Proposal

[neekolas]

We are working on adding a much-requested feature to the XMTP protocol, and wanted to get developer feedback on our proposed solution. Many developers have asked us how they can group or filter messages in a particular context. Some examples:

• A marketplace app wanting to only show messages related to a particular item
• A customer support app wanting to group messages by case ID
• A developer wanting to only show conversations originating from within their app
• Developers using bespoke custom content types who would only like to see messages with that content type

While developers can use metadata in a custom content type to filter messages after they have been retrieved from our nodes, this can lead to inefficient performance (they have to retrieve all messages and then ignore the ones that don't match). It's also a worse developer experience (all the filtering logic lives in the application).

Traditional methods of server-side filtering aren't an option for us since messages are end-to-end encrypted and the nodes are unaware of the contents developers might want to use as predicates of a filter. So, we have developed a solution to this problem that does not compromise user privacy.

Proposed solution

We propose several changes to the Conversations API that will allow for developers to add an identifier to conversations, attach metadata to a conversation, and to filter conversations based on the identifier.

The exact API is still being iterated on, and anything below may change before release.

Here's a code example to illustrate how this might work for an app that wants to offer two types of conversations: one for "foo" messages and one for "bar" messages.

type ConversationContext = {
  // Messages will be grouped by conversation ID
  conversationId: string
  // Additional metadata available to developers, but not used in filtering
  metadata: {
    [k: string]: string
  }
}

const fooCtx: ConversationContext = {
  conversationId: 'myapp.xyz/foo',
  metadata: {
    title: 'foo conversation',
  },
}
// Get or create a "foo" conversation with blockchain address 0x123
const fooConvo = await client.conversations.findOrCreate('0x123', fooCtx)
await fooConvo.send('gm')
// Stream new conversations with a predicate
for await (const convo of client.conversations.stream(predicate)) {
  console.log(convo.conversationId)
}

// Get or create a "bar" conversation with a different user
const barConvo = await client.conversations.findOrCreate('0x456', {
  conversationId: 'myapp.xyz/bar',
  metadata: {
    title: 'bar conversation',
  },
})

// Get all the "bar" messages. This will not retrieve any "foo" messages from the nodes
const barMessages = await barConvo.messages()

type IdFilter = (conversationId: string) => boolean
// A conversation predicate can be used to filter conversations in list endpoints
type ConversationPredicate = {
  addresses?: string[]
  conversationIds: string[] | IdFilter
}

// List all my foo conversations with friends, using a ConversationPredicate
const predicate = {
  conversationIds: ['myapp.xyz/foo'], // multiple types will be used as an OR query
}
const fooConvos = await client.conversations.list(predicate) // would return one conversation

// List all the conversations from myapp.xyz, by checking the prefix of the type using an IdFilter
const myAppConvos = await client.conversations.list({
  conversationIds: (conversationId: string) =>
    conversationId.startsWith('myapp.xyz'),
})
// would return two conversations
for (const convo of myAppConvos) {
  console.log(convo.peerAddress)
  console.log(convo.id)
}

// Get the foo conversation with 0x123
const myConvosWith123 = await client.conversations.list({
  addresses: ['0x123'],
  conversationIds: ['myapp.xyz/foo'],
}) // would return one result

How it works

Before digging in to how this new system works, it'll be helpful to understand how XMTP works today.

XMTP Today

Topics

All messages on XMTP are stored on a topic. Topics are public. Conversations are initiated by sending a message to 3 different topics:

• sender's intro-topic - intro-walletA
• recipient's intro-topic - intro-walletB
• conversation topic - dm-walletA-walletB

Subsequent messages are sent to the conversation topic only. There can be multiple introductory messages from the same sender in a recipient's intro topic. All further communication after the initial message in a user session will go into the same conversation topic as it is strictly tied to participants' wallet addresses.

Messages

Conversation messages include a plain text header identifying the sender's and recipient's public key bundle used to encrypt the message. The header aids decryption by either party. The payload is encrypted.

Upcoming changes

In order to facilitate conversation tagging and enhance user privacy we are introducing a new way of handling introductions and message encryption.

Invite topics

Replace the intro topics with invite topics. Messages in the invite topics will carry an invitation from the initiator of the conversation which will include:

• randomly generated topic name that will be used for the conversation

  • make sure the name is sufficiently large to make the topic space sufficiently sparse so that it is hard to randomly guess topics (32 bytes).
  • the name MUST be generated with sufficient randomness to make topic name collisions with other invitations highly improbable

• randomly generated bytes used as the key material for encryption of messages in this conversation

The invitation message is encrypted using the X3DH scheme we use for messages currently.

The initiator still sends the invitation to their own invite topic as well to have a record of the the topic and key material. The invitation message would be identical, i.e. the sender is still the initiator, which can be useful to distinguish own invitations from others.

Invitations include a context field that contains the conversationId and metadata about the invitation. That data is used by the client to filter invitations, which then returns a filtered list of topics that can be used to query the nodes.

message InvitationV1 {
    // Supported encryption schemes
    // AES256-GCM-HKDF-SHA256
    message Aes256gcmHkdfsha256 {
        bytes key_material = 1; // randomly generated key material (32 bytes)
    }

    // The Tag type
    message Context {
        // Expected to be a URI (ie xmtp.org/convo1)
        string conversation_id = 1;
        // Key value map of additional metadata that would be exposed to
        // application developers but not used for filtering
        map<string, string> metadata = 2;
    }
    // topic name chosen for this conversion.
    // It MUST be randomly generated bytes (length >= 32),
    // then base64 encoded without padding
    string topic = 1;

    // A context object defining metadata and conversation ID
    Context context = 2;

    // message encryption scheme and keys for this conversation.
    oneof encryption {
        // Specify the encryption method to process the key material properly.
        Aes256gcmHkdfsha256 aes256_gcm_hkdf_sha256 = 3;
    }
}

Conversation Topics

Topic name is determined by the invitation that initiated it (expected to be a random string). Messages in the topic will omit the sender/recipient pub key bundles from the message header. The payload is encrypted using the scheme indicated by the invitation (e.g. AES256-GCM-HKDF-SHA256). The key material from the invitation is used as the input into the scheme's key derivation function (e.g. HKDF).

The omitted sender header is replaced with a new SignedContent wrapper that will embed EncodedContent and will be encrypted inside Ciphertext.payload.

message SignedContent {
    bytes content = 1; // contains EncodedContent
    PublicKeyBundle sender = 2;
    Signature signature = 3;
}

The signature signs a digest of concatenated message timestamp, topic name and the content bytes using the sender's pre-key.

Caveats

• There is no validation or central registry of conversationIdss. We suggest that developers use a URI as a convention, but are not able to enforce restrictions as the conversationId only exists inside an encrypted payload. Collisions of conversationId are possible if multiple developers send messages using the same type between the same two addresses, and it will be up to the application developer to handle any unexpected messages resulting from those collisions.
• Some conventions will need to be formed around conversationIds and metadata to enable interoperability between applications
• Conversation metadata is restricted to Map<string, string> values
• Conversation metadata cannot be changed after the conversation has been initiated
• This is explicitly for adding metadata to conversations. If you need to add more data to individual messages, the best path forward is still to use content types
• Some of the specifics of the API may change before we ship the feature

Questions for developers

• What would you use this for in your app?
• For those use-cases, what would you use as the conversation_id
• What would you see yourself including in the metadata

[joshstevens19]

I really love the vision and excellent work, here are a few thoughts of mine:

Metadata

Standard

metadata, in theory, is just a lump of data the user wishes to attach to a conversation, I like how you're allowing them to pass in the metadata like this but would it be nice to set a metadata standard XMTP for this? XMTP is a single place MANY UIs can build and consume, I can make a UI allowing people to see all their messages and someone in the US can do this a million times over, what this means is imagine you didn't have a standard, and my UI sets "titles" another UI sets "displayName" another UI sets "header", in theory they mean the same thing but now the UIs won't be in sync with each other, UI 1 will show no title and UI 2 will. Standards solve this. What I propose is you flesh out this interface:

metadata: {
    [k: string]: string
}

Into a standard required interface (everything can be optional) but your setting the standards for developers which you can keep building on. Without a standard, it could become a huge mess and UIs won't be able to just build and inherit each other. As a user, if I find another cool UI and it didn't have all my state would I use it? maybe not.

Changing metadata

"Conversation metadata cannot be changed after the conversation has been initiated"

I think this is fine for 1<>1 DMs naturally you do not name it and naturally, you use the person you talking to profile pic but when you start doing group chats changing this metadata IMO would be super important. Like WhatsApp, a subset of users has admin permission to change that stuff can come later for sure but just flagging.

Filtering

I think is 100% fine to filter based on a naming standard you set. I am happy with the interface with the way you did conversationIds: (conversationId: string) it allows you to do a regex to make it stronger validation (which you need as either party can start chats meaning even with a generated name it won't ever be a 1 fits all, a regex fits better). Also, would we be able to make this a promise? i was thinking you could add more logic in that, for example, if you wanted to do an API call on hook and abstract this away nicely in the lambda.

I would also entertain the idea if you already have this info to pass in the conversation context it would be very useful, only if you have it at this point you may not.

type IdFilter = (conversationId: string, convo: Convo) => boolean

You can imagine this being useful in terms of making sure the from user still owns the identity for example a lens profile or not but you can of course do this a step after but could be useful to have that info (even if you don't have the entire convo and just have the peerAddress who it is from, as said you may not have it if so ignore).

Other thoughts

• Do you guys support paging for historic messages + all conversations? I imagine loading this amount in memory would be pretty intense on a client and in some cases probably break a few things. it would be perfect if we had paging for all this stuff to allow it to stay lightweight. (maybe a thing already)

overall super happy with this think it solves a lot of things, great work guys!

[neekolas]

@joshstevens19 appreciate the thoughtful comments.

Metadata

Standards solve this. What I propose is you flesh out this interface:

Definitely something I've been concerned about as well. I don't think the standards can come top-down, or in advance of real-world usage of the feature. If I try and dream up what fields developers are going to need, I am 100% going to be wrong. I'd love to see standards evolve around actual implementations of the protocol.

This is V1 of the invitation protocol, and I'm inclined to leave it with maximum flexibility. The XIP Process feels like the right way to elevate developer conventions into first-class citizens on the invite as structured fields for V2.

Changing metadata

I think this is fine for 1<>1 DMs naturally you do not name it and naturally, you use the person you talking to profile pic but when you start doing group chats changing this metadata IMO would be super important

100% agree. As part of enabling group chat, we are going to need some more flexible ways of mutating ongoing conversations (to support adding and removing members). The goal would be to fold metadata updates into that framework once it's available.

Filtering

Also, would we be able to make this a promise?

Great suggestion. I can totally see the use-case for asynchronous filtering. That should be workable. The performance concerns (waiting for the event loop) should be manageable given this is not an operation that has to run very frequently and that can be batched with Promise.all

I would also entertain the idea if you already have this info to pass in the conversation context it would be very useful, only if you have it at this point you may not.

We do have all the information since we have already downloaded the invites at the point we are running the filters. At least for the list and stream APIs, I don't have a problem with passing the full object. One thing I am trying to preserve is that the conversation_id should be all that is required when using findOrCreate. That will simplify the matching logic if we need to later stitch together multiple invitations for a single topic (which would be required for key/topic rotation)

Do you guys support paging for historic messages + all conversations? I imagine loading this amount in memory would be pretty intense on a client and in some cases probably break a few things. it would be perfect if we had paging for all this stuff to allow it to stay lightweight. (maybe a thing already)

We support paging for messages today. Paging for conversations is a "maybe". The reason being that we may have to do a pre-fetch of all the invites in order to support grouping together multiple invites for the same conversation (to handle topic/key rotations). Definitely a hot topic internally right now.

[joshstevens19]

oh cool i agree with everything you saying, the only thing I have to say is about the

One thing I am trying to preserve is that the conversation_id should be all that is required when using findOrCreate. That will simplify the matching logic if we need to later stitch together multiple invitations for a single topic (which would be required for key/topic rotation)

I get what you're saying here but I can see an example where in LENS you may want to know more than just the id, and you may wish to filter in 1 place while it's looping over already instead of having to do another filter again later once you got the results.

• You grab back all the conversations for the wallet
• You filter on the profile ID with a regex on the conversation ID, this allows you to only bring back relevant chats to the profile you're on
• You extract the profile id you are having a conversation with as well
• You now need to check that the profile id is still owned by the peerAddress this conversation is with, if it is not you want to filter this away as well or do something special with it, this is where the full convo would be useful to be able to have this logic in a single place.

That is just our use case I can imagine others have similar when trying to detach the messages from the wallet itself.. will leave that up to you thought! nice work again!

[killthebuddh4]

A few quick questions/thoughts.

One imminently useful standard would be something like a "default conversation id". Or maybe it could make sense for XMTP to "officially recommend" creating conversations without conversation ids unless there is some good reason to do so. If every new XMTP application developer creates conversations with conversationId: mynewsite.com, it could quickly undermine the awesomeness of the "universal inbox" that XMTP supports. I haven't thought that much about what "some good reason" means in this case, but I'd be happy to help think that through if my point here resonates.

A related but different question is: is the conversationId intended to be exposed to the user? I was considering a situation where I have 3 conversations between user A and user B, 2 of which have conversationIds that I know nothing about. What would a good UX be in this situation? Without a standard default conversation or knowing anything about the conversationIds, I was thinking some sort of dropdown menu would be useful, but what would I display to the user? Should I assume the conversationId will be recognizable to the user?

Last, less related question: It looks like the conversation filtering might not have made it into v7, correct? I guess filtering will have to happen on the client anyways, so maybe adding predicates to the SDK API wouldn't get us much more than filtering the conversations manually anyways?

[darickdang]

I was considering a situation where I have 3 conversations between user A and user B, 2 of which have conversationIds that I know nothing about. What would a good UX be in this situation? Without a standard default conversation or knowing anything about the conversationIds, I was thinking some sort of dropdown menu would be useful, but what would I display to the user? Should I assume the conversationId will be recognizable to the user?

Seeing conversations in their respective domains should be the default since they'd be displayed within the context of the service/platform they're being used in. Filtering all of a user's conversations as per-domain vs "All conversations" would be the baseline UX where a user could see conversations specific to the platform/service they're happening on but then being able to toggle over to "All conversations" to see the rest of their conversations wherever/however they received them.

This could happen with a toggle/tab at the conversation inbox list level. This would also help ensure the experience at a per-domain level but also give a user access to all their messages should they need to reference them.

However, allowing a user to see all the interactions and conversations with a particular wallet address could be beneficial as well, especially if it's a wallet they frequently interact with (ie. a friend/colleague). I don't think it'd necessarily be a dropdown to choose which conversations to display (not sure if a user would need/want that granularity) but instead using a toggle element to turn on/off seeing only per-domain conversations (the one they're currently messaging on) or all conversations with a specific wallet address.

[killthebuddh4]

I think I follow, but I also get the feeling I'll need to make some assumptions about the conversationIds that I can't count on to be strictly true. Thanks for the thoughts!

[neekolas]

Last, less related question: It looks like the conversation filtering might not have made it into v7, correct? I guess filtering will have to happen on the client anyways, so maybe adding predicates to the SDK API wouldn't get us much more than filtering the conversations manually anyways?

Exactly. It was adding no additional value over filtering the array, so I cut it.

[neekolas]

Should I assume the conversationId will be recognizable to the user?

You can expect the domain to be recognizable. Everything past that part I would assume to be not human readable.