"Webhook" Vals

[404Wolf]

I was tinkering with setting up a twilio/sms webhook with valtown (still fighting SMS verification), and was thinking it would be really cool if you could abstract this type of trigger for vals. In brainstoring possible generalized triggers for notification like things (e.g. sms), I have arrived on the idea of a Webhook val. This is in some sense a generalization on Email vals.

Consider email vals, which for the sake of example we will assume are implemented using some sort of webhook with an email service of some sort (for this example, let's say aws SNS/SMS). That means that when you receive an email, aws's mail server notices, and POSTs val town, who then transforms the POST into an interface Email that vals get to see Emails as, and your val is called. The val provides you with the "trigger" for using it, which is an email address. You just implement a function to determine what to do when an Email is received, and the Email is the input to the function.

Now, consider a hypothetical SMS triggered val, implemented with a Webhook val. It also will ("internally" -- it is defined somewhere, but I don't have to think about it) handle receiving messy Twilio POSTs just like we assumed email vals do. It will also have a nice output schema, a ts object, so let's call it an Interface SMS for consistency. I will implement a function to determine what to do when an SMS is received. Since this is a "Webhook" val, Val Town will provide me with a unique Webhook URL. I can give that to twilio, and I'm set. Just like an email val, my function (e.g. webhookValHandler) will get SMS object as input.

Before continuing, I want to point out that Email vals provide just one output schema, Email, but that doesn't need to hold for Webhook vals. I wanted an analog to email for the SMS example, but note that there's no reason to limit ourselves to a single output. We can have multiple endpoints for our webhook, e.g. SMS, Voice, ...), where our Webhook definition will provide them all to our function as separate arguments.

To make your FIRST Webhook val for receiving SMS:

1. First, I'll go to my "Webhook" definitions (which could take the form of another val, or maybe a section in settings, or section of the website). I will
   1. Define HTTP handlers (sort of like an HTTP val, but we're returning javascript objects conforming to output schemas like Email). The endpoints will handle POSTs from, say, amazon, or Twilio, and pipe along (e.g. return) typescript objects.
   2. The output objects that these handlers return will conform to output Schemas defined as typescript interfaces (e.g. Email, SMS, Voice, ...).
      Then I'll also provide a description, which will have step by steps for how to GIVE Twilio the webhooks that will be generated by val town (e.g. A big chunk of text that says "go to homepage, go to phone numbers, ..,..,.. paste webhook into "receive sms" box)
2. Then, go to my dashboard. Click "Create New Webhook". Going forward (for making SMS vals), this is the ONLY step!
   1. A dialog shows up. "Choose Webhook." I am presented with a list of Webhooks I defined, so, wolf/SMS is at the top of the list, but ALSO, other people's definitions (if they are public), so, like, johnsmith/Stripe, or janedoe/Slack
   2. I click wolf/SMS. Just like an email val there is a "Copy Webhook" button in the top right of the val. When I click the button, or maybe before, I get a popup with instructions on how to give Twilio that Webhook (defined in 1.).
   3. The val is pre filled with all of the output functions we need/want:

export async function smsWebhookValHandler (s: SMS) { }
export async function voiceWebhookValHandler (v: Voice) { }

We can also crank this idea further for generalizing triggers. Like. if you guys add websockets, it would be cool if you can make abstractions with a similar format for things like discord bots I think the hardest part of executing this idea is that it requires some sort of "Webhook" center, but I wanted to share it, since I think it enables a lot of possibilities: Webhook discoverability. Github webhooks, or Telegram for example. It would be fun to see the possibilities, AND Reduced friction for generic-ish callbacks.

[stevekrouse]

I love these ideas! Zapier in particular gets a lot out of having generic triggers and actions.

There are a lot of vals that accept webhooks like what you're talking about, but because they also contain other "action"-specific stuff, they're not as re-composable as one might wish. For example this val handles stripe webhooks and then does other stuff: https://www.val.town/v/stevekrouse/newStripeEvent

This val similarly 'reacts' to new Twitter events and it's really only an implementation detail that it uses polling to create synthetic 'events': https://www.val.town/v/stevekrouse/twitterAlert.

It would be so cool to have abstractions that let folks recompose these triggers more fluidly than what we have now.

One thing that comes to mind is middleware. You could imagine a 'stripe webhook middleware' that does this processing so that you can import & wrap you handler to accept pre-processed stripe events

I think a good first start in this direction would be to make a series of these 'trigger vals' that only create these synthetic triggers with very comprehensive readmes to explain how to set them up. Then we could have a page that lists all of these.

We're also definitely considering Zapier-style triggers that we auto-configure for you via OAuth, but we'll probably need to hire a person or two before we have the bandwidth to experiment in that direction

[404Wolf]

Yeah, Zapier seems really cool, and the idea of doing oauth for the user to automatically set up the integrations is also pretty neat. They seem like a pretty great source of inspiration. I like this idea of creating "pipelines" of vals!

The fact that Val Town takes away all of the friction of defining endpoints definitely seems to make it a great choice for webhooks, but yeah, it would be nice to make handlers more modular, that's the idea behind this proposal. I think this middleware idea is neat, would you define the handler-endpoints in the "middleware vals," or would they just be for processing and piping along?
-- if it helps clarify my idea: for the twitter example, my idea would be instead of calling discordWebhook, to call receiver, an input to the function, and to use the twitter alert you would create some new "val town twitter alerts" val, and define its "trigger" to be the val you linked (but that calls receiver). Then you could very quickly create 15 different vals that use the same trigger for different things (they'd just all be different receivers).

Making trigger vals that call out to other vals to handle events sounds like a good first step, sort of like creating a 'design pattern' for the purpose, but how would you propagate the events to the handler? If the trigger vals process the inbound webhooks and and send for processing elsewhere, I think the "elsewhere" would most ideally be the thing that says to use those triggers in the first place (I think it makes sense for these webhook vals to declare what triggers them and who their middleware is, and not the other way around). If you mean to transform, serialize, and re-POST to an HTTP val, it just feels like we're back to square one (and then there's annoying security implications we have to handle two times in a row).

[pomdtr]

An addition to this idea: It would be nice to have a way to trigger vals from the val.town UI.

Ex: Adding a custom item to this section, that would create a POST request to one of my val, allowing me to process it however I want.