Skip to content

Latest commit

 

History

History
175 lines (122 loc) · 9.09 KB

README.md

File metadata and controls

175 lines (122 loc) · 9.09 KB

Amazon Lex sample Voice API application

This sample application allows you to call a phone number to interact with an Amazon Lex bot using Vonage Voice API, including getting real time transcripts and sentiment analysis.

This application uses a Lex reference connection code (more details below) for the actual 2-way audio interaction with the Lex bot.

Amazon Lex access

In order to get started, you will need to have an AWS account, as well as a bot on Amazon Lex.

Use an existing Amazon Lex bot or create a new one.

You will need your Lex bot name, as well as the Alias of the bot, which is located under Settings -> Aliases (when your bot page is open).

You will also need to know an active AWS Access Key ID and Secret Key pair.

If necessary, create a new pair of keys:

  • Log in to your AWS Management Console.
  • Click on your user name at the top right of the page.
  • Click on the My Security Credentials link from the drop-down menu.
  • Go to Access keys section,
  • Click on [Create New Access Key] (*)

(*) Note: Your AWS account may be limited to only 2 active Access Keys. To create a new pair of Keys, you may need to "Make Inactive" an existing active Access Key ID, however before doing so, you need to absolutely make sure that key is not used by your other applications.

About this sample application

This sample application makes use of Vonage Voice API to answer incoming voice calls and set up a websocket connection to stream audio to and from the Lex reference connection for each call.

The Lex reference connection code will:

  • Send audio to Lex bot from caller's speech,
  • Stream audio responses from the Lex bot to the caller via the websocket,
  • Post back in real time transcripts and optionally sentiment scores (if enabled in Lex Bot) via webhooks call back to this Voice API sample application.

Once this application will be running, you call in to the phone number linked to your application (as explained below) to interact via voice with your Lex bot.

Set up the Lex reference connection code - Host server public hostname and port

First set up a Lex reference connection code from https://github.com/nexmo-community/lex-reference-connection.

Default local (not public!) reference connection code port is: 5000.

If you plan to test using Local deployment with ngrok for both the Lex reference connection code and this sample application, you may set up multiple ngrok tunnels.

For the next steps, you will need:

  • The Lex reference connection code server's public hostname and if necessary public port,
    e.g. xxxxxxxx.ngrok.io, xxxxxxxx.herokuapp.com, myserver.mycompany.com:32000 (as LEX_REFERENCE_CONNECTION, no port is necessary with ngrok or heroku as public hostname)

Sample Voice API application - Host server public hostname and port

Default local (not public!) of this sample application port is: 8000.

If you plan to test using Local deployment with ngrok for both this sample application and the Lex reference connection code, you may set up multiple ngrok tunnels.

For the next steps, you will need:

  • The server's public hostname and if necessary public port on where this application is running,
    e.g. yyyyyyyy.ngrok.io, yyyyyyyy.herokuapp.com, myserver.mycompany.com:32001 (as host), no port is necessary with ngrok or heroku as public hostname.

Set up your Vonage Voice API application credentials and phone number

Log in to your or sign up for a Vonage APIs account.

Go to Your applications, access an existing application or + Create a new application.

Under Capabilities section (click on [Edit] if you do not see this section):

Enable Voice

  • Under Answer URL, leave HTTP GET, and enter https://<host>:<port>/answer (replace <host> and <port> with the public host name and if necessary public port of the server where this sample application is running)
  • Under Event URL, select HTTP POST, and enter https://<host>:<port>/event (replace <host> and <port> with the public host name and if necessary public port of the server where this sample application is running)
  • Click on [Generate public and private key] if you did not yet create or want new ones, save the private.key file in this application folder.
    IMPORTANT: Do not forget to click on [Save changes] at the bottom of the screen if you have created a new key set.
  • Link a phone number to this application if none has been linked to the application.

Please take note of your application ID and the linked phone number (as they are needed in the very next section.)

For the next steps, you will need:

  • Your application ID (as APP_ID),
  • The phone number linked to your application (your phone will call that number),
  • Your Vonage API key (as API_KEY)
  • Your Vonage API secret, not signature secret, (as API_SECRET)
  • The Lex reference connection code server public hostname and port (as LEX_REFERENCE_CONNECTION)
  • The AWS Access Key (as AWS_KEY)
  • The AWS Secret Key (as AWS_SECRET)
  • The Lex bot name (as BOT_NAME)
  • The Lex bot alias (as BOT_ALIAS)

Overview on how this sample Voice API application works

  • On an incoming call to the phone number linked to your application, GET /answer route plays a TTS greeting to the caller ("action": "talk"), then start a websocket connection to the Lex reference connection ("action": "connect"),
  • Once the websocket is established (GET /ws_event with status "answered"), it plays a TTS greeting to Lex bot, as Lex expects the user to speak first, we need to start the conversation as one would do in a phone call, with the answerer greeting the caller. The result is that the caller will immediately hear the Lex bot initial greeting (e.g. "How may I help you?") without having to say anything yet. You can customize that inital TTS played to Lex to correspond to your Lex bot programming and use case.
  • Transcript and sentiment scores will be received by this application in real time,
  • When the caller hangs up, both phone call leg and websocket leg will be automatically terminated.

The parameter sensitivity allows you to set the VAD (Voice Activity Detection) sensitivity from the most sensitive (value = 0) to the least sensitive (value = 3), this is an integer value.

The path portion of the uri in "action": "connect" is the same as the path to the PostContent endpoint within Lex but with your server host address, e.g. xxxxx.ngrok.io.
The botName is set by BOT_NAME argument,
the botAlias is set by BOT_ALIAS argument,
the userId may be set to any value as needed by your application logic, it cannot be an empty value.

Running Lex sample Voice API application

You may select one of the following 2 types of deployments.

Local deployment

To run your own instance of this sample application locally, you'll need an up-to-date version of Node.js (we tested with version 14.3.0).

Download this sample application code to a local folder, then go to that folder.

Copy the .env.example file over to a new file called .env:

cp .env.example .env

Edit .env file, and set the 8 parameter values:
API_KEY=
API_SECRET=
APP_ID=
LEX_REFERENCE_CONNECTION=
AWS_KEY=
AWS_SECRET=
BOT_NAME=
BOT_ALIAS=

Install dependencies once:

npm install

Launch the applicatiom:

node lex-voice-application

Command Line Heroku deployment

Install git.

Install Heroku command line and login to your Heroku account.

Download this sample application code to a local folder, then go to that folder.

If you do not yet have a local git repository, create one:

git init
git add .
git commit -am "initial"

Deploy this application to Heroku from the command line using the Heroku CLI:

heroku create myappname

On your Heroku dashboard where your application page is shown, click on Settings button, add the following Config Vars and set them with their respective values:
API_KEY
API_SECRET
APP_ID
LEX_REFERENCE_CONNECTION
AWS_KEY
AWS_SECRET
BOT_NAME
BOT_ALIAS

git push heroku master

On your Heroku dashboard where your application page is shown, click on Open App button, that hostname is the one to be used under your corresponding Vonage Voice API application Capabilities (click on your application, then [Edit]).
See more details in above section Set up your Vonage Voice API application credentials and phone number.