WebSocket

Introduction

The aim of this document is to explain WebSocket integration details for Data Flow.

Please refer to any WebSocket documentation for detailed technical information about WebSocket itself. You can find one here.

• This sample application has been prepared with W3CWebSocket client in nodejs, so you can use most of it in your browser.
• Realtime audio recording has not been implemented in this example to keep it simple.
• You can find the code here:

sample-with-node.js

• You can find the sample audio file here.

stereo-8KHz.pcm

Example

With WebSocket API, you start a stream operation, send your data in small chunks and listen to responses as they are produced.

Before we start working, make sure to install WebSocket in our Node with either of these commands: yarn add websocket or npm install websocket

Add require() statements for the necessary libraries and create a WebSocket client object,

var W3CWebSocket = require('websocket').w3cwebsocket;
var fs = require('fs');
var client = new W3CWebSocket("wss://dataflow.eu.knovvu.com/project-runner");

Then let's add logs to onclose and onerror handlers for informational purposes

client.onerror = function(error)
{
  console.log(error);
};
client.onclose = function(error)
{
  console.log('Client Closed' + error.code + " - " + error.reason);
};

We can start the operation by sending the start message when the connection has been established:

var sampleRate = 8000;
...
function startOperation()
{
    if (client.readyState === client.OPEN)
    {
              client.send(JSON.stringify(
                {
                    "MessageName": "Start",
                    "ProjectName": "demo-stream",
                    "Token":"[token]",						
                    "Tenant":"[tenantId]",
                    "Audio":
                    {
                        "Encoding": "ulaw",
                        "SampleRate": sampleRate.toString(),
                        "ChannelCount": "2"
                    },
                    "Settings":{
                        "Punctuation": true,
                        "Emotion": true,
                        "Age": true,
                        "Gender": true
                    }
                }
            ));
    }
}
startOperation();
};

In the start message we provide any necessary information about the operation we want to perform.
This start message points to one of our default projects demo-stream. In the project you can see four constraints on connections. You can set these parameters to true or false under the Settings in this start message.

The rest is mostly event driven. We implement an on message handler for the WebSocket client.

client.onmessage = function(e)
{
  ...
  if(payload["MessageName"] === "StartResponse")
  ...
  else if(payload["MessageName"] === "FinalizeResponse")
  ...
  else if(payload["MessageName"] === "FinalResult")
  ...
  else if(payload["Event"] != null)
  ...
}

A brief observation would show us how a sample flow would occur. We start by handling the start response. If the response is successful, we go ahead and start sending the audio file piece by piece.

if(payload["MessageName"] === "StartResponse")
{
  if(payload["OperationResult"] === "Success")
  {
    console.log('successfully started a new operation');
  }
  else 
  {
    console.log('failed to start a new operation. Error message:  ' + payload["OperationResult"]);
	return;
  }
		
  console.log('reading pcm');
  audioData = fs.readFileSync('stereo-8KHz.pcm');
  console.log('file byte count : ' + audioData.length);		
  sendMoreAudioOrFinalize();
}

In this example for the sake of simplicity we send a prerecorded audio file in an approximately real-time pace. For a real world application, instead of an audio file the data source will probably be an IVR or a media platform and the arrival of the audio will be in real-time.

Getting the outputs is straightforward. Just check any incoming events and do whatever you want with it. You may want to check message format documentation.

else if(payload["Event"] != null)
{
  eventData = payload["Event"];
  console.log('received event : ' + eventData["Name"]);
}

It is also good practice to send finalize when you are done with the operation.

function finalize()
{
  client.send(JSON.stringify({"MessageName": "Finalize"}));
}

Server will respond with a FinalizeResponse message which basically confirms the operation has really been finished.