VB Realtime

Updated on 19 Jul 2024
2 Minutes to read
Contributors

Article summary

Did you find this summary helpful?

Thank you for your feedback

Integration node for Voice Biometrics (VB) Realtime.

Parameters:

name	description	default
Address	Address of Voice Biometrics	https://vb.eu.knovvu.com
IgnoreSslErrors	Ignore any certificate errors if VB Realtime address contains "https"	false
CredentialId	Id of the credential that will be used for VB Realtime http calls.	None
AudioSegmentSizeMsec	The audio package length that will be sent to VB in milliseconds	5000
SelfAddress	Data Flow CallBack address. You need to change this if you are using a VB deployment in a different Kubernetes cluster than Data Flow	http://{PrivateIp}:{PrivateManagementPort}/

Inputs

Audio:
Expects Single-Channel Audio that has gone through a VAD Node.

Events:

name	description	possible nodes this event may be received from
VBStreamState	This event informs the node if it should start or stop streaming audio to Voice Biometrics	External Relay
Speech Started	Signals the start of speech in realtime	VAD
Speech Ended	Signals the end of speech in realtime	VAD

Outputs

Audio:
none

Events:
none

Remarks :

Requirements :

"StartMessage" requirements

"StartMessage" should contain

"CallProperties.UserProperties.ExternalId" This let's Voice Biometrics know which Agent is currently in call this session.
"CallProperties.ExternalId" This conversation id is given to Data Flow by the Client. This lets Voice Biometrics know about which specific conversation the VB Analysis will take place.
Definitions for ChannelTags. This specifies which channel belongs to the customer and which to the agent.
Every other "CallProperties" property is optional, but nice to have.

  {
    "CallProperties": 
    {
      "UserProperties":
      {
        "ExternalId":"AgentsExternalId"
      },
      "ExternalId":"ConversationId"
    },
    "Settings":
    {
      "ChannelTags": 
      [
        "a_tag_for_channel_0",
        "a_tag_for_channel_1;another_tag_for_channel_1"
      ]
    }    
  }

"StartMessage" example

{
    "CallProperties": 
    {
      "UserProperties":
      {
        "ExternalId":"AnExternalId"
      },
      "ExternalId":"AnExternalIdForConversation"
    },
    "Settings":
    {
      "ChannelTags": 
      [
        "customer",
        "agent"
      ]
    }    
 }

Project Structure
VB Realtime node expects to receive audio that went through a VAD from a single channel. So, in projects with multiple channels, multiple VB Realtime nodes that are connected to different VAD nodes should be used. In cases with multiple VB Realtime nodes, a Channel Switch Node is recommended for starting transactions on the intended channel .

A minimum viable project that utilizes VB Realtime node can be built as such:

An example of using two VB Realtime nodes that enables user to authenticate either of the channels during runtime can be built as such:

Finally, an example of utilizing VB Realtime Node for authenticating specifically the "customer" channel for our default project "ca-online" can be seen below.

Detailed Workflow
VB Realtime node is a node that can send real time audio data to Voice Biometrics in real-time.

This node registers the current call to Voice Biometrics using
/biometrics/api/votg/v1/calls/{callId}
The same endpoint is used to unregister the call at the end of the call.

After Register, the node enters the "Idle" state and waits for the transaction started event from the External Relay.
Once the "VBStreamStateEvent.Data.State == Started" event is received, node enters "Stream" state for transaction Id: "VBStreamStateEvent.Data.TransactionId".
During the stream state, The node accumulates audio and packs them in lengths of "AudioSegmentSizeMsec". The node sends these audio packs to the endpoint:
"/biometrics/api/votg/v1/transactions/{transactionId}/call-data"

After receiving transaction stopped event from the external relay, the node enters "Idle" state once again. After which, the node can receive another transaction and send data to said transaction.

Supported flow types: Stream

Was this article helpful?

What's Next

TTS Http