2. Audio-Based Integration

Audio-Based Integration is utilized when customer's IVR system can perform Voice Activity Detection, and both SR and TTS services being managed by the Knovvu Virtual Agent. Audio can be included as an attachment or delivered in a multipart format, making it suitable for scenarios where the entire dialog traffic is audio-based.

Request Types:

• Audio Base64: Audio encoded in Base64 format, included in the attachments field as content.

• Audio Minio: Text message with a path to an audio file stored in Minio, included in the attachments field as contentUrl.

• Audio URL: A link to audio file, included in the attachments field as contentUrl.

• Audio Multipart: Audio file, included in request body as form data as AudioForSr.

Response Types
The responseType parameter must be included within the channelData{} field of the activity object and can be set as one of the following values for each request types:

• AudioBase64
• AudioMinio
• AudioMultiPart

Parameters to be sent in the request header:

1. Sending Audio Request with Attachment

This feature is intended for situations where audio is attached to the request. The attached audio file is downloaded and and sent to the SR service for transcribed and further process the input.

Supported contentType for the attachment field includes:

• audio/base64
• audio/url
• audio/minio

Endpoint:
URL: {{baseUrl}}/magpie/ext-api/messages/synchronized
HTTP Method: POST
Content Type: application/json

1.1. Audio Base64

{
    "text": "",
    "type": "message",
    "channelId": "ivr-external",
    "conversation": {
        "id": "{{X-Knovvu-Conversation-Id}}"
    },
    "attachments": [
        {
            "contentType": "audio/base64",
            "content": "{{base64Data}}"
        }
    ],
    "channelData": {
        "ResponseType": "AudioBase64"   // "AudioMultiPart", "AudioMinio" 
    }
}

1.2. Audio Minio

{
    "text": "",
    "type": "message",
    "channelId": "ivr-external",
    "conversation": {
        "id": "{{X-Knovvu-Conversation-Id}}"
    },
    "attachments": [
        {
            "contentType": "audio/minio",
            "contentUrl": "{{audioMinioKey}}"
        }
    ],
    "channelData": {
        "ResponseType": "AudioMinio"   // "AudioMultiPart", "AudioBase64", 
    }
}

1.3. Audio URL

{
    "text": "",
    "type": "message",
    "channelId": "ivr-external",
    "conversation": {
        "id": "{{X-Knovvu-Conversation-Id}}"
    },
    "attachments": [
        {
            "contentType": "audio/url",
            "contentUrl": "{{audioUrl}}"
        }
    ],
    "channelData": {
        "ResponseType": "AudioMinio"   // "AudioMultiPart", "AudioBase64", 
    }
}

2. Sending Request with Audio File

This configuration is used when sending a request with an audio file in the body as form data to the Knovvu VA Synchronized API. The body section of the request should contain the following key parameters:

• activity: An activity object representing the message and its associated metadata.

• AudioForSr: An audio file (e.g., audio.wav) uploaded as form-data.

Here is an example activity object:

{
    "text": "",
    "type": "message",
    "channelId": "ivr-external",
    "conversation": {
        "id": "{{X-Knovvu-Conversation-Id}}"
    },
    "attachments": [],
    "channelData": {
        "ResponseType": "AudioMultiPart"  // "AudioBase64", "AudioMinio"
    }
}

Here is an expected response: