Knovvu TTS Load Test Procedure
Method:
- A console application which is developed at Sestek is used.
- The results to be calculated are printed back to the console along with the parameter sets, and the features of the relevant machine, etc., is added to the report.
Assumptions:
- These tests target only the core text-to-speech engine under load, without any additional API layers, network and DB operations.
Prerequisites:
- Test data is prepared.
- Prepare a list of sentences for test language.
- A server is prepared.
- Voices and resources are copied to the server.
- TTS Docker image is copied to the server.
- Run TTS Docker image.
- External TTS Docker image is copied to the server. (for external TTS)
- Run external TTS Docker image. (for external TTS)
- Third party dependency installations are done, if necessary.
- It is ensured that no other major process is eating resources of the server. A dedicated bare metal server would be the best.
Tests:
- Test parameters are determined.
- CPU/GPU use
- Number of parallel operations
- Number of sentences to test
- Select at least one voice for each model type (Neural Network architecture).
- All library initializations are done and at least one synthesis operation is done as a warm-up, thus all required resources are loaded into memory.
- Generated audio durations and process durations are collected.
Result Format:
- RTF (real time factor)
- How many seconds a synthesis takes to process 1 second of audio.
- Gives insight about latency.
- Specs of the server (CPU, GPU, RAM)
- Model types
- Test sentences stats
- Generated audio stats
Notes:
None