Meaning
This alert is triggered when the total number of logs generated by Virtual Agent services (excluding the usher container) exceeds 65,000 in a 5-minute window. It may indicate excessive logging activity, which could impact performance or storage. The alert fires at a warning level.
Full context
An unusually high volume of logs may result from verbose debugging, unexpected errors, or log loops.
Impact
- Potential risk of log storage exhaustion.
- Reduced observability due to log noise.
- Possible indication of misconfigured logging or application issues.
Diagnosis
- Review which service(s) are generating excessive logs.
- Check log content for patterns: errors, retries, or debug-level verbosity.
- Correlate with recent deployments or configuration changes.
Mitigation
- Adjust logging levels if unnecessary verbosity is detected.
- Investigate and fix the root cause if logs indicate a recurring problem.
- Escalate if the log rate threatens stability or storage capacity.