ContainerOOMKilled

Meaning

This alert is triggered when a container is terminated due to an out-of-memory (OOM) condition and has restarted within the last minute. It highlights memory pressure or misconfigured resource limits. The alert continues firing for 5 minutes to ensure visibility.

Impact

• Application instability due to container restarts.
• Potential service disruption or unavailability.
• Memory thrashing and performance degradation on the node.

Diagnosis

• Identify the affected namespace, pod, and container from the alert.
• Check the pod’s memory limits and actual memory usage.
• Inspect container logs before the crash for memory-intensive operations or leaks.
• Use metrics or dashboards to assess memory trends over time.

Mitigation

• Increase memory limits if the usage is legitimate and sustainable.
• Optimize application memory usage or fix memory leaks.
• Consider horizontal scaling if workload spikes are expected.
• Monitor node memory pressure to avoid cascading OOMs across other pods.