Mikolaj Pawlikowski talked about this Bloomberg project at KubeCon 2018 last week.

Goldpinger makes calls between its instances for visibility and alerting. It runs as a DaemonSet on Kubernetes and produces Prometheus metrics that can be scraped, visualised and alerted on. Oh, and it gives you the graph below for your cluster. Check out the video explainer.