Apache Kafka on Kubernetes with Strimzi — Part 3: Monitoring our Strimzi Kafka Cluster with Prometheus and Grafana

Prometheus

Grafana

Our Requirements

Adding Metrics to Kafka

$ kubectl apply -f kafka-deployment.yaml -n kafka

Deploying Prometheus

Introducing Prometheus Operator

  • Prometheus. Defines a desired Prometheus deployment.
  • ServiceMonitor. Defines how a dynamic set of services should be monitored
  • PodMonitor. Defines how a dynamic set of pods should be monitored
  • AlertManager. Defines a desired Alertmanager deployment.
  • PrometheusRule. Defines a desired Prometheus rule to be consumed by one or more Prometheus instances.
Operator workflow and relationships (source)

Deploying Prometheus Operator

$ curl -s https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/master/bundle.yaml | sed -e 's/namespace: .*/namespace: monitoring/' > bundle.yaml
$ curl -s https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/master/bundle.yaml | sed -e '' 's/namespace: .*/namespace: monitoring/' > bundle.yaml
kubectl apply -f bundle.yaml -n monitoring
$ kubectl get pods -n monitoring
NAME READY STATUS RESTARTS
prometheus-operator-7568575f58-2phj6 1/1 Running 0

$ kubectl get svc -n monitoring
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
prometheus-operator ClusterIP None <none> 8080/TCP

Deploying Prometheus

$ kubectl create secret generic additional-scrape-configs --from-file=prometheus-additional.yaml -n monitoring
  namespaceSelector:
matchNames:
- kafka
$ kubectl apply -f strimzi-pod-monitor.yaml -n monitoring
$ kubectl apply -f prometheus-rules.yaml -n monitoring
$ kubectl apply -f prometheus.yaml -n monitoring
$ kubectl get pods -n monitoring
NAME READY STATUS RESTARTS
grafana-7b7bfd6ffc-lmg6z 1/1 Running 0
prometheus-operator-7568575f58-2phj6 1/1 Running 1
prometheus-prometheus-0 3/3 Running 1

Deploying Grafana

$ kubectl apply -f grafana.yaml -n monitoring
$ kubectl port-forward svc/grafana 3000:3000 -n monitoring
$ kubectl get svc -n monitoring
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
grafana ClusterIP 10.103.151.53 <none> 3000/TCP
prometheus-operated ClusterIP None <none> 9090/TCP
prometheus-operator ClusterIP None <none> 8080/TCP
strimzi-kafka.json
strimzi-kafka-exporter.json
strimzi-operators.json
strimzi-zookeeper.json
Grafana Dashboard for our Kafka Cluster
Grafana Dashboard for our Kafka Cluster (Kafka Exporter)

Conclusion

--

--

--

CTO at Daneshgar Technology Co. Ltd | Interested in Distributed Systems, Cloud Computing, Serverless computing, Data Stream Processing and Microservices

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Bankruptcy Attorney Salt Lake City

CISSP: Exam thoughts and learning materials

Rediscovering My Passion for My Career as a Dev

Complexity by the Visual Shape of Programs

images/Chp4_VisualShapes.png

SunBible Update: Jan 2021

Hack the System: Fluence Workshop 1 [Video + Documentation]

Install Mac Os On Blank Hard Drive From Internet

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sina Nourian

Sina Nourian

CTO at Daneshgar Technology Co. Ltd | Interested in Distributed Systems, Cloud Computing, Serverless computing, Data Stream Processing and Microservices

More from Medium

Secure Microservices Configuration Properties via HashiCorp Vault

Creating Spring Boot container in a minute with Cloud Native Buildpacks for Azure Container…

Container Orchestration Open-Source Platform - Kubernetes

Helm incubator kafka setup with SSL auth