Apache Kafka is an open source, distributed publish-subscribe messaging system. It is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. Moein monitoring system is capable of monitoring Kafka by collecting performance metrics of broker, topics, consumers, producers, and Java Virtual Machine (JVM). Performance indicators of this queuing system are:

Controller:

1. Active Controller Count
2. Offline Partitions Count
3. Leader Election Rate
4. Leader Election Latency
5. Unclean Leader Elections Rate

Serves:

1. Under Replicated Partitions
2. IsrShrinks Rate
3. IsrExpands Per Sec
4. Topic Input Rate
5. Topic Output Rate
6. Topic Message Rate
7. Partition Count
8. Leader Count
9. Request Handler Avg Idle Percent

Network:

1. Produce Total Time
2. Fetch Follower Average Time
3. Fetch Consumer Average Time
4. Network Process Avg Idle Percentage

Topic:

1. Input Rate
2. Output Rate
3. Failed Fetch Request Rate
4. Failed Produce Request Rate
5. Message In Rate

Java Host and Virtual Machine KPI:

Host KPIs:

1. Committed Virtual Memory
2. Total Swap Space
3. Free Swap Space Size
4. Used Swap Space
5. Swap Space Usage
6. Host Free Memory
7. Host Total Memory
8. Host Used Memory
9. Host Memory Used Percentage
10. CPU Time
11. CPU Usage
12. Host CPU Usage
13. Number Of Processors
14. Open File Descriptors
15. Maximum File Descriptors
16. JVM Up Time

Threads :

1. Daemon Thread Count
2. Peak Thread Count
3. Active Thread Count
4. Total Started Thread Count
5. Current Thread CPU Time
6. Current Thread User Time

Memory :

1. Committed Heap Memory
2. Initial Heap Memory
3. Maximum Heap Memory
4. Used Heap Memory
5. Heap Memory Used Percentage
6. Committed Non-Heap Memory
7. Initial Non-Heap Memory
8. Maximum Non-Heap Memory
9. Used Non-Heap Memory
10. Non-Heap Memory Used Percentage

Java Classes:

1. Total Classes
2. Current Locaded Classes
3. Unloaded Classes

Garbage Collectors:

1. Garbage Collection Count
2. Garbage Collection Rate
3. Garbage Collection Time
4. Average Garbage Collection Time

Communication Protocols:

• JMX