Amazon Managed Streaming for Apache Kafka (Amazon MSK) is a fully managed service that enables you to build and run applications that use Apache Kafka to process streaming data.

Amazon MSK provides the control-plane operations, such as those for creating, updating, and deleting clusters.

  • You can use Apache Kafka data-plane operations, such as those for producing and consuming data.
  • It runs open-source versions of Apache Kafka.

This means existing applications, tooling, and plugins from partners and the Apache Kafka community are supported without requiring changes to application code.

Use the AWS public cloud integration to discover and collect metrics against the AWS service.

External reference

Monitoring an Amazon MSK Cluster

Setup

To set up the AWS integration and discover the AWS service, go to AWS Integration Discovery Profile and select MSK Cluster.

Event support

CloudTrail event support

  • Not Supported

CloudWatch alarm support

  • Supported
  • Configurable in OpsRamp AWS Integration Discovery Profile.

Supported metrics

OpsRamp MetricMetric Display NameUnitAggregation Type
aws_kafka_CpuIdle

Percentage of CPU idle time.
CpuIdlePercentAverage
aws_kafka_CpuSystem

Percentage of CPU in kernel space.
CpuSystemPercentAverage
aws_kafka_CpuUser

Percentage of CPU in user space.
CpuUserPercentAverage
aws_kafka_KafkaAppLogsDiskUsed

Percentage of disk space used for application logs.
KafkaAppLogsDiskUsedPercentAverage
aws_kafka_KafkaDataLogsDiskUsed

Percentage of disk space used for data logs.
KafkaDataLogsDiskUsedPercentAverage
aws_kafka_MemoryBuffered

Size in bytes of buffered memory for the broker.
MemoryBufferedBytesAverage
aws_kafka_MemoryCached

Size in bytes of cached memory for the broker.
MemoryCachedBytesAverage
aws_kafka_MemoryFree

Size in bytes of memory that is free and available for the broker.
MemoryFreeBytesAverage
aws_kafka_MemoryUsed

Size in bytes of memory that is in use for the broker.
MemoryUsedBytesAverage
aws_kafka_NetworkRxDropped

Number of dropped receive packages.
NetworkRxDroppedCountAverage
aws_kafka_NetworkRxErrors

Number of network receive errors for the broker.
NetworkRxErrorsCountAverage
aws_kafka_NetworkRxPackets

Number of packets received by the broker.
NetworkRxPacketsCountAverage
aws_kafka_NetworkTxDropped

Number of dropped transmit packages.
NetworkTxDroppedCountAverage
aws_kafka_NetworkTxErrors

Number of network transmit errors for the broker.
NetworkTxErrorsCountAverage
aws_kafka_NetworkTxPackets

Number of packets transmitted by the broker.
NetworkTxPacketsCountAverage
aws_kafka_RootDiskUsed

Percentage of the root disk used by the broker.
RootDiskUsedPercentAverage
aws_kafka_SwapFree

Size in bytes of swap memory that is available for the broker.
SwapFreeBytesAverage
aws_kafka_SwapUsed

Size in bytes of swap memory that is in use for the broker.
SwapUsedBytesAverage
aws_kafka_ZooKeeperRequestLatencyMsMean

Mean latency in milliseconds for ZooKeeper requests from broker.
ZooKeeperRequestLatencyMsMeanMillisecondsAverage
aws_kafka_ZooKeeperSessionState

Connection status of brokers ZooKeeper session.
ZooKeeperSessionStateCountAverage