Abstract: A method and system performing a method for managing dynamic messaging using a Kafka-based monitoring application deployed within a cloud environment having at least one software application in operation is provided. The Kafka-based monitoring application monitors consumer lag for a specific topic associated with the software application, schedules queries to detect consumer lag for the specific topic at predetermined intervals. detects, via an application programming interface in communication with the Kafka-based monitoring application, any consumer lag and retrieves a list of any affected consumers and associated consumer information; and determines one or more pools of the affected consumers, and performs a scaling operation to scale up or down consumers on the one or more pools based on the consumer lag detected.

Abstract: A method and system performing a method for managing dynamic messaging using a Kafka-based monitoring application deployed within a cloud environment having at least one software application in operation is provided. The Kafka-based monitoring application monitors consumer lag for a specific topic associated with the software application, schedules queries to detect consumer lag for the specific topic at predetermined intervals. detects, via an application programming interface in communication with the Kafka-based monitoring application, any consumer lag and retrieves a list of any affected consumers and associated consumer information; and determines one or more pools of the affected consumers, and performs a scaling operation to scale up or down consumers on the one or more pools based on the consumer lag detected.