Abstract: A method is provided for space-efficient bookkeeping using bit-level locking. Each row of a bookkeeping table stores binary data comprising a plurality of bits. Each bit of the plurality of bits corresponds to a particular message and a particular consumer of the message, and indicates whether the particular message has been dequeued by the particular consumer. The bookkeeping table is stored on-disk and data corresponding to the bookkeeping table is stored in volatile memory. When a message is dequeued the in-memory bookkeeping data is updated, and the bookkeeping table is updated based on the in-memory bookkeeping data when the dequeue is committed.

Abstract: A DBMS maintains delayed and non-delayed messages within a non-delay shard and a delay shard that function as a pair. The DBMS stages non-delayed messages in enqueue-time order within the non-delay shard, and maintains a non-delay dequeue pointer that moves in physical order of the enqueued non-delay messages. The DBMS stages delayed messages in enqueue-time order within bucket shards of the delay shard. Each bucket shard, of a delay shard, represents a time span of delivery times, and messages are assigned thereto based on message delivery time. The delay dequeue pointer comprises a sub-routine to determine the message with the minimum delivery time within a current bucket shard with a time span that includes the current time. The DBMS delivers the next available message out of the delayed and non-delayed shards by comparing the delivery time of the next available message from each shard and delivering the earliest available message.

Abstract: A method and apparatus for message cache sizing are provided. A plurality of messages from one or more enqueuers are enqueued in a queue comprising at least one shard. Each shard includes one or more subshards. A message cache configured to store a plurality of cached subshards is maintained in memory. The plurality of messages is dequeued from the message cache by one or more dequeuers. An estimated enqueue rate is determined for each shard. An estimated dequeue rate is determined for each dequeuer-shard pair. Estimated restore overhead data is determined over a range of candidate message cache sizes based on the estimated enqueue rate for each shard and the estimated dequeue rate for each dequeuer-shard pair. A preferred message cache size is determined based on the estimated restore overhead data.

Abstract: A method is provided for space-efficient bookkeeping using bit-level locking. Each row of a bookkeeping table stores binary data comprising a plurality of bits. Each bit of the plurality of bits corresponds to a particular message and a particular consumer of the message, and indicates whether the particular message has been dequeued by the particular consumer. The bookkeeping table is stored on-disk and data corresponding to the bookkeeping table is stored in volatile memory. When a message is dequeued the in-memory bookkeeping data is updated, and the bookkeeping table is updated based on the in-memory bookkeeping data when the dequeue is committed.

Abstract: A DBMS maintains delayed and non-delayed messages within a non-delay shard and a delay shard that function as a pair. The DBMS stages non-delayed messages in enqueue-time order within the non-delay shard, and maintains a non-delay dequeue pointer that moves in physical order of the enqueued non-delay messages. The DBMS stages delayed messages in enqueue-time order within bucket shards of the delay shard. Each bucket shard, of a delay shard, represents a time span of delivery times, and messages are assigned thereto based on message delivery time. The delay dequeue pointer comprises a sub-routine to determine the message with the minimum delivery time within a current bucket shard with a time span that includes the current time. The DBMS delivers the next available message out of the delayed and non-delayed shards by comparing the delivery time of the next available message from each shard and delivering the earliest available message.

Abstract: A method and apparatus for message cache management for message queues is provided. A plurality of messages from a plurality of enqueuers are enqueued in a queue comprising one or more shards, each shard comprising one or more subshards. A message cache is maintained in memory. Enqueuing a message includes enqueuing the message in a current subshard of a particular shard, which includes storing the message in a cached subshard corresponding to the current subshard of the particular shard. For each dequeuer-shard pair, a dequeue rate is determined. Estimated access time data is generated that includes an earliest estimated access time for each of a plurality of subshards based on the dequeuer-shard pair dequeue rates. A set of subshards is determined for storing as cached subshards in the message cache based on the earliest estimated access times for the plurality of subshards.

Abstract: A method and apparatus for message cache sizing are provided. A plurality of messages from one or more enqueuers are enqueued in a queue comprising at least one shard. Each shard includes one or more subshards. A message cache configured to store a plurality of cached subshards is maintained in memory. The plurality of messages is dequeued from the message cache by one or more dequeuers. An estimated enqueue rate is determined for each shard. An estimated dequeue rate is determined for each dequeuer-shard pair. Estimated restore overhead data is determined over a range of candidate message cache sizes based on the estimated enqueue rate for each shard and the estimated dequeue rate for each dequeuer-shard pair. A preferred message cache size is determined based on the estimated restore overhead data.

Abstract: A method and apparatus for message cache management for message queues is provided. A plurality of messages from a plurality of enqueuers are enqueued in a queue comprising one or more shards, each shard comprising one or more subshards. A message cache is maintained in memory. Enqueuing a message includes enqueuing the message in a current subshard of a particular shard, which includes storing the message in a cached subshard corresponding to the current subshard of the particular shard. For each dequeuer-shard pair, a dequeue rate is determined. Estimated access time data is generated that includes an earliest estimated access time for each of a plurality of subshards based on the dequeuer-shard pair dequeue rates. A set of subshards is determined for storing as cached subshards in the message cache based on the earliest estimated access times for the plurality of subshards.

Abstract: An in-database sharded queue for a shared-disk database is provided. First messages from at least one first enqueuer are stored on a first queue shard. The first enqueuer/s execute in a first instance of a shared-disk database system. The first shard includes a first plurality of subshards. Second messages from at least one second enqueuer are stored on a second queue shard. The second enqueuer/s execute in a second instance of the shared-disk database system. The second shard includes a second plurality of subshards. In volatile memory of the respective instances, a first message cache and a second message cache is maintained for the first shard and the second shard. The respective message caches include at least one buffer associated with the plurality of subshards.

Abstract: Handling memory pressure in an in-database sharded queue is described. Messages from a plurality of enqueuers are stored in a plurality of shards of a sharded queue. Messages from a first enqueuer are stored in a first shard. A queue table corresponding to the sharded queue is maintained. In volatile memory, a plurality of message caches is maintained, each message cache corresponding to a shard of the plurality of shards. Memory pressure is detected based on memory usage of the volatile memory. To store a specific message from the enqueuer, the specific message is stored in rows of the queue table that are assigned to the first shard. When memory pressure is not detected, the specific message is stored in a first message cache corresponding to the first shard. Subscribers of the sharded queue are caused to dequeue messages from the plurality of shards.

Abstract: An in-database sharded queue for a shared-disk database is provided. First messages from at least one first enqueuer are stored on a first queue shard. The first enqueuer/s execute in a first instance of a shared-disk database system. The first shard includes a first plurality of subshards. Second messages from at least one second enqueuer are stored on a second queue shard. The second enqueuer/s execute in a second instance of the shared-disk database system. The second shard includes a second plurality of subshards. In volatile memory of the respective instances, a first message cache and a second message cache is maintained for the first shard and the second shard. The respective message caches include at least one buffer associated with the plurality of subshards.

Abstract: Handling memory pressure in an in-database sharded queue is described. Messages from a plurality of enqueuers are stored in a plurality of shards of a sharded queue. Messages from a first enqueuer are stored in a first shard. A queue table corresponding to the sharded queue is maintained. In volatile memory, a plurality of message caches is maintained, each message cache corresponding to a shard of the plurality of shards. Memory pressure is detected based on memory usage of the volatile memory. To store a specific message from the enqueuer, the specific message is stored in rows of the queue table that are assigned to the first shard. When memory pressure is not detected, the specific message is stored in a first message cache corresponding to the first shard. Subscribers of the sharded queue are caused to dequeue messages from the plurality of shards.