Abstract: Systems and methods for implementing bulk handling in asynchronous processing are described. For example, an exemplary system includes a processor and a memory to execute instructions at the system; a broker to enqueue received messages; a push connection from the broker to a thread, in which the broker is to push an initial message to the thread for processing; a pull connection from the thread to the broker, in which the thread is to request one or more additional messages for bulk processing with the initial message from the broker; a query interface to issue a database query from the thread to a data store, the database query specifying the initial message and the one or more additional messages as a set of messages to obtain a lock for the set of messages; and a message processing engine to handle the set of messages in bulk based on whether the lock for the set of messages is obtained.

Abstract: Techniques are disclosed relating to caching web application data. In some embodiments, a computing system maintains a multi-tenant database and an in-memory cache for the database. In some embodiments, the computing system is configured to store data in an entry in response to a store request and retrieve data in response to a read request. In various embodiments, data cached by one tenant is not visible to other tenants. The cache may be partitioned and each partition may have an assigned capacity. The computing system may be configured to evict cached data based on capacity thresholds, least-recently-used information, time-to-live information, amount of data in an entry, etc. The cache may indicate misses when requested data is not present. Various disclosed techniques may improve web application performance while maintaining privacy between tenants.

Abstract: Techniques are disclosed relating to an in-memory cache. In some embodiments, in response to determining that data for a requested entry is not present in the cache (e.g., because it has been evicted), a computing system is configured to invoke cached program code associated with the entry. In some embodiments, the computing system is configured to provide data generated by the program code in response to requests that indicate the entry. In some embodiments, the computing system is configured to store the generated data in the cache. In various embodiments, this may avoid cache misses and provide configurability in responding to requests to access the cache.

Abstract: Techniques are disclosed relating to an in-memory, software-managed cache configured to store web application data. In some embodiments, operations to cache data specify a visibility parameter for the data, among a plurality of namespaces. In some embodiments, requests to access cached data are checked, based on a request's namespace and the visibility parameter for the cached data, to determine whether they are allowed to proceed. In some embodiments, this may facilitate caching data using shared computing systems and data structures while maintaining configurable privacy for cached data.

Abstract: Techniques for partition balancing. Organization identifiers (orgIDs) are acquired for one or more organizations within a multi-organization on-demand services environment having multiple nodes, each of which have one or more partitions. A mapping of the orgIDs to the partitions within the multi-organization on-demand services environment is determined. Resource consumption for the organizations is determined. Resource consumption for the partitions based on the resource consumption of the organization and the mapping of the organization to the partitions is determined. A score for the partitions based on the resource consumption is generated. The partitions are redistributed among the nodes based on the partition scores.

Abstract: Techniques for partition balancing. Organization identifiers (orgIDs) are acquired for one or more organizations within a multi-organization on-demand services environment having multiple nodes, each of which have one or more partitions. A mapping of the orgIDs to the partitions within the multi-organization on-demand services environment is determined. Resource consumption for the organizations is determined. Resource consumption for the partitions based on the resource consumption of the organization and the mapping of the organization to the partitions is determined. A score for the partitions based on the resource consumption is generated. The partitions are redistributed among the nodes based on the partition scores.

Abstract: Techniques are disclosed relating to an in-memory cache for web application data. In some embodiments, received transactions include multiple operations, including one or more cache operations to access the in-memory cache. In some embodiments, transactions are performed atomically. In some embodiments, data for the one or more cache operations is stored locally in memory by an application server outside of the in-memory cache until the transaction is successfully completed. This may improve performance and facilitate atomicity, in some embodiments.

Abstract: Techniques are disclosed relating to caching web application data. In some embodiments, a computing system maintains a multi-tenant database and an in-memory cache for the database. In some embodiments, the computing system is configured to store data in an entry in response to a store request and retrieve data in response to a read request. In various embodiments, data cached by one tenant is not visible to other tenants. The cache may be partitioned and each partition may have an assigned capacity. The computing system may be configured to evict cached data based on capacity thresholds, least-recently-used information, time-to-live information, amount of data in an entry, etc. The cache may indicate misses when requested data is not present. Various disclosed techniques may improve web application performance while maintaining privacy between tenants.

Abstract: Techniques are disclosed relating to an in-memory cache for web application data. In some embodiments, received transactions include multiple operations, including one or more cache operations to access the in-memory cache. In some embodiments, transactions are performed atomically. In some embodiments, data for the one or more cache operations is stored locally in memory by an application server outside of the in-memory cache until the transaction is successfully completed. This may improve performance and facilitate atomicity, in some embodiments.

Abstract: Techniques are disclosed relating to an in-memory cache. In some embodiments, in response to determining that data for a requested entry is not present in the cache (e.g., because it has been evicted), a computing system is configured to invoke cached program code associated with the entry. In some embodiments, the computing system is configured to provide data generated by the program code in response to requests that indicate the entry. In some embodiments, the computing system is configured to store the generated data in the cache. In various embodiments, this may avoid cache misses and provide configurability in responding to requests to access the cache.

Abstract: Techniques are disclosed relating to an in-memory, software-managed cache configured to store web application data. In some embodiments, operations to cache data specify a visibility parameter for the data, among a plurality of namespaces. In some embodiments, requests to access cached data are checked, based on a request's namespace and the visibility parameter for the cached data, to determine whether they are allowed to proceed. In some embodiments, this may facilitate caching data using shared computing systems and data structures while maintaining configurable privacy for cached data.

Abstract: Techniques for partition balancing. Organization identifiers (orgIDs) are acquired for one or more organizations within a multi-organization on-demand services environment having multiple nodes, each of which have one or more partitions. A mapping of the orgIDs to the partitions within the multi-organization on-demand services environment is determined. Resource consumption for the organizations is determined. Resource consumption for the partitions based on the resource consumption of the organization and the mapping of the organization to the partitions is determined. A score for the partitions based on the resource consumption is generated. The partitions are redistributed among the nodes based on the partition scores.

Abstract: In accordance with embodiments, there are provided mechanisms and methods for facilitating a fair allocation and usage of thread resources for user messages according to one embodiment in an on-demand services environment. In one embodiment and by way of example, a method includes tracking resource usage associated with a plurality of messages performed and completed in response to requests placed by a user via a user interface at a computing device, where the tracking includes recording an amount of resources consumed by each completed message. The method may further include classifying the plurality of messages into job types based on the tracked resource usage associated with the plurality of messages, and enforcing, based on the classification of the messages, a message selection process to ensure equitable distribution of resources to subsequently received messages.

Abstract: In accordance with embodiments disclosed herein, there are provided methods, systems, and apparatuses including, for supporting transactional message handling in an on-demand service environment including, for example: enqueuing a message specifying a transaction to be processed via a host organization; inserting a row into a database of the host organization associating the message with a status of pending, wherein the row is autocommitted to the database upon insertion; updating the status for the row to ready if a commit operation for the transaction is initiated; requesting a lock on the row; and performing final processing for the transaction based on the status for the message and based further on whether the lock is obtained for the row. Final processing may include, for example, a transaction roll back, a transaction commit, a transaction requeue, a termination of transaction processing, or an orphaned transaction clean up.

Abstract: In accordance with embodiments, there are provided mechanisms and methods for facilitating dynamic workload scheduling and routing of message queues for fair management of the resources for application servers in an on-demand services environment. In one embodiment and by way of example, a method includes detecting an organization of a plurality of organization that is starving for resources. The organization may be seeking performance of a job request at a computing system within a multi-tenant database system. The method may further include consulting, based on a routing policy, a routing table for a plurality of queues available for processing the job request, selecting a queue of the plurality of queues for the organization based on a fair usage analysis obtained from the routing policy, and routing the job request to the selected queue.

Abstract: In accordance with embodiments, there are provided mechanisms and methods for facilitating dynamic workload-aware shuffling of message types in message queues in an on-demand services environment in a multi-tenant environment according to one embodiment. In one embodiment and by way of example, a method includes monitoring, in runtime, workload patterns associated with incoming workloads, where each workload is associated with a plurality of message types and the plurality of message types are inspected and processed in a predetermined order. The method may further include collecting, in runtime, information regarding each message type based on a workload pattern of its associated workload, and dynamically shuffling the order based on the information, where shuffling includes ordering a first set of message types having one or more messages ahead of a second set of message types having fewer or no messages.

Abstract: Systems and techniques for utilizing resource aware queues and/or service sharing in a multi-server environment. Requests directed to an application server are received into one or more queues in front of the application server. An acknowledgement of the requests is provided in response to the requests being received by the one or more queues. Metadata associated with the requests is utilized to manage the one or more queues. The requests from the one or more queues are processed based on the metadata.

Abstract: In accordance with embodiments, there are provided mechanisms and methods for facilitating sliding window resource tracking in message queues for fair management of resources for application servers in an on-demand services environment. In one embodiment and by way of example, a method includes monitoring, in real-time, in-flight jobs in message queues for incoming jobs from organizations in a distributed environment having application servers in communication over a network, applying local sliding windows to the message queues to estimate wait time associated with each incoming job in a message queue. A local sliding window may include segment of time being monitored in each message queue for estimating the wait time. The method may further include allocating, in real-time, based on the estimated wait time, thread resources to one or more of the incoming jobs associated with the one or more of the organizations.

Abstract: In accordance with embodiments, there are provided mechanisms and methods for routing messages to a server. These mechanisms and methods for routing messages to a server can enable dynamic data migration within a system, increased efficiency, improved user experience, avoidance of data loss, etc.

Abstract: Systems and techniques for utilizing resource aware queues and/or service sharing in a multi-server environment. Requests directed to an application server are received into one or more queues in front of the application server. An acknowledgement of the requests is provided in response to the requests being received by the one or more queues. Metadata associated with the requests is utilized to manage the one or more queues. The requests from the one or more queues are processed based on the metadata.