Abstract: A technique for short-circuiting normal read-copy update (RCU) grace period computations in the presence of expedited RCU grace periods. The technique may include determining during normal RCU grace period processing whether at least one expedited RCU grace period elapsed during a normal RCU grace period. If so, the normal RCU grace period is ended. If not, the normal RCU grace period processing is continued. Expedited RCU grace periods may be implemented by expedited RCU grace period processing that periodically awakens a kernel thread that implements the normal RCU grace period processing. The expedited RCU grace period processing may conditionally throttle wakeups to the kernel thread based on CPU utilization.

Abstract: A technique for shielding real-time workloads from operating system (OS) jitter due to expedited read-copy update (RCU) grace periods. In accordance with the disclosed technique, a kernel parameter is set to indicate that expedited RCU grace periods are to be suppressed. The kernel parameter is checked to see if it is set. A normal non-expedited RCU grace period is invoked in response to attempted invocation of an expedited RCU grace period if the check reveals that the kernel parameter is set.

Abstract: A technique for shielding real-time workloads from operating system (OS) jitter due to expedited read-copy update (RCU) grace periods. In accordance with the disclosed technique, a kernel parameter is set to indicate that expedited RCU grace periods are to be suppressed. The kernel parameter is checked to see if it is set. A normal non-expedited RCU grace period is invoked in response to attempted invocation of an expedited RCU grace period if the check reveals that the kernel parameter is set.

Abstract: In order to scale Read-Copy Update (RCU) callback offloading from no-callbacks (No-CBs) CPUs, a set of RCU callback offload kernel threads (rcuo kthreads) may be spawned and each may be assigned to one of the No-CBs CPUs to invoke RCU callbacks generated by workloads running on the No-CBs CPUs at CPUs that are not No-CBs CPUs. Groups of the rcuo kthreads may be established, with each rcuo kthread group having one leader kthread and one or more follower rcuo kthreads. The leader rcuo kthreads may be periodically awakened without waking up the follower kthreads when an RCU grace period ends and an RCU callback needs to be invoked, or when a new RCU callback arrives and a new RCU grace period needs to be started. The leader rcuo kthreads may periodically awaken their associated follower rcuo kthreads for which the leader rcuo kthreads have sole responsibility to wake.

Abstract: A technique for low overhead contention-based switching between ticket locking and queued locking to access shared data may include establishing a ticket lock, establishing a queue lock, operating in ticket lock mode using the ticket lock to access the shared data during periods of relatively low data contention, and operating in queue lock mode using the queue lock to access the shared data during periods of relatively high data contention.

Abstract: A technique for low overhead contention-based switching between ticket locking and queued locking to access shared data may include establishing a ticket lock, establishing a queue lock, operating in ticket lock mode using the ticket lock to access the shared data during periods of relatively low data contention, and operating in queue lock mode using the queue lock to access the shared data during periods of relatively high data contention.

Abstract: In some embodiments, a method includes executing an atomic transaction in a system having a transactional memory. The method includes receiving a signal interrupt during executing of the atomic transaction. The method includes storing a state of the signal interrupt to enable subsequent execution of the signal interrupt. The method includes returning to executing the atomic transaction until the atomic transaction is at least one of completed and aborted. The method includes after executing the atomic transaction is at least one of completed and aborted, determining whether the signal interrupt is received during executing of the atomic transaction. The method includes after determining that the signal interrupt is received during executing of the atomic transaction, retrieving the state of the signal interrupt. The method includes executing an interrupt handler for processing the signal interrupt and returning from executing of the atomic transaction.

Abstract: In some embodiments, an apparatus includes a processor that is configured to execute computer usable program code to perform operations. The operations include executing an atomic transaction in a system having a transactional memory. The operations include receiving a signal interrupt during executing of the atomic transaction. The operations include storing a state of the signal interrupt to enable subsequent execution of the signal interrupt. The operations include returning to executing the atomic transaction until the atomic transaction is at least one of completed and aborted. The operations include after executing the atomic transaction is at least one of completed and aborted, determining whether the signal interrupt is received during executing of the atomic transaction. The operations include after determining that the signal interrupt is received during executing of the atomic transaction, retrieving the state of the signal interrupt.

Abstract: A grace period detection technique for a preemptible read-copy update (RCU) implementation that uses a combining tree for quiescent state tracking. When a leaf level bitmask indicating online/offline CPUs is fully cleared due to all of its assigned CPUs going offline as a result of hotplugging operations, the bitmask state is not immediately propagated to the root level of the combining tree as in prior art RCU implementations. Instead, propagation is deferred until all tasks are removed from an associated leaf level task list tracking tasks that were preempted inside an RCU read-side critical section. Deferring bitmask propagation obviates the need to migrate the task list to the combining tree root level in order to prevent premature grace period termination. The task list can remain at the leaf level. In this way, CPU hotplugging is accommodated while avoiding excessive degradation of real-time latency stemming from the now-eliminated task list migration.

Abstract: A grace period detection technique for a preemptible read-copy update (RCU) implementation that uses a combining tree for quiescent state tracking. When a leaf level bitmask indicating online/offline CPUs is fully cleared due to all of its assigned CPUs going offline as a result of hotplugging operations, the bitmask state is not immediately propagated to the root level of the combining tree as in prior art RCU implementations. Instead, propagation is deferred until all tasks are removed from an associated leaf level task list tracking tasks that were preempted inside an RCU read-side critical section. Deferring bitmask propagation obviates the need to migrate the task list to the combining tree root level in order to prevent premature grace period termination. The task list can remain at the leaf level. In this way, CPU hotplugging is accommodated while avoiding excessive degradation of real-time latency stemming from the now-eliminated task list migration.

Abstract: Read-copy update (RCU) grace period initialization and CPU hotplugging are synchronized without a sleeplock. Periodic RCU grace period initialization includes, for each node of a combining tree, copying a first bitmask indicating online/offline status of a set of CPUs to a second bitmask indicating RCU quiescent state status of the CPUs. Periodic CPU hotplug operations include indicating CPU online/offline status to leaf nodes associated with the CPUs. This status is indicated without manipulating any bits in the first bitmask of the leaf nodes. Prior to each RCU grace period initialization, RCU grace period pre-initialization is performed at each leaf node. The RCU grace period pre-initialization includes (1) updating the first bitmask to account for the CPU hotplug operations, and (2) if this results in the first bitmask transitioning between fully cleared and not-fully cleared states, the state change is conditionally propagated to a higher level node.

Abstract: Read-copy update (RCU) grace period initialization and CPU hotplugging are synchronized without a sleeplock. Periodic RCU grace period initialization includes, for each node of a combining tree, copying a first bitmask indicating online/offline status of a set of CPUs to a second bitmask indicating RCU quiescent state status of the CPUs. Periodic CPU hotplug operations include indicating CPU online/offline status to leaf nodes associated with the CPUs. This status is indicated without manipulating any bits in the first bitmask of the leaf nodes. Prior to each RCU grace period initialization, RCU grace period pre-initialization is performed at each leaf node. The RCU grace period pre-initialization includes (1) updating the first bitmask to account for the CPU hotplug operations, and (2) if this results in the first bitmask transitioning between fully cleared and not-fully cleared states, the state change is conditionally propagated to a higher level node.

Abstract: A technique for suspending transactional memory transactions without stack corruption. A first function that begins a transactional memory transaction is allocated a stack frame on a default program stack, then returns. Prior to suspending the transaction, or after suspending the transaction but prior to allocating any suspended mode stack frames, either of the following operations is performed: (1) switch from the default program stack to an alternative program stack, or (2) switch from a default region of the default program stack where the first function's stack frame was allocated to an alternative region of the default program stack. Prior to resuming the transaction, or after resuming the transaction but prior to allocating any transaction mode stack frames, either of the following operations is performed: (1) switch from the alternative program stack to the default program stack, or (2) switch from the alternative stack region to the default stack region.

Abstract: A TASKS_RCU grace period is detected whose quiescent states comprise a task undergoing a voluntary context switch, a task running in user mode, and a task running in idle-mode. A list of all runnable tasks is built. The runnable task list is scanned in one or more scan passes. Each scan pass through the runnable task list searches to identify tasks that have passed through a quiescent state by either performing a voluntary context switch, running in user mode, or running in idle-mode. If found, such quiescent state tasks are removed from the runnable task list. Searching performed during a scan pass includes identifying quiescent state tickless user mode tasks that have been running continuously in user mode on tickless CPUs that have not received a scheduling clock interrupt since commencement of the TASKS_RCU grace period. If the runnable task list is empty, the TASKS_RCU grace period is ended.

Abstract: A technique for suspending transactional memory transactions without stack corruption. A first function that begins a transactional memory transaction is allocated a stack frame on a default program stack, then returns. Prior to suspending the transaction, or after suspending the transaction but prior to allocating any suspended mode stack frames, either of the following operations is performed: (1) switch from the default program stack to an alternative program stack, or (2) switch from a default region of the default program stack where the first function's stack frame was allocated to an alternative region of the default program stack. Prior to resuming the transaction, or after resuming the transaction but prior to allocating any transaction mode stack frames, either of the following operations is performed: (1) switch from the alternative program stack to the default program stack, or (2) switch from the alternative stack region to the default stack region.

Abstract: A TASKS_RCU grace period is detected whose quiescent states comprise a task undergoing a voluntary context switch, a task running in user mode, and a task running in idle-mode. A list of all runnable tasks is built. The runnable task list is scanned in one or more scan passes. Each scan pass through the runnable task list searches to identify tasks that have passed through a quiescent state by either performing a voluntary context switch, running in user mode, or running in idle-mode. If found, such quiescent state tasks are removed from the runnable task list. Searching performed during a scan pass includes identifying quiescent state tickless user mode tasks that have been running continuously in user mode on tickless CPUs that have not received a scheduling clock interrupt since commencement of the TASKS_RCU grace period. If the runnable task list is empty, the TASKS_RCU grace period is ended.

Abstract: In order to scale Read-Copy Update (RCU) callback offloading from no-callbacks (No-CBs) CPUs, a set of RCU callback offload kernel threads (rcuo kthreads) may be spawned and each may be assigned to one of the No-CBs CPUs to invoke RCU callbacks generated by workloads running on the No-CBs CPUs at CPUs that are not No-CBs CPUs. Groups of the rcuo kthreads may be established, with each rcuo kthread group having one leader kthread and one or more follower rcuo kthreads. The leader rcuo kthreads may be periodically awakened without waking up the follower kthreads when an RCU grace period ends and an RCU callback needs to be invoked, or when a new RCU callback arrives and a new RCU grace period needs to be started. The leader rcuo kthreads may periodically awaken their associated follower rcuo kthreads for which the leader rcuo kthreads have sole responsibility to wake.

Abstract: A technique for applying hardware transaction memory to an arbitrarily large data structure is disclosed. A data updater traverses the data structure to locate an update point using a lockless synchronization technique that synchronizes the data updater with other updaters that may be concurrently updating the data structure. At the update point, the updater performs an update on the data structure using a hardware transactional memory transaction that operates at the update point.

Abstract: A technique for detecting full-system idle state in an adaptive-tick kernel includes detecting non-timekeeping CPU idle state, initiating a hysteresis period, waiting for the hysteresis period to end, manipulating a data structure whose state indicates whether a scheduling clock tick may be disabled on all CPUs, and disabling the scheduling clock tick if the data structure is in an appropriate state. In a first embodiment, non-timekeeping CPUs manipulate a global counter when entering an idle state, but add hysteresis to avoid thrashing the counter. Timekeeping is turned off based on the count maintained on the global counter. In a second embodiment, a Read-Copy Update (RCU) dynticks-idle subsystem running on a timekeeping CPU manipulates a global state variable whose states indicate whether all non-timekeeping CPUs are in an idle state, and if so, for how long. Timekeeping is turned off based on the state of the global state variable.

Abstract: A TASKS_RCU grace period is detected whose quiescent states comprise a task undergoing a voluntary context switch, a task running in user mode, and a task running in idle-mode. A list of all runnable tasks is built. The runnable task list is scanned in one or more scan passes. Each scan pass through the runnable task list searches to identify tasks that have passed through a quiescent state by either performing a voluntary context switch, running in user mode, or running in idle-mode. If found, such quiescent state tasks are removed from the runnable task list. Searching performed during a scan pass includes identifying quiescent state tickless user mode tasks that have been running continuously in user mode on tickless CPUs that have not received a scheduling clock interrupt since commencement of the TASKS_RCU grace period. If the runnable task list is empty, the TASKS_RCU grace period is ended.