Abstract: An online system dynamically determines time periods during which interaction data points are collected for application states being tested as part of an application state experiment. The online system collects interaction data points that occurred after a time when instructions were transmitted to apply the first application state and labels those with a state label corresponding to the first application state. When the online system detects that an interaction data minimum has been met, the online system transmits instructions to the client devices to present a user interface in accordance with a second application state. The online system applies a transition period between when the second set of instructions are transmitted and when the online system starts labeling interaction data points with a state label for the second application state. After the transition period, the online system labels interaction data points with state labels for the second application state.

Abstract: The online concierge system generates task units based on orders and assigns batches of task units to pickers. The online concierge system generates task units based on received orders. The online concierge system generates permutations of these task units to generate candidate sets of task batches. The online concierge system scores each of these candidate sets, and selects a set of task batches to assign to pickers based on the scores. Additionally, to determine which task UI to display to the picker, the picker client device uses a UI state machine. The UI state machine is a state machine where each state corresponds to a task UI to display on the picker client device. The state transitions between the UI states of the UI state machine indicate which UI state to transition to from a current UI state based on the next task unit in the received task batch.