Abstract: Described herein are systems and methods for automatically building automations from desktop recordings using program synthesis. The problem of building automations for desktop applications can be lifted to a generalized concept of automations that operate on worlds whose “world state” can change asynchronously to the actions of the automation. Advantageously, in contrast with synthesis systems that take input-output demonstrations to synthesize a function that maps between them, the method presented here can synthesize from time-series traces of actions to automations that generalizes each step.