Abstract: An online system generates a custom audience based on user engagement with distributed content items. The online system monitors and stores user engagement with content items. A content provider submits to the online system a request to generate a new custom audience and selects audience parameters and user engagement types, such as users who watched a specified amount or percentage of a video, users who interacted with an online system page associated with the content provider, or users who clicked through online system content to a page associated with the content provider. Responsive to the request, the online system retrieves the corresponding user engagement data and applies the audience parameters to generate the custom audience and present it to the content provider.