Abstract: A method to create a photo diary includes creating an entry for a time period in the photo diary. Creating the entry includes selecting candidate frames from digital videos created or received in the time period. Selecting the candidate frames includes performing people and pet detection on the digital videos, extract frames with people and pets from the digital videos perform people recognition on the frames with people to identify frames with recognized persons. The method further includes sorting the candidate frames with recognized persons and pets into groups based on criteria, selecting representative frames from the groups, and presenting the representative frames in the entry.