Patents by Inventor Hartwig Adam

Hartwig Adam has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20210297603
    Abstract: Implementations of the present disclosure include actions of receiving image data of an image capturing a scene, receiving data describing one or more entities determined from the scene, the one or more entities being determined from the scene, determining one or more actions based on the one or more entities, each action being provided at least partly based on search results from searching the one or more entities, and providing instructions to display an action interface comprising one or more action elements, each action element being to induce execution of a respective action, the action interface being displayed in a viewfinder
    Type: Application
    Filed: June 7, 2021
    Publication date: September 23, 2021
    Inventors: Teresa Ko, Hartwig Adam, Mikkel Crone Koser, Alexei Masterov, Andrews-Junior Kimbembe, Matthew J. Bridges, Paul Chang, David Petrou, Adam Berenzweig
  • Publication number: 20210248179
    Abstract: The present disclosure is directed to processing imagery using one or more machine learning (ML) models. In particular, data describing imagery comprising a plurality of different and distinct frames can be received; and based at least in part on one or more ML models and the data describing the imagery, and for each frame of the plurality of different and distinct frames, one or more scores can be determined for the frame. Each score of the score(s) can indicate a determined measure of suitability of the frame with respect to one or more of various different and distinct uses for which the ML model(s) are configured to determine suitability of imagery.
    Type: Application
    Filed: September 18, 2018
    Publication date: August 12, 2021
    Inventors: David Karam, Li Zhang, Ariel Gilder, Yuzo Watanabe, Eric Penner, Farooq Ahmad, Hartwig Adam
  • Patent number: 11074504
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for instance segmentation. In one aspect, a system generates: (i) data identifying one or more regions of the image, wherein an object is depicted in each region, (ii) for each region, a predicted type of object that is depicted in the region, and (iii) feature channels comprising a plurality of semantic channels and one or more direction channels. The system generates a region descriptor for each of the one or more regions, and provides the region descriptor for each of the one or more regions to a segmentation neural network that processes a region descriptor for a region to generate a predicted segmentation of the predicted type of object depicted in the region.
    Type: Grant
    Filed: November 14, 2018
    Date of Patent: July 27, 2021
    Assignee: Google LLC
    Inventors: Liang-Chieh Chen, Alexander Hermans, Georgios Papandreou, Gerhard Florian Schroff, Peng Wang, Hartwig Adam
  • Patent number: 11032478
    Abstract: Implementations of the present disclosure include actions of receiving image data of an image capturing a scene, receiving data describing one or more entities determined from the scene, the one or more entities being determined from the scene, determining one or more actions based on the one or more entities, each action being provided at least partly based on search results from searching the one or more entities, and providing instructions to display an action interface comprising one or more action elements, each action element being to induce execution of a respective action, the action interface being displayed in a viewfinder.
    Type: Grant
    Filed: May 22, 2020
    Date of Patent: June 8, 2021
    Assignee: Google LLC
    Inventors: Teresa Ko, Hartwig Adam, Mikkel Crone Koser, Alexei Masterov, Andrews-Junior Kimbembe, Matthew J. Bridges, Paul Chang, David Petrou, Adam Berenzweig
  • Publication number: 20210141827
    Abstract: A system and method of identifying objects is provided. In one aspect, the system and method includes a hand-held device with a display, camera and processor. As the camera captures images and displays them on the display, the processor compares the information retrieved in connection with one image with information retrieved in connection with subsequent images. The processor uses the result of such comparison to determine the object that is likely to be of greatest interest to the user. The display simultaneously displays the images the images as they are captured, the location of the object in an image, and information retrieved for the object.
    Type: Application
    Filed: January 25, 2021
    Publication date: May 13, 2021
    Inventors: David Petrou, Matthew J. Bridges, Shailesh Nalawadi, Hartwig Adam, Matthew R. Casey, Hartmut Neven, Andrew Harp
  • Publication number: 20210081796
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining neural network architectures. One of the methods includes obtaining training data for a dense image prediction task; and determining an architecture for a neural network configured to perform the dense image prediction task, comprising: searching a space of candidate architectures to identify one or more best performing architectures using the training data, wherein each candidate architecture in the space of candidate architectures comprises (i) the same first neural network backbone that is configured to receive an input image and to process the input image to generate a plurality of feature maps and (ii) a different dense prediction cell configured to process the plurality of feature maps and to generate an output for the dense image prediction task; and determining the architecture for the neural network based on the best performing candidate architectures.
    Type: Application
    Filed: November 30, 2020
    Publication date: March 18, 2021
    Inventors: Barret Zoph, Jonathon Shlens, Yukun Zhu, Maxwell Donald Collins, Liang-Chieh Chen, Gerhard Florian Schroff, Hartwig Adam, Georgios Papandreou
  • Patent number: 10922350
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for associating still images and videos. One method includes receiving a plurality of images and a plurality of videos and determining whether the images are related to the videos. The determining includes, for an image and a video, extracting features from the image and extracting features frames of the video, and comparing the features to determine whether the image is related to the video. The method further includes maintaining a data store storing data associating each image with each video determined to be related to the image.
    Type: Grant
    Filed: July 15, 2019
    Date of Patent: February 16, 2021
    Assignee: Google LLC
    Inventors: Ming Zhao, Yang Song, Hartwig Adam, Ullas Gargi, Yushi Jing, Henry Allan Rowley
  • Patent number: 10902055
    Abstract: A system and method of identifying objects is provided. In one aspect, the system and method includes a hand-held device with a display, camera and processor. As the camera captures images and displays them on the display, the processor compares the information retrieved in connection with one image with information retrieved in connection with subsequent images. The processor uses the result of such comparison to determine the object that is likely to be of greatest interest to the user. The display simultaneously displays the images the images as they are captured, the location of the object in an image, and information retrieved for the object.
    Type: Grant
    Filed: January 16, 2020
    Date of Patent: January 26, 2021
    Assignee: Google LLC
    Inventors: David Petrou, Matthew J. Bridges, Shailesh Nalawadi, Hartwig Adam, Matthew R. Casey, Hartmut Neven, Andrew Harp
  • Patent number: 10853726
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining neural network architectures. One of the methods includes obtaining training data for a dense image prediction task; and determining an architecture for a neural network configured to perform the dense image prediction task, comprising: searching a space of candidate architectures to identify one or more best performing architectures using the training data, wherein each candidate architecture in the space of candidate architectures comprises (i) the same first neural network backbone that is configured to receive an input image and to process the input image to generate a plurality of feature maps and (ii) a different dense prediction cell configured to process the plurality of feature maps and to generate an output for the dense image prediction task; and determining the architecture for the neural network based on the best performing candidate architectures.
    Type: Grant
    Filed: May 29, 2019
    Date of Patent: December 1, 2020
    Assignee: Google LLC
    Inventors: Barret Zoph, Jonathon Shlens, Yukun Zhu, Maxwell Donald Emmet Collins, Liang-Chieh Chen, Gerhard Florian Schroff, Hartwig Adam, Georgios Papandreou
  • Publication number: 20200288063
    Abstract: Implementations of the present disclosure include actions of receiving image data of an image capturing a scene, receiving data describing one or more entities determined from the scene, the one or more entities being determined from the scene, determining one or more actions based on the one or more entities, each action being provided at least partly based on search results from searching the one or more entities, and providing instructions to display an action interface comprising one or more action elements, each action element being to induce execution of a respective action, the action interface being displayed in a viewfinder
    Type: Application
    Filed: May 22, 2020
    Publication date: September 10, 2020
    Inventors: Teresa Ko, Hartwig Adam, Mikkel Crone Koser, Alexei Masterov, Andrews-Junior Kimbembe, Matthew J. Bridges, Paul Chang, David Petrou, Adam Berenzweig
  • Publication number: 20200226382
    Abstract: Methods, systems, and apparatus for efficient image analysis. In some aspects, a system includes a camera configured to capture images, one or more environment sensors configured to detect movement of the camera, a data processing apparatus, and a memory storage apparatus in data communication with the data processing apparatus. The data processing apparatus can access, for each of a multitude of images captured by a mobile device camera, data indicative of movement of the camera at a time at which the camera captured the image. The data processing apparatus can also select, from the images, a particular image for analysis based on the data indicative of the movement of the camera for each image, analyze the particular image to recognize one or more objects depicted in the particular image, and present content related to the one or more recognized objects.
    Type: Application
    Filed: March 27, 2020
    Publication date: July 16, 2020
    Inventors: Matthew J. Bridges, Alessandro Fin, Hartwig Adam, Jeffrey M. Gilbert
  • Patent number: 10701272
    Abstract: Implementations of the present disclosure include actions of receiving image data of an image capturing a scene, receiving data describing one or more entities determined from the scene, the one or more entities being determined from the scene, determining one or more actions based on the one or more entities, each action being provided at least partly based on search results from searching the one or more entities, and providing instructions to display an action interface comprising one or more action elements, each action element being to induce execution of a respective action, the action interface being displayed in a viewfinder.
    Type: Grant
    Filed: September 12, 2019
    Date of Patent: June 30, 2020
    Assignee: Google LLC
    Inventors: Teresa Ko, Hartwig Adam, Mikkel Crone Koser, Alexei Masterov, Andrews-Junior Kimbembe, Matthew J. Bridges, Paul Chang, David Petrou, Adam Berenzweig
  • Publication number: 20200175375
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for instance segmentation. In one aspect, a system generates: (i) data identifying one or more regions of the image, wherein an object is depicted in each region, (ii) for each region, a predicted type of object that is depicted in the region, and (iii) feature channels comprising a plurality of semantic channels and one or more direction channels. The system generates a region descriptor for each of the one or more regions, and provides the region descriptor for each of the one or more regions to a segmentation neural network that processes a region descriptor for a region to generate a predicted segmentation of the predicted type of object depicted in the region.
    Type: Application
    Filed: November 14, 2018
    Publication date: June 4, 2020
    Inventors: Liang-Chieh Chen, Alexander Hermans, Georgios Papandreou, Gerhard Florian Schroff, Peng Wang, Hartwig Adam
  • Publication number: 20200151211
    Abstract: A system and method of identifying objects is provided. In one aspect, the system and method includes a hand-held device with a display, camera and processor. As the camera captures images and displays them on the display, the processor compares the information retrieved in connection with one image with information retrieved in connection with subsequent images. The processor uses the result of such comparison to determine the object that is likely to be of greatest interest to the user. The display simultaneously displays the images the images as they are captured, the location of the object in an image, and information retrieved for the object.
    Type: Application
    Filed: January 16, 2020
    Publication date: May 14, 2020
    Inventors: David Petrou, Matthew J. Bridges, Shailesh Nalawadi, Hartwig Adam, Matthew R. Casey, Hartmut Neven, Andrew Harp
  • Patent number: 10621435
    Abstract: Methods, systems, and apparatus for efficient image analysis. In some aspects, a system includes a camera configured to capture images, one or more environment sensors configured to detect movement of the camera, a data processing apparatus, and a memory storage apparatus in data communication with the data processing apparatus. The data processing apparatus can access, for each of a multitude of images captured by a mobile device camera, data indicative of movement of the camera at a time at which the camera captured the image. The data processing apparatus can also select, from the images, a particular image for analysis based on the data indicative of the movement of the camera for each image, analyze the particular image to recognize one or more objects depicted in the particular image, and present content related to the one or more recognized objects.
    Type: Grant
    Filed: February 9, 2018
    Date of Patent: April 14, 2020
    Assignee: Google LLC
    Inventors: Matthew J. Bridges, Alessandro Fin, Hartwig Adam, Jeffrey M. Gilbert
  • Publication number: 20200090039
    Abstract: A computer-implemented method for generating a unified machine learning model using a neural network on a data processing apparatus is described. The method includes the data processing apparatus determining respective learning targets for each of a plurality of object verticals. The data processing apparatus determines the respective learning targets based on two or more embedding outputs of the neural network. The method also includes the data processing apparatus training the neural network to identify data associated with each of the plurality of object verticals. The data processing apparatus trains the neural network using the respective learning targets and based on a first loss function. The data processing apparatus uses the neural network trained to generate a unified machine learning model, where the model is configured to identify particular data items associated with each of the plurality of object verticals.
    Type: Application
    Filed: November 17, 2017
    Publication date: March 19, 2020
    Inventors: Yang Song, Yuan Li, Bo Wu, Chao-Yeh Chen, Xiao Zhang, Hartwig Adam
  • Patent number: 10552476
    Abstract: A system and method of identifying objects is provided. In one aspect, the system and method includes a hand-held device with a display, camera and processor. As the camera captures images and displays them on the display, the processor compares the information retrieved in connection with one image with information retrieved in connection with subsequent images. The processor uses the result of such comparison to determine the object that is likely to be of greatest interest to the user. The display simultaneously displays the images the images as they are captured, the location of the object in an image, and information retrieved for the object.
    Type: Grant
    Filed: September 6, 2019
    Date of Patent: February 4, 2020
    Assignee: Google LLC
    Inventors: David Petrou, Matthew J. Bridges, Shailesh Nalawadi, Hartwig Adam, Matthew R. Casey, Hartmut Neven, Andrew Harp
  • Publication number: 20200007774
    Abstract: Implementations of the present disclosure include actions of receiving image data of an image capturing a scene, receiving data describing one or more entities determined from the scene, the one or more entities being determined from the scene, determining one or more actions based on the one or more entities, each action being provided at least partly based on search results from searching the one or more entities, and providing instructions to display an action interface comprising one or more action elements, each action element being to induce execution of a respective action, the action interface being displayed in a viewfinder
    Type: Application
    Filed: September 12, 2019
    Publication date: January 2, 2020
    Inventors: Teresa Ko, Hartwig Adam, Mikkel Crone Koser, Alexei Masterov, Andrews-Junior Kimbembe, Matthew J. Bridges, Paul Chang, David Petrou, Adam Berenzweig
  • Publication number: 20190391996
    Abstract: A system and method of identifying objects is provided. In one aspect, the system and method includes a hand-held device with a display, camera and processor. As the camera captures images and displays them on the display, the processor compares the information retrieved in connection with one image with information retrieved in connection with subsequent images. The processor uses the result of such comparison to determine the object that is likely to be of greatest interest to the user. The display simultaneously displays the images the images as they are captured, the location of the object in an image, and information retrieved for the object.
    Type: Application
    Filed: September 6, 2019
    Publication date: December 26, 2019
    Inventors: David Petrou, Matthew J. Bridges, Shailesh Nalawadi, Hartwig Adam, Matthew R. Casey, Hartmut Neven, Andrew Harp
  • Patent number: 10515114
    Abstract: A facial recognition search system identifies one or more likely names (or other personal identifiers) corresponding to the facial image(s) in a query as follows. After receiving the visual query with one or more facial images, the system identifies images that potentially match the respective facial image in accordance with visual similarity criteria. Then one or more persons associated with the potential images are identified. For each identified person, person-specific data comprising metrics of social connectivity to the requester are retrieved from a plurality of applications such as communications applications, social networking applications, calendar applications, and collaborative applications. An ordered list of persons is then generated by ranking the identified persons in accordance with at least metrics of visual similarity between the respective facial image and the potential image matches and with the social connection metrics.
    Type: Grant
    Filed: July 9, 2018
    Date of Patent: December 24, 2019
    Assignee: Google LLC
    Inventors: David Petrou, Andrew Rabinovich, Hartwig Adam