Patents by Inventor Brett D. Miller
Brett D. Miller has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11860368Abstract: A device for MR/VR systems that includes a two-dimensional array of cameras that capture images of respective portions of a scene. The cameras are positioned along a spherical surface so that the cameras have adjacent fields of view. The entrance pupils of the cameras are positioned at or near the user's eye while the cameras also form optimized images at the sensor. Methods for reducing the number of cameras in an array, as well as methods for reducing the number of pixels read from the array and processed by the pipeline, are also described.Type: GrantFiled: September 16, 2022Date of Patent: January 2, 2024Assignee: Apple Inc.Inventors: Noah D. Bedard, Ricardo J. Motta, Emanuele Mandelli, Daniel R. Hennigan, Kathrin Berkner Cieslicki, Branko Petljanski, Brett D. Miller, Kenichi Saito
-
Publication number: 20230409122Abstract: In one implementation, a method of performing an action is performed at a device including an image sensor, one or more processors, and non-transitory memory. The method includes receiving, from the image sensor, one or more images of a physical environment. The method includes detecting, in the one or more images of the physical environment, one or more actionable items respectively associated with one or more actions. The method includes detecting, in the one or more images of the physical environment, a hand gesture indicating a particular actionable item. The method includes in response to detecting the hand gesture, performing an action associated with the particular actionable item.Type: ApplicationFiled: June 19, 2023Publication date: December 21, 2023Inventors: Brett D. Miller, Daniel K. Boothe, Martin E. Johnson
-
Publication number: 20230410458Abstract: In one implementation, a method of detecting an occlusion of an image sensor is performed at a device including one or more image sensors, one or more speakers, one or more processors, and non-transitory memory. The method includes receiving, from the one or more image sensors, an image of a physical environment. The method includes detecting, based on the image of the physical environment, an occlusion at least partially occluding the one or more image sensors. The method includes classifying the occlusion as one of a plurality of occlusion types. The method includes playing, via the one or more speakers, an occlusion notification based on the classification.Type: ApplicationFiled: June 19, 2023Publication date: December 21, 2023Inventors: Brett D. Miller, Daniel K. Boothe, Martin E. Johnson
-
Publication number: 20230410348Abstract: In one implementation, a method of playing an audio notification is performed at a device including one or more image sensors, one or more speakers, one or more processors, and non-transitory memory. The method includes receiving, from the one or more image sensors, an image of a physical environment having a device field-of-view different than a user field-of-view. The method includes detecting, in the image of the physical environment, an object at a location in the physical environment. The method includes determining that the location in the physical environment is outside an area of the user field-of-view. The method includes, in response to determining that the location in the physical environment is outside the area of the user field-of-view, playing, via the speaker, an audio notification of the detection.Type: ApplicationFiled: June 19, 2023Publication date: December 21, 2023Inventors: Brett D. Miller, Daniel K. Boothe, Martin E. Johnson
-
Publication number: 20230409278Abstract: In one implementation, a method of playing audio data is performed at a device including a frame configured for insertion into an outer ear, a speaker coupled to the frame, an image sensor coupled to the frame, one or more processors, and non-transitory memory. The method includes capturing, using the image sensor, one or more images of a physical environment. The method includes generating audio data based on the one or more images of the physical environment. The method includes playing, via the speaker, the audio data.Type: ApplicationFiled: June 19, 2023Publication date: December 21, 2023Inventors: Brett D. Miller, Daniel K. Boothe, Martin E. Johnson
-
Publication number: 20230334617Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.Type: ApplicationFiled: June 20, 2023Publication date: October 19, 2023Inventors: Brett D. Miller, Minwoong Kim, Ricardo J. Motta
-
Patent number: 11720996Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.Type: GrantFiled: February 14, 2022Date of Patent: August 8, 2023Assignee: Apple Inc.Inventors: Brett D. Miller, Minwoong Kim, Ricardo J. Motta
-
Publication number: 20230185093Abstract: A device for MR/VR systems that includes a two-dimensional array of cameras that capture images of respective portions of a scene. The cameras are positioned along a spherical surface so that the cameras have adjacent fields of view. The entrance pupils of the cameras are positioned at or near the user’s eye while the cameras also form optimized images at the sensor. Methods for reducing the number of cameras in an array, as well as methods for reducing the number of pixels read from the array and processed by the pipeline, are also described.Type: ApplicationFiled: September 16, 2022Publication date: June 15, 2023Applicant: Apple Inc.Inventors: Noah D. Bedard, Ricardo J. Motta, Emanuele Mandelli, Daniel R. Hennigan, Kathrin Berkner Cieslicki, Branko Petljanski, Brett D. Miller, Kenichi Saito
-
Patent number: 11448886Abstract: A device for MR/VR systems that includes a two-dimensional array of cameras that capture images of respective portions of a scene. The cameras are positioned along a spherical surface so that the cameras have adjacent fields of view. The entrance pupils of the cameras are positioned at or near the user's eye while the cameras also form optimized images at the sensor. Methods for reducing the number of cameras in an array, as well as methods for reducing the number of pixels read from the array and processed by the pipeline, are also described.Type: GrantFiled: September 27, 2019Date of Patent: September 20, 2022Assignee: Apple Inc.Inventors: Noah D. Bedard, Ricardo J. Motta, Emanuele Mandelli, Daniel R. Hennigan, Kathrin Berkner Cieslicki, Branko Petljanski, Brett D. Miller, Kenichi Saito
-
Publication number: 20220172319Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.Type: ApplicationFiled: February 14, 2022Publication date: June 2, 2022Inventors: Brett D. Miller, Minwoong Kim, Ricardo J. Motta
-
Patent number: 11250541Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.Type: GrantFiled: September 7, 2018Date of Patent: February 15, 2022Assignee: Apple Inc.Inventors: Brett D. Miller, Minwoong Kim, Ricardo J. Motta
-
Publication number: 20220035160Abstract: A device for MR/VR systems that includes a two-dimensional array of cameras that capture images of respective portions of a scene. The cameras are positioned along a spherical surface so that the cameras have adjacent fields of view. The entrance pupils of the cameras are positioned at or near the user's eye while the cameras also form optimized images at the sensor. Methods for reducing the number of cameras in an array, as well as methods for reducing the number of pixels read from the array and processed by the pipeline, are also described.Type: ApplicationFiled: September 27, 2019Publication date: February 3, 2022Applicant: Apple Inc.Inventors: Noah D. Bedard, Ricardo J. Motta, Emanuele Mandelli, Daniel R. Hennigan, Kathrin Berkner Cieslicki, Branko Petljanski, Brett D. Miller, Kenichi Saito
-
Patent number: 11217021Abstract: A mixed reality system that includes a head-mounted display (HMD) that provides 3D virtual views of a user's environment augmented with virtual content. The HMD may include sensors that collect information about the user's environment (e.g., video, depth information, lighting information, etc.), and sensors that collect information about the user (e.g., the user's expressions, eye movement, hand gestures, etc.). The sensors provide the information as inputs to a controller that renders frames including virtual content based at least in part on the inputs from the sensors. The HMD may display the frames generated by the controller to provide a 3D virtual view including the virtual content and a view of the user's environment for viewing by the user.Type: GrantFiled: March 21, 2019Date of Patent: January 4, 2022Assignee: Apple Inc.Inventors: Ricardo J. Motta, Brett D. Miller, Tobias Rick, Manohar B. Srikanth
-
Patent number: 11187914Abstract: A scene camera system that includes two or more mirrors that reflect light from a respective portion of a field of view (FOV) in front of the system and two or more cameras that each capture the light reflected by a respective one of the two or more mirrors. By using the mirrors to reflect the light, the cameras' entrance pupils are imaged to a location closer to a user's eyes to thus achieve a more accurate representation of the perspective of the user.Type: GrantFiled: September 25, 2019Date of Patent: November 30, 2021Assignee: Apple Inc.Inventors: Daniel R. Hennigan, Noah D. Bedard, Branko Petljanski, Jason C. Sauers, Edward S. Huo, Ricardo J. Motta, Brett D. Miller, Yury Petrov
-
Patent number: 10659757Abstract: This disclosure pertains to operations, systems, and computer readable media to capture images of a scene using a camera array and process the captured images based on a viewer's point of view (POV) for immersive augmented reality, live display wall, head mounted display, video conferencing, and similar applications. In one implementation, the disclosed subject matter provides a complete view to a viewer by combining images captured by a camera array. In another implementation, the disclosed subject matter tracks the viewer's POV as he moves from one location to another and displays images in accordance with his varying POV. The change of the viewer's POV is inclusive of movements in the X, Y, and Z dimensions.Type: GrantFiled: January 7, 2019Date of Patent: May 19, 2020Assignee: Apple Inc.Inventors: Ricardo J. Motta, Gary L. Vondran, Jr., Manohar B. Srikanth, Brett D. Miller
-
Publication number: 20200103669Abstract: A scene camera system that includes two or more mirrors that reflect light from a respective portion of a field of view (FOV) in front of the system and two or more cameras that each capture the light reflected by a respective one of the two or more mirrors. By using the mirrors to reflect the light, the cameras' entrance pupils are imaged to a location closer to a user's eyes to thus achieve a more accurate representation of the perspective of the user.Type: ApplicationFiled: September 25, 2019Publication date: April 2, 2020Applicant: Apple Inc.Inventors: Daniel R. Hennigan, Noah D. Bedard, Branko Petljanski, Jason C. Sauers, Edward S. Huo, Ricardo J. Motta, Brett D. Miller, Yury Petrov
-
Publication number: 20190260979Abstract: This disclosure pertains to operations, systems, and computer readable media to capture images of a scene using a camera array and process the captured images based on a viewer's point of view (POV) for immersive augmented reality, live display wall, head mounted display, video conferencing, and similar applications. In one implementation, the disclosed subject matter provides a complete view to a viewer by combining images captured by a camera array. In another implementation, the disclosed subject matter tracks the viewer's POV as he moves from one location to another and displays images in accordance with his varying POV. The change of the viewer's POV is inclusive of movements in the X, Y, and Z dimensions.Type: ApplicationFiled: January 7, 2019Publication date: August 22, 2019Inventors: Ricardo J. Motta, Gary L. Vondran, JR., Manohar B. Srikanth, Brett D. Miller
-
Publication number: 20190221044Abstract: A mixed reality system that includes a head-mounted display (HMD) that provides 3D virtual views of a user's environment augmented with virtual content. The HMD may include sensors that collect information about the user's environment (e.g., video, depth information, lighting information, etc.), and sensors that collect information about the user (e.g., the user's expressions, eye movement, hand gestures, etc.). The sensors provide the information as inputs to a controller that renders frames including virtual content based at least in part on the inputs from the sensors. The HMD may display the frames generated by the controller to provide a 3D virtual view including the virtual content and a view of the user's environment for viewing by the user.Type: ApplicationFiled: March 21, 2019Publication date: July 18, 2019Applicant: Apple Inc.Inventors: Ricardo J. Motta, Brett D. Miller, Tobias Rick, Manohar B. Srikanth
-
Publication number: 20190080432Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.Type: ApplicationFiled: September 7, 2018Publication date: March 14, 2019Inventors: Brett D. Miller, Minwoong Kim, Ricardo J. Motta
-
Patent number: 10178371Abstract: This disclosure pertains to operations, systems, and computer readable media to capture images of a scene using a camera array and process the captured images based on a viewer's point of view (POV) for immersive augmented reality, live display wall, head mounted display, video conferencing, and similar applications. In one implementation, the disclosed subject matter provides a complete view to a viewer by combining images captured by a camera array. In another implementation, the disclosed subject matter tracks the viewer's POV as he moves from one location to another and displays images in accordance with his varying POV. The change of the viewer's POV is inclusive of movements in the X, Y, and Z dimensions.Type: GrantFiled: May 5, 2017Date of Patent: January 8, 2019Assignee: Apple Inc.Inventors: Ricardo J. Motta, Gary L. Vondran, Jr., Manohar B. Srikanth, Brett D. Miller