Patents by Inventor Dieter Schmalstieg

Dieter Schmalstieg has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 12106688
    Abstract: A head-wearable extended reality (XR) device comprises a frame and a light field display arrangement mounted to the frame. The light field display arrangement comprises a first display layer and a second display layer. At least one of the first display layer or the second display layer may be selectively displaceable relative to the frame.
    Type: Grant
    Filed: May 15, 2023
    Date of Patent: October 1, 2024
    Assignee: Snap Inc.
    Inventors: Christoph Ebner, Alexander Plopski, Dieter Schmalstieg
  • Patent number: 12020378
    Abstract: The present disclosure relates to methods and apparatus for graphics processing. The apparatus may identify at least one mesh associated with at least one frame. The apparatus may also divide the at least one mesh into a plurality of groups of primitives, each of the plurality of groups of primitives including at least one primitive and a plurality of vertices. The apparatus may also compress the plurality of groups of primitives into a plurality of groups of compressed primitives, the plurality of groups of compressed primitives being associated with random access. Additionally, the apparatus may decompress the plurality of groups of compressed primitives, at least one first group of the plurality of groups of compressed primitives being decompressed in parallel with at least one second group of the plurality of groups of compressed primitives.
    Type: Grant
    Filed: August 11, 2021
    Date of Patent: June 25, 2024
    Assignee: QUALCOMM Incorporated
    Inventors: Dieter Schmalstieg, Markus Steinberger, Daniel Mlakar
  • Patent number: 11875452
    Abstract: The present disclosure relates to methods and apparatus for graphics processing. The apparatus may configure a plurality of billboards associated with a viewpoint of a first frame of a plurality of frames, the plurality of billboards being configured in one or more layers at least partially around the viewpoint, the configuration of the plurality of billboards being based on one or more volumetric elements between at least one of the plurality of billboards and the viewpoint. The apparatus may also render an image associated with each of the one or more volumetric elements between at least one billboard of the plurality of billboards and the viewpoint, the rendered image including a set of pixels. The apparatus may also store data in the at least one billboard based on the rendered image associated with each of the one or more volumetric elements, the data corresponding to the set of pixels.
    Type: Grant
    Filed: August 11, 2021
    Date of Patent: January 16, 2024
    Assignee: QUALCOMM Incorporated
    Inventors: Dieter Schmalstieg, Markus Steinberger, Wolfgang Tatzgern
  • Patent number: 11631212
    Abstract: The present disclosure relates to methods and apparatus for graphics processing. Aspects of the present disclosure can determine at least one scene including one or more viewpoints. Also, aspects of the present disclosure can divide the at least one scene into a plurality of zones based on each of the one or more viewpoints. Further, aspects of the present disclosure can determine whether a zone based on one viewpoint of the one or more viewpoints is substantially similar to a zone based on another viewpoint of the one or more viewpoints. Aspects of the present disclosure can also generate a geometry buffer for each of the plurality of zones based on the one or more viewpoints. Moreover, aspects of the present disclosure can combine the geometry buffers for each of the plurality of zones based on the one or more viewpoints.
    Type: Grant
    Filed: June 29, 2021
    Date of Patent: April 18, 2023
    Assignee: QUALCOMM Incorporated
    Inventors: Dieter Schmalstieg, Bernhard Kerbl, Philip Voglreiter
  • Patent number: 11622113
    Abstract: The present disclosure relates to methods and apparatus for graphics processing at a server and/or a client device. In some aspects, the apparatus may convert application data for at least one frame, the application data corresponding to one or more image functions or one or more data channels. The apparatus may also encode the application data for the at least one frame, the application data being associated with a data stream, the application data being encoded via a video encoding process. The apparatus may also transmit the encoded application data for the at least one frame. Additionally, the apparatus may receive application data for at least one frame, the application data being associated with a data stream. The apparatus may also decode the application data for the at least one frame; and convert the application data for the at least one frame.
    Type: Grant
    Filed: August 11, 2021
    Date of Patent: April 4, 2023
    Assignee: QUALCOMM Incorporated
    Inventors: Dieter Schmalstieg, Pascal Stadlbauer, Markus Steinberger
  • Publication number: 20230101978
    Abstract: Aspects presented herein relate to methods and devices for graphics processing including an apparatus, e.g., a GPU. The apparatus may divide at least one scene into a plurality of meshlets, each of the meshlets including a plurality of primitives, and each of the primitives including plurality of vertices. The apparatus may also calculate a pair of texture coordinates for each of the plurality of vertices. Further, the apparatus may select a size of each of the plurality of meshlets in the at least one scene based on the pair of the texture coordinates and based on a perspective projection of each of the plurality of meshlets. The apparatus may also calculate layout information in a meshlet atlas for each of the meshlets in the at least one scene. Moreover, the apparatus may shade each of a plurality of pixels in the meshlet atlas based on the calculated layout information.
    Type: Application
    Filed: September 21, 2022
    Publication date: March 30, 2023
    Inventors: Thomas NEFF, Jörg Hermann MÜLLER, Markus STEINBERGER, Dieter SCHMALSTIEG
  • Patent number: 11380047
    Abstract: The present disclosure relates to methods and apparatus for graphics processing. The apparatus can determine geometry information for each of a plurality of primitives associated with a viewpoint in a scene. The apparatus can also calculate at least one of surface information and disocclusion information based on the geometry information for each of the plurality of primitives, where the surface information and the disocclusion information may be associated with a volumetric grid based on a viewing area corresponding to the viewpoint. Also, the apparatus can calculate visibility information for each of the plurality of primitives based on at least one of the surface information and the disocclusion information, where the visibility information may be associated with the volumetric grid. The apparatus can also determine whether each of the plurality of primitives is visible based on the visibility information for each of the plurality of primitives.
    Type: Grant
    Filed: June 5, 2020
    Date of Patent: July 5, 2022
    Assignee: QUALCOMM Incorporated
    Inventors: Philip Voglreiter, Dieter Schmalstieg, Markus Steinberger
  • Patent number: 11321905
    Abstract: Graphics processing in a client device includes receiving, as part of a vector streaming split rendering process, a bit stream encoding a sequence of shading atlases of the process. Each shading atlas includes blocks. The client decodes a particular shading atlas of the sequence of shading atlases, and determines that a particular block of the particular shading atlas was unsuccessfully decoded. The client identifies a stored block of a prior shading atlas of the sequence of shading atlases as a successfully decoded earlier version of the particular block. The client renders the particular shading atlas using the stored block instead of the particular block.
    Type: Grant
    Filed: September 29, 2020
    Date of Patent: May 3, 2022
    Assignee: QUALCOMM Incorporated
    Inventors: Mina Ayman Saleh Yanni Makar, Soumyajit Bijankumar Deb, Bojan Vrcelj, Arjun Sitaram, Dieter Schmalstieg
  • Publication number: 20220060708
    Abstract: The present disclosure relates to methods and apparatus for graphics processing at a server and/or a client device. In some aspects, the apparatus may convert application data for at least one frame, the application data corresponding to one or more image functions or one or more data channels. The apparatus may also encode the application data for the at least one frame, the application data being associated with a data stream, the application data being encoded via a video encoding process. The apparatus may also transmit the encoded application data for the at least one frame. Additionally, the apparatus may receive application data for at least one frame, the application data being associated with a data stream. The apparatus may also decode the application data for the at least one frame; and convert the application data for the at least one frame.
    Type: Application
    Filed: August 11, 2021
    Publication date: February 24, 2022
    Inventors: Dieter SCHMALSTIEG, Pascal STADLBAUER, Markus STEINBERGER
  • Publication number: 20220058872
    Abstract: The present disclosure relates to methods and apparatus for graphics processing. The apparatus may identify at least one mesh associated with at least one frame. The apparatus may also divide the at least one mesh into a plurality of groups of primitives, each of the plurality of groups of primitives including at least one primitive and a plurality of vertices. The apparatus may also compress the plurality of groups of primitives into a plurality of groups of compressed primitives, the plurality of groups of compressed primitives being associated with random access. Additionally, the apparatus may decompress the plurality of groups of compressed primitives, at least one first group of the plurality of groups of compressed primitives being decompressed in parallel with at least one second group of the plurality of groups of compressed primitives.
    Type: Application
    Filed: August 11, 2021
    Publication date: February 24, 2022
    Inventors: Dieter SCHMALSTIEG, Markus STEINBERGER, Daniel MLAKAR
  • Publication number: 20220058860
    Abstract: The present disclosure relates to methods and apparatus for graphics processing. The apparatus may configure a plurality of billboards associated with a viewpoint of a first frame of a plurality of frames, the plurality of billboards being configured in one or more layers at least partially around the viewpoint, the configuration of the plurality of billboards being based on one or more volumetric elements between at least one of the plurality of billboards and the viewpoint. The apparatus may also render an image associated with each of the one or more volumetric elements between at least one billboard of the plurality of billboards and the viewpoint, the rendered image including a set of pixels. The apparatus may also store data in the at least one billboard based on the rendered image associated with each of the one or more volumetric elements, the data corresponding to the set of pixels.
    Type: Application
    Filed: August 11, 2021
    Publication date: February 24, 2022
    Inventors: Dieter SCHMALSTIEG, Markus STEINBERGER, Wolfgang TATZGERN
  • Publication number: 20210383597
    Abstract: The present disclosure relates to methods and apparatus for graphics processing. The apparatus can determine geometry information for each of a plurality of primitives associated with a viewpoint in a scene. The apparatus can also calculate at least one of surface information and disocclusion information based on the geometry information for each of the plurality of primitives, where the surface information and the disocclusion information may be associated with a volumetric grid based on a viewing area corresponding to the viewpoint. Also, the apparatus can calculate visibility information for each of the plurality of primitives based on at least one of the surface information and the disocclusion information, where the visibility information may be associated with the volumetric grid. The apparatus can also determine whether each of the plurality of primitives is visible based on the visibility information for each of the plurality of primitives.
    Type: Application
    Filed: June 5, 2020
    Publication date: December 9, 2021
    Inventors: Philip VOGLREITER, Dieter SCHMALSTIEG, Markus STEINBERGER
  • Publication number: 20210327122
    Abstract: The present disclosure relates to methods and apparatus for graphics processing. Aspects of the present disclosure can determine at least one scene including one or more viewpoints. Also, aspects of the present disclosure can divide the at least one scene into a plurality of zones based on each of the one or more viewpoints. Further, aspects of the present disclosure can determine whether a zone based on one viewpoint of the one or more viewpoints is substantially similar to a zone based on another viewpoint of the one or more viewpoints. Aspects of the present disclosure can also generate a geometry buffer for each of the plurality of zones based on the one or more viewpoints. Moreover, aspects of the present disclosure can combine the geometry buffers for each of the plurality of zones based on the one or more viewpoints.
    Type: Application
    Filed: June 29, 2021
    Publication date: October 21, 2021
    Inventors: Dieter SCHMALSTIEG, Bernhard KERBL, Philip VOGLREITER
  • Publication number: 20210312691
    Abstract: Graphics processing in a client device includes receiving, as part of a vector streaming split rendering process, a bit stream encoding a sequence of shading atlases of the process. Each shading atlas includes blocks. The client decodes a particular shading atlas of the sequence of shading atlases, and determines that a particular block of the particular shading atlas was unsuccessfully decoded. The client identifies a stored block of a prior shading atlas of the sequence of shading atlases as a successfully decoded earlier version of the particular block. The client renders the particular shading atlas using the stored block instead of the particular block.
    Type: Application
    Filed: September 29, 2020
    Publication date: October 7, 2021
    Inventors: Mina Ayman Saleh Yanni MAKAR, Soumyajit Bijankumar DEB, Bojan VRCELJ, Arjun SITARAM, Dieter SCHMALSTIEG
  • Publication number: 20210304483
    Abstract: Methods, systems, and devices for computer graphics are described. A device may perform a shadow rendering operation in an object-space shading pipeline for a set of potentially visible surfaces. The device may represent the potentially visible surfaces in an atlas. The device may determine a set of world-space coordinates corresponding to the one or more potentially visible surfaces and may store the set of world-space coordinates in a geometry buffer (G-buffer). The device may determine a number of shadow factors associated with the potentially visible surfaces based on a transformation of the world-space coordinates stored in the G-buffer and a set of depth values associated with each shadow caster present in a scene. The device may store the shadow factors in an atlas shadow mask and perform a shading of the potentially visible surfaces in the atlas based on the atlas shadow mask.
    Type: Application
    Filed: March 24, 2020
    Publication date: September 30, 2021
    Inventors: Joseph Daniel Garvey, Dieter Schmalstieg, Bojan Vrcelj
  • Patent number: 11132831
    Abstract: The present disclosure relates to methods and apparatus for graphics processing. Aspects of the present disclosure can determine at least one scene including one or more viewpoints. Also, aspects of the present disclosure can divide the at least one scene into a plurality of zones based on each of the one or more viewpoints. Further, aspects of the present disclosure can determine whether a zone based on one viewpoint of the one or more viewpoints is substantially similar to a zone based on another viewpoint of the one or more viewpoints. Aspects of the present disclosure can also generate a geometry buffer for each of the plurality of zones based on the one or more viewpoints. Moreover, aspects of the present disclosure can combine the geometry buffers for each of the plurality of zones based on the one or more viewpoints.
    Type: Grant
    Filed: March 2, 2020
    Date of Patent: September 28, 2021
    Assignee: QUALCOMM Incorporated
    Inventors: Dieter Schmalstieg, Bernhard Kerbl, Philip Voglreiter
  • Patent number: 11069128
    Abstract: An example system includes a first computing device comprising a first graphics processing unit (GPU) implemented in circuitry, and a second computing device comprising a second GPU implemented in circuitry. The first GPU is configured to perform a first portion of an image rendering process to generate intermediate graphics data and send the intermediate graphics data to the second computing device. The second GPU is configured to perform a second portion of the image rendering process to render an image from the intermediate graphics data. The first computing device may be a video game console, and the second computing device may be a virtual reality (VR) headset that warps the rendered image to produce a stereoscopic image pair.
    Type: Grant
    Filed: July 24, 2018
    Date of Patent: July 20, 2021
    Assignee: QUALCOMM Incorporated
    Inventors: Dieter Schmalstieg, Gerhard Reitmayr, Serafin Diaz Spindola
  • Patent number: 10867431
    Abstract: The present disclosure relates to methods and devices for operation of a GPU. The device can determine a first subset of primitives associated with a set of objects within an image. The first subset of primitives can be based on a first viewpoint with respect to the set of objects. The device can also determine, for a second viewpoint with respect to the set of objects, a second subset of primitives excluding the first subset of primitives. In some aspects, the second subset of primitives can have a difference in depth with respect to the first subset of primitives that is less than a threshold depth. Additionally, the device can mark the first subset of primitives and the second subset of primitives as visible. Further, the device can generate graphical content based on the marked first subset of primitives and the marked second subset of primitives.
    Type: Grant
    Filed: December 17, 2018
    Date of Patent: December 15, 2020
    Assignee: QUALCOMM Technologies, Inc.
    Inventors: Mark Dokter, Dieter Schmalstieg, Markus Steinberger
  • Publication number: 20200364926
    Abstract: The present disclosure relates to methods and devices for operation of a graphics processing unit (GPU). The device can determine a first shading value for each primitive in a first set of primitives associated with objects in a first frame. The device can also determine a second shading value for each primitive in a second set of primitives associated with objects in a second frame. Additionally, the device can calculate a shading difference for each primitive in both the first set of primitives and the second set of primitives. In some aspects, the shading difference is the difference between the first shading value and the second shading value for the primitive. Moreover, the device can shade each primitive in response to determining the shading difference is greater than a threshold, where each shaded primitive is in a third set of primitives.
    Type: Application
    Filed: May 16, 2019
    Publication date: November 19, 2020
    Inventors: Joerg Hermann MUELLER, Thomas NEFF, Dieter SCHMALSTIEG
  • Patent number: 10796458
    Abstract: Aspects of the present disclosure provide a method for compressing a point cloud. The method includes determining a sub-division technique for the volume based on one or more numbers derived from a distribution of points in the volume, dividing the volume into a number of sub-volumes according to the sub-division technique, and determining whether each sub-volume of the sub-volumes is occupied by at least one point. The method includes generating a bit sequence for the volume comprising a control code that is based on the sub-division technique and an occupancy indicator that indicates whether or not each sub-volume is occupied by at least one point.
    Type: Grant
    Filed: April 23, 2018
    Date of Patent: October 6, 2020
    Assignee: QUALCOMM Incorporated
    Inventors: Vinay Melkote Krishnaprasad, Ajit Venkat Rao, Bojan Vrcelj, Dieter Schmalstieg