Patents by Inventor Jan Ernst

Jan Ernst has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 10303942
    Abstract: A short-term cloud forecasting system includes a cloud segmentation processor that receives image data from images captured by an all sky camera. The cloud segmentation processor calculates a probability for each pixel in an image that the pixel is representative of a cloud. A cloud motion estimation processor calculates a motion vector representing estimated cloud motion calculates weights representing the likelihood that the cloud motion will cause a cloud to occlude the sun at a time in the near future. An uncertainty processor calculates one or more uncertainty indexes that quantify the confidence that a cloud forecast is accurate. Combining the cloud probabilities, the global motion vector and the at least one uncertainty index, in a sun occlusion prediction processor produces a short-term cloud forecast based on the image data that may be used as input to control systems for controlling a power grid.
    Type: Grant
    Filed: February 16, 2017
    Date of Patent: May 28, 2019
    Assignee: Siemens Aktiengesellschaft
    Inventors: Ti-chiun Chang, Jan Ernst, Jeremy-Ralph Wiles, Joachim Bamberger, Andrei Szabo
  • Publication number: 20190130603
    Abstract: Systems, methods, and computer-readable media are disclosed for determining feature representations of 2.5D image data using deep learning techniques. The 2.5D image data may be synthetic image data generated from 3D simulated model data such as 3D CAD data. The 2.5D image data may be indicative of any number of pose estimations/camera poses representing virtual or actual viewing perspectives of an object modeled by the 3D CAD data. A neural network such as a convolution neural network (CNN) may be trained using the 2.5D image data as training data to obtain corresponding feature representations. The pose estimations/camera poses may be stored in a data repository in association with the corresponding feature representations. The learnt CNN may then be used to determine an input feature representation from an input 2.5D image and index the input feature representation against the data repository to determine matching pose estimation(s).
    Type: Application
    Filed: March 9, 2017
    Publication date: May 2, 2019
    Inventors: Shanhui Sun, Kai Ma, Stefan Kluckner, Ziyan Wu, Jan Ernst, Vivek Kumar Singh, Terrence Chen
  • Publication number: 20190102909
    Abstract: Systems, methods, and computer-readable media are disclosed for automated identification of parts of a parts assembly using image data of the parts assembly and 3D simulated model data of the parts assembly. The 3D simulated model data may be 3D CAD data of the parts assembly. An image of the parts assembly is captured by a mobile device and sent to a back-end server for processing. The back-end server determines a feature representation corresponding to the image and searches a repository to locate a matching feature representation stored in association with a corresponding pose estimation. The matching pose estimation is rendered as an overlay on the image of the parts assembly, thereby enabling automated identification of parts within the image or some user-selected portion of the image.
    Type: Application
    Filed: March 9, 2017
    Publication date: April 4, 2019
    Inventors: Stefan Kluckner, Shanhui Sun, Kai Ma, Ziyan Wu, Arun Innanje, Jan Ernst, Terrence Chen
  • Publication number: 20190080475
    Abstract: A method for identifying a feature in a first image comprises establishing an initial database of image triplets, and in a pose estimation processor, training a deep learning neural network using the initial database of image triplets, calculating a pose for the first image using the deep learning neural network, comparing the calculated pose to a validation database populated with images data to identify an error case in the deep learning neural network, creating a new set of training data including a plurality of error cases identified in a plurality of input images and retraining the deep learning neural network using the new set of training data. The deep learning neural network may be iteratively retrained with a series of new training data sets. Statistical analysis is performed on a plurality of error cases to select a subset of the error cases included in the new set of training data.
    Type: Application
    Filed: March 13, 2017
    Publication date: March 14, 2019
    Inventors: Kai Ma, Shanhui Sun, Stefan Kluckner, Ziyan Wu, Terrence Chen, Jan Ernst
  • Patent number: 10215714
    Abstract: Method and system for detecting defects on surface of object are presented. An imaging device captures images of surface of object under ambient and dark field illumination conditions. The images are processed with a plurality of image operations to detect area of potential defect at location on surface of object based on predictable pattern consisting of bright and shadow regions. Kernels are defined corresponding to configurations of dark field illumination sources to enhance detecting potential defect. Areas of potential defect are cut from processed images to sub images. Sub images are stitched together to generate hypothesis of potential defect at location on surface of object. The hypothesis is classified with a classifier to determine whether the potential defect is true defect. The classifier is trained with training data having characteristics of true defect. The method provides efficient automated detection of micro defects on surface of object.
    Type: Grant
    Filed: August 16, 2017
    Date of Patent: February 26, 2019
    Assignee: SIEMENS ENERGY, INC.
    Inventors: Ziyan Wu, Rameswar Panda, Jan Ernst, Kevin P. Bailey
  • Publication number: 20190056333
    Abstract: Method and system for detecting defects on surface of object are presented. An imaging device captures images of surface of object under ambient and dark field illumination conditions. The images are processed with a plurality of image operations to detect area of potential defect at location on surface of object based on predictable pattern consisting of bright and shadow regions. Kernels are defined corresponding to configurations of dark field illumination sources to enhance detecting potential defect. Areas of potential defect are cut from processed images to sub images. Sub images are stitched together to generate hypothesis of potential defect at location on surface of object. The hypothesis is classified with a classifier to determine whether the potential defect is true defect. The classifier is trained with training data having characteristics of true defect. The method provides efficient automated detection of micro defects on surface of object.
    Type: Application
    Filed: August 16, 2017
    Publication date: February 21, 2019
    Inventors: Ziyan Wu, Rameswar Panda, Jan Ernst, Kevin P. Bailey
  • Publication number: 20190057498
    Abstract: Method and system for detecting line defects on surface of object are presented. An imaging device captures images of surface of object under ambient and dark field illumination conditions. The images are processed with a plurality of image operations to detect areas of potential defects based on predictable pattern consisting of bright and shadow regions. Areas of potential defect are cut from processed images to sub images. Sub images are stitched together to generate hypotheses of potential defects at locations on surface of object. The hypotheses are classified to determine whether the potential defects are true defects at the locations. Line defect is detected by refining line segments detected on the processed image based on criteria. The criteria include distance from the true defects to the line segments and slops between the true defects and the line segments are less than threshold values.
    Type: Application
    Filed: August 16, 2017
    Publication date: February 21, 2019
    Inventors: Rameswar Panda, Ziyan Wu, Jan Ernst, Kevin P. Bailey
  • Patent number: 10192301
    Abstract: Method and system for detecting line defects on surface of object are presented. An imaging device captures images of surface of object under ambient and dark field illumination conditions. The images are processed with a plurality of image operations to detect areas of potential defects based on predictable pattern consisting of bright and shadow regions. Areas of potential defect are cut from processed images to sub images. Sub images are stitched together to generate hypotheses of potential defects at locations on surface of object. The hypotheses are classified to determine whether the potential defects are true defects at the locations. Line defect is detected by refining line segments detected on the processed image based on criteria. The criteria include distance from the true defects to the line segments and slops between the true defects and the line segments are less than threshold values.
    Type: Grant
    Filed: August 16, 2017
    Date of Patent: January 29, 2019
    Assignee: SIEMENS ENERGY, INC.
    Inventors: Rameswar Panda, Ziyan Wu, Jan Ernst, Kevin P. Bailey
  • Publication number: 20180330194
    Abstract: Embodiments of the present invention provide a computer-implemented method for training an RGB-D classifier for a scene classification task. The method receives task-relevant labeled depth data, task-irrelevant RGB-D data, and a given trained representation in RGB. The method simulates an RGB representation using only the task-irrelevant RGB-D data. The method builds a joint neural network using only the task-irrelevant RGB-D data and the task-relevant labeled depth data.
    Type: Application
    Filed: September 29, 2017
    Publication date: November 15, 2018
    Inventors: Kuan-Chuan Peng, Ziyan Wu, Jan Ernst
  • Publication number: 20180330205
    Abstract: Aspects include receiving a request to perform an image classification task in a target domain. The image classification task includes identifying a feature in images in the target domain. Classification information related to the feature is transferred from a source domain to the target domain. The transferring includes receiving a plurality of pairs of task-irrelevant images that each includes a task-irrelevant image in the source domain and in the target domain. The task-irrelevant image in the source domain has a fixed correspondence to the task-irrelevant image in the target domain. A target neural network is trained to perform the image classification task in the target domain. The training is based on the plurality of pairs of task-irrelevant images. The image classification task is performed in the target domain and includes applying the target neural network to an image in the target domain and outputting an identified feature.
    Type: Application
    Filed: September 29, 2017
    Publication date: November 15, 2018
    Inventors: Ziyan Wu, Kuan-Chuan Peng, Jan Ernst
  • Patent number: 10073848
    Abstract: A database stores reference photographs of an assembly. The reference photographs are from different orientations relative to the assembly. By matching the query photograph to one or more of the reference photographs, the pose of the assembly in the query photograph is determined. Based on the pose, the pixels of the two-dimensional query photograph are related to a three-dimensional representation from engineering data. Using labeled parts from the engineering data, the parts represented in the query photograph are identified, and part information (e.g., segmentation, number, or other metadata) is provided relative to the query photograph.
    Type: Grant
    Filed: March 17, 2015
    Date of Patent: September 11, 2018
    Assignee: Siemens Aktiengesellschaft
    Inventors: Stefan Kluckner, Arun Innanje, Jan Ernst, Terrence Chen
  • Patent number: 10068326
    Abstract: A method for inspecting an object to assist in determining whether the object has a surface defect. The method includes moving the object in a first direction and illuminating the object under ambient lighting conditions. The method also includes capturing at least one image of the object under the ambient lighting conditions while the object moves in the first direction. In addition, the object is illuminated under object lighting conditions and at least one image of the object under the object lighting conditions is captured while the object moves in the first direction to provide at least one object image. Further, the method includes selecting at least one object image having at least one indication of a possible defect to provide images having defect candidates and comparing the defect candidates with previously defined characteristics associated with the defect to facilitate determination of whether a defect exists.
    Type: Grant
    Filed: March 18, 2016
    Date of Patent: September 4, 2018
    Assignee: SIEMENS ENERGY, INC.
    Inventors: Kevin P. Bailey, Ziyan Wu, Jan Ernst, Terrence Chen, Birgi Tamersoy
  • Publication number: 20180232557
    Abstract: A short-term cloud forecasting system includes a cloud segmentation processor that receives image data from images captured by an all sky camera. The cloud segmentation processor calculates a probability for each pixel in an image that the pixel is representative of a cloud. A cloud motion estimation processor calculates a motion vector representing estimated cloud motion calculates weights representing the likelihood that the cloud motion will cause a cloud to occlude the sun at a time in the near future. An uncertainty processor calculates one or more uncertainty indexes that quantify the confidence that a cloud forecast is accurate. Combining the cloud probabilities, the global motion vector and the at least one uncertainty index, in a sun occlusion prediction processor produces a short-term cloud forecast based on the image data that may be used as input to control systems for controlling a power grid.
    Type: Application
    Filed: February 16, 2017
    Publication date: August 16, 2018
    Inventors: Ti-chiun Chang, Jan Ernst, Jeremy-Ralph Wiles, Joachim Bamberger, Andrei Szabo
  • Patent number: 9838583
    Abstract: A method for verifying a lighting setup used for inspecting a micro defect. The method includes simulating a scene including a micro defect, light source and imaging device. A position of the light source and imaging device is then optimized to form an optimized simulated setup for viewing micro defect. A shadow calibration reference (SCR) having a simulated shadow field is then rendered in a location. Next, a physical imaging device and light source are positioned based on information from the optimized simulated setup to form an optimized physical setup. A physical SCR based on information from the SCR rendering is fabricated. Next, an image is captured of a physical SCR in a corresponding location associated with each SCR rendering. The optimized physical setup is verified if at least one shadow parameter from the SCR rendering is substantially similar to a corresponding shadow parameter in a corresponding image.
    Type: Grant
    Filed: September 21, 2015
    Date of Patent: December 5, 2017
    Assignee: SIEMENS ENERGY, INC.
    Inventors: Jan Ernst, Ziyan Wu, Kevin P. Bailey, Terrence Chen
  • Publication number: 20170270651
    Abstract: A method for inspecting an object to assist in determining whether the object has a surface defect. The method includes moving the object in a first direction and illuminating the object under ambient lighting conditions. The method also includes capturing at least one image of the object under the ambient lighting conditions while the object moves in the first direction. In addition, the object is illuminated under object lighting conditions and at least one image of the object under the object lighting conditions is captured while the object moves in the first direction to provide at least one object image. Further, the method includes selecting at least one object image having at least one indication of a possible defect to provide images having defect candidates and comparing the defect candidates with previously defined characteristics associated with the defect to facilitate determination of whether a defect exists.
    Type: Application
    Filed: March 18, 2016
    Publication date: September 21, 2017
    Inventors: Kevin P. Bailey, Ziyan Wu, Jan Ernst, Terrence Chen, Birgi Tamersoy
  • Publication number: 20170085760
    Abstract: A method for verifying a lighting setup used for inspecting a micro defect. The method includes simulating a scene including a micro defect, light source and imaging device. A position of the light source and imaging device is then optimized to form an optimized simulated setup for viewing micro defect. A shadow calibration reference (SCR) having a simulated shadow field is then rendered in a location. Next, a physical imaging device and light source are positioned based on information from the optimized simulated setup to form an optimized physical setup. A physical SCR based on information from the SCR rendering is fabricated. Next, an image is captured of a physical SCR in a corresponding location associated with each SCR rendering. The optimized physical setup is verified if at least one shadow parameter from the SCR rendering is substantially similar to a corresponding shadow parameter in a corresponding image.
    Type: Application
    Filed: September 21, 2015
    Publication date: March 23, 2017
    Inventors: Jan Ernst, Ziyan Wu, Kevin P. Bailey, Terrence Chen
  • Publication number: 20160275079
    Abstract: A database stores reference photographs of an assembly. The reference photographs are from different orientations relative to the assembly. By matching the query photograph to one or more of the reference photographs, the pose of the assembly in the query photograph is determined. Based on the pose, the pixels of the two-dimensional query photograph are related to a three-dimensional representation from engineering data. Using labeled parts from the engineering data, the parts represented in the query photograph are identified, and part information (e.g., segmentation, number, or other metadata) is provided relative to the query photograph.
    Type: Application
    Filed: March 17, 2015
    Publication date: September 22, 2016
    Inventors: Stefan Kluckner, Arun Innanje, Jan Ernst, Terrence Chen
  • Patent number: 9378582
    Abstract: Computer assisted design data is rendered in the cloud. A client-server relationship is provided for 3D rendering. To reduce the burden on the server, the 3D rendering adapts based on the client capabilities. Where possible, some of the 3D rendering is performed by the server and some by the client machine. The 3D rendering by the client machine may be limited to avoid transfer of geometrical data of the CAD data. Different textures or shaders are used for rendering images associated with motion. Dictionary information is accumulated by the client machine to reduce the number of coefficients later transferred to the client machine for 3D rendering. The model represented by the CAD data is used to predict rendered images so that video compression may be performed. The server sparsely renders an image and compressive sensing is used by the client machine to generate the complete image.
    Type: Grant
    Filed: July 31, 2012
    Date of Patent: June 28, 2016
    Assignee: Siemens Product Lifecycle Management Software Inc.
    Inventors: Edward Slavin, III, Jan Ernst, Yakup Genc
  • Publication number: 20150301226
    Abstract: A method for predicting short-term cloud coverage includes a computer calculating an estimated cloud velocity field at a current time value based on sky images. The computer determines a segmented cloud model based on the sky images, a future sun location corresponding to a future time value, and sun pixel locations at the future time value based on the future sun location. Next, the computer applies a back-propagation algorithm to the sun pixel locations using the estimated cloud velocity field to yield propagated sun pixel locations corresponding to a previous time value. Then, the computer predicts cloud coverage for the future sun location based on the propagated sun pixel locations and the segmented cloud model.
    Type: Application
    Filed: April 17, 2014
    Publication date: October 22, 2015
    Applicant: Siemens Aktiengesellschaft
    Inventors: Shanhui Sun, Jan Ernst, Archana Sapkota, Eberhard Ritzhaupt-Kleissl, Jeremy Ralph Wiles, Terrence Chen
  • Publication number: 20150302575
    Abstract: A method for predicting location of the sun in an image space. The method includes providing a set of calibration images and offline intrinsic calibration of a camera and optical element. An extrinsic parameter calibration is then performed based on the calibration images and mapping between local three dimensional coordinates and real world three dimensional coordinates to provide an extrinsic projection matrix. The method also includes providing a real time image of the sky and determining sun location in spherical space based on the extrinsic projection matrix and a real time sun location in the world coordinate system for the real time image. A three dimensional vector is then mapped to provide a corrected two dimensional ideal point. Next, an inverse affine transformation is performed to provide a two dimensional real image point in image space.
    Type: Application
    Filed: May 13, 2015
    Publication date: October 22, 2015
    Inventors: Shanhui Sun, Jan Ernst, Joachim Bamberger, Jeremy Ralph Wiles