Patents by Inventor Mostafa El-Khamy

Mostafa El-Khamy has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 10733714
    Abstract: A method and an apparatus are provided. The method includes receiving a video with a first plurality of frames having a first resolution; generating a plurality of warped frames from the first plurality of frames based on a first type of motion compensation; generating a second plurality of frames having a second resolution, wherein the second resolution is of higher resolution than the first resolution, wherein each of the second plurality of frames having the second resolution is derived from a subset of the plurality of warped frames using a convolutional network; and generating a third plurality of frames having the second resolution based on a second type of motion compensation, wherein each of the third plurality of frames having the second resolution is derived from a fusing a subset of the second plurality of frames.
    Type: Grant
    Filed: April 5, 2018
    Date of Patent: August 4, 2020
    Assignee: Samsung Electronics Co., Ltd
    Inventors: Mostafa El-Khamy, Haoyu Ren, Jungwon Lee
  • Patent number: 10726525
    Abstract: An image denoising neural network training architecture includes an image denoising neural network and a clean data neural network, and the image denoising neural network and clean data neural network share information between each other.
    Type: Grant
    Filed: March 20, 2018
    Date of Patent: July 28, 2020
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Mostafa El-Khamy, Igor Fedorov, Jungwon Lee
  • Publication number: 20200227070
    Abstract: A method and system for providing end-to-end multi-task denoising for joint signal distortion ratio (SDR) and perceptual evaluation of speech quality (PESQ) optimization is herein disclosed. According to one embodiment, an method includes receiving a noisy signal, generating a denoised output signal, determining a signal distortion ratio (SDR) loss function based on the denoised output signal, determining a perceptual evaluation of speech quality (PESQ) loss function based on the denoised output signal, and optimizing an overall loss function based on the PESQ loss function and the SDR loss function.
    Type: Application
    Filed: June 25, 2019
    Publication date: July 16, 2020
    Inventors: Jaeyoung KIM, Mostafa EL-KHAMY, Jungwon LEE
  • Publication number: 20200202109
    Abstract: A system to recognize objects in an image includes an object detection network outputs a first hierarchical-calculated feature for a detected object. A face alignment regression network determines a regression loss for alignment parameters based on the first hierarchical-calculated feature. A detection box regression network determines a regression loss for detected boxes based on the first hierarchical-calculated feature. The object detection network further includes a weighted loss generator to generate a weighted loss for the first hierarchical-calculated feature, the regression loss for the alignment parameters and the regression loss of the detected boxes. A backpropagator backpropagates the generated weighted loss.
    Type: Application
    Filed: March 3, 2020
    Publication date: June 25, 2020
    Inventors: Mostafa EL-KHAMY, Arvind YEDLA, Marcel NASSAR, Jungwon LEE
  • Publication number: 20200202128
    Abstract: A method for computing a dominant class of a scene includes: receiving an input image of a scene; generating a segmentation map of the input image, the segmentation map including a plurality of pixels, each of the pixels being labeled with a corresponding class of a plurality of classes; computing a plurality of area ratios based on the segmentation map, each of the area ratios corresponding to a different class of the plurality of classes of the segmentation map; applying inference to generate a plurality of ranked labels based on the area ratios; and outputting a detected dominant class of the scene based on the plurality of ranked labels.
    Type: Application
    Filed: June 25, 2019
    Publication date: June 25, 2020
    Inventors: Qingfeng Liu, Mostafa El-Khamy, Rama Mythili Vadali, Tae-ui Kim, Andrea Kang, Dongwoon Bai, Jungwon Lee, Maiyuran Wijay, Jaewon Yoo
  • Patent number: 10679351
    Abstract: Detecting objects in an image includes: extracting core instance features from the image; calculating feature maps at multiscale resolutions from the core instance features; calculating detection boxes from the core instance features; calculating segmentation masks for each detection box of the detection boxes at the multiscale resolutions of the feature maps; merging the segmentation masks at the multiscale resolutions to generate an instance mask for each object detected in the image; refining the confidence scores of the merged segmentation masks by auxiliary networks calculating pixel level metrics; and outputting the instance masks as the detected objects.
    Type: Grant
    Filed: January 4, 2018
    Date of Patent: June 9, 2020
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Mostafa El-Khamy, Zhizhong Li, Jungwon Lee
  • Patent number: 10673467
    Abstract: An apparatus and a method. The apparatus includes a receiver including an input for receiving a codeword of length mj, where m and j are each an integer; a processor configured to determine a decoding node tree structure with mj leaf nodes for the received codeword and receive an integer i indicating a level at which parallelism of order m is applied to the decoding node tree structure; and m successive cancellation decoders (SCDs) configured to decode, in parallel, each child node in the decoding node tree structure at level i.
    Type: Grant
    Filed: December 11, 2018
    Date of Patent: June 2, 2020
    Assignee: Samsung Electronics Co., Ltd
    Inventors: Mostafa El-Khamy, Hsien-Ping Lin, Jungwon Lee
  • Publication number: 20200160533
    Abstract: A system for disparity estimation includes one or more feature extractor modules configured to extract one or more feature maps from one or more input images; and one or more semantic information modules connected at one or more outputs of the one or more feature extractor modules, wherein the one or more semantic information modules are configured to generate one or more foreground semantic information to be provided to the one or more feature extractor modules for disparity estimation at a next training epoch.
    Type: Application
    Filed: June 25, 2019
    Publication date: May 21, 2020
    Inventors: Xianzhi Du, Mostafa El-Khamy, Jungwon Lee
  • Patent number: 10657364
    Abstract: A method and apparatus are provided. The method includes receiving an image, detecting an object in the image, determining, by a primary object detector, a primary confidence detection score of the object, determining, by a classification network, a confidence scaling factor of the object, and adjusting the primary confidence detection score based on multiplying the primary confidence detection score by the confidence scaling factor.
    Type: Grant
    Filed: February 16, 2017
    Date of Patent: May 19, 2020
    Assignee: Samsung Electronics Co., Ltd
    Inventors: Mostafa El-Khamy, Xianzhi Du, Jungwon Lee
  • Publication number: 20200134848
    Abstract: An electronic device and method are herein disclosed. The electronic device includes a first camera with a first field of view (FOV), a second camera with a second FOV that is narrower than the first FOV, and a processor configured to capture a first image with the first camera, the first image having a union FOV, capture a second image with the second camera, determine an overlapping FOV between the first image and the second image, generate a disparity estimate based on the overlapping FOV, generate a union FOV disparity estimate, and merge the union FOV disparity estimate with the overlapping FOV disparity estimate.
    Type: Application
    Filed: March 26, 2019
    Publication date: April 30, 2020
    Inventors: Mostafa El-Khamy, Xianzhi Du, Haoyu Ren, Jungwon Lee
  • Publication number: 20200134499
    Abstract: A method and system are herein disclosed. The method includes developing a joint latent variable model having a first variable, a second variable, and a joint latent variable representing common information between the first and second variables, generating a variational posterior of the joint latent variable model, training the variational posterior, and performing inference of the first variable from the second variable based on the variational posterior.
    Type: Application
    Filed: April 3, 2019
    Publication date: April 30, 2020
    Inventors: Jongha RYU, Yoo Jin CHOI, Mostafa EL-KHAMY, Jungwon LEE
  • Patent number: 10635891
    Abstract: A system to recognize objects in an image includes an object detection network outputs a first hierarchical-calculated feature for a detected object. A face alignment regression network determines a regression loss for alignment parameters based on the first hierarchical-calculated feature. A detection box regression network determines a regression loss for detected boxes based on the first hierarchical-calculated feature. The object detection network further includes a weighted loss generator to generate a weighted loss for the first hierarchical-calculated feature, the regression loss for the alignment parameters and the regression loss of the detected boxes. A backpropagator backpropagates the generated weighted loss.
    Type: Grant
    Filed: June 30, 2018
    Date of Patent: April 28, 2020
    Assignee: SAMSUNG ELECTRONICS CO., LTD.
    Inventors: Mostafa El-Khamy, Arvind Yedla, Marcel Nassar, Jungwon Lee
  • Patent number: 10599974
    Abstract: An apparatus and a method. The apparatus includes a first recurrent network in a first layer; a second recurrent network in a second layer connected to the first recurrent network; a distant input gate connected to the second recurrent network; a first highway gate connected to the distant input gate and the second recurrent network; a first elementwise product projection gate connected to the distant input gate, the highway gate, and the second recurrent network; a second highway gate connected to the first recurrent network and the second recurrent network; and a second elementwise product projection gate connected to the first recurrent network, the second highway gate, and the second recurrent network.
    Type: Grant
    Filed: November 4, 2016
    Date of Patent: March 24, 2020
    Assignee: Samsung Electronics Co., Ltd
    Inventors: Mostafa El-Khamy, Jungwon Lee, Jaeyoung Kim
  • Publication number: 20200090305
    Abstract: In a method for super resolution imaging, the method includes: receiving, by a processor, a low resolution image; generating, by the processor, an intermediate high resolution image having an improved resolution compared to the low resolution image; generating, by the processor, a final high resolution image based on the intermediate high resolution image and the low resolution image; and transmitting, by the processor, the final high resolution image to a display device for display thereby.
    Type: Application
    Filed: November 22, 2019
    Publication date: March 19, 2020
    Inventors: Mostafa El-Khamy, Jungwon Lee, Haoyu Ren
  • Patent number: 10489887
    Abstract: In a method for super resolution imaging, the method includes: receiving, by a processor, a low resolution image; generating, by the processor, an intermediate high resolution image having an improved resolution compared to the low resolution image; generating, by the processor, a final high resolution image based on the intermediate high resolution image and the low resolution image; and transmitting, by the processor, the final high resolution image to a display device for display thereby.
    Type: Grant
    Filed: August 7, 2017
    Date of Patent: November 26, 2019
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Mostafa El-Khamy, Jungwon Lee, Haoyu Ren
  • Publication number: 20190347554
    Abstract: A method and system for constructing a convolutional neural network (CNN) model are herein disclosed. The method includes regularizing spatial domain weights, providing quantization of the spatial domain weights, pruning small or zero weights in a spatial domain, fine-tuning a quantization codebook, compressing a quantization output from the quantization codebook, and decompressing the spatial domain weights and using either sparse spatial domain convolution and sparse Winograd convolution after pruning Winograd-domain weights.
    Type: Application
    Filed: September 25, 2018
    Publication date: November 14, 2019
    Inventors: Yoo Jin CHOI, Mostafa EL-KHAMY, Jungwon LEE
  • Patent number: 10380741
    Abstract: Apparatuses and methods of manufacturing same, systems, and methods for object detection using a region-based deep learning model are described. In one aspect, a method is provided, in which a region proposal network (RPN) is used to identify regions of interest (RoI) in an image by assigning a confidence levels, the assigned confidence levels of the RoIs are used to boost the background score assigned by the downstream classifier to each RoI, and the background scores are used in a softmax function to calculate the final class probabilities for each object class.
    Type: Grant
    Filed: April 4, 2017
    Date of Patent: August 13, 2019
    Assignee: Samsung Electronics Co., Ltd
    Inventors: Arvind Yedla, Marcel Nassar, Mostafa El-Khamy, Jungwon Lee
  • Publication number: 20190173498
    Abstract: Method for decoding signal includes receiving signal, where signal includes at least one symbol; decoding signal in stages, where each at least one symbol of signal is decoded into at least one bit per stage, wherein Log-Likelihood Ratio (LLR) for each at least one bit at each stage is determined, and identified in vector LAPP; performing Cyclic Redundancy Check (CRC) on LAPP, and stopping if LAPP passes CRC; otherwise, determining magnitudes of LLRs in LAPP; identifying K LLRs in LAPP with smallest magnitudes and indexing K LLRs as r={r(1), r(2), . . . , r(K)}; setting Lmax to maximum magnitude of LLRs in LAPP or maximum possible LLR quantization value; setting v=1; generating {tilde over (L)}A(r(k))=LA(r(k))?Lmaxvksign[LAPP(r(k))], for k=1, 2, . . .
    Type: Application
    Filed: February 11, 2019
    Publication date: June 6, 2019
    Inventors: Mostafa EL-KHAMY, Jinhong WU, Jungwon LEE, Inyup KANG
  • Publication number: 20190173497
    Abstract: A method, system, and non-transitory computer-readable recording medium of decoding a signal are provided. The method includes receiving signal to be decoded, where signal includes at least one symbol; decoding signal in stages, where each at least one symbol of signal is decoded into at least one bit per stage, wherein Log-Likelihood Ratio (LLR) and a path metric are determined for each possible path for each at least one bit at each stage; determining magnitudes of the LLRs; identifying K bits of the signal with smallest corresponding LLR magnitudes; identifying, for each of the K bits, L possible paths with largest path metrics at each decoder stage for a user-definable number of decoder stages; performing forward and backward traces, for each of the L possible paths, to determine candidate codewords; performing a Cyclic Redundancy Check (CRC) on the candidate codewords; and stopping after a first candidate codeword passes the CRC.
    Type: Application
    Filed: February 11, 2019
    Publication date: June 6, 2019
    Inventors: Mostafa EL-KHAMY, Jinhong WU, Jungwon LEE, Inyup KANG
  • Publication number: 20190139205
    Abstract: A method and an apparatus are provided. The method includes receiving a video with a first plurality of frames having a first resolution; generating a plurality of warped frames from the first plurality of frames based on a first type of motion compensation; generating a second plurality of frames having a second resolution, wherein the second resolution is of higher resolution than the first resolution, wherein each of the second plurality of frames having the second resolution is derived from a subset of the plurality of warped frames using a convolutional network; and generating a third plurality of frames having the second resolution based on a second type of motion compensation, wherein each of the third plurality of frames having the second resolution is derived from a fusing a subset of the second plurality of frames.
    Type: Application
    Filed: April 5, 2018
    Publication date: May 9, 2019
    Inventors: Mostafa El-Khamy, Haoyu Ren, Jungwon Lee