Patents by Inventor Mostafa El-Khamy

Mostafa El-Khamy has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11195093
    Abstract: An apparatus, a method, a method of manufacturing and apparatus, and a method of constructing an integrated circuit are provided. The apparatus includes a teacher network; a student network; a plurality of knowledge bridges between the teacher network and the student network, where each of the plurality of knowledge bridges provides a hint about a function being learned, and where a hint includes a mean square error or a probability; and a loss function device connected to the plurality of knowledge bridges and the student network. The method includes training a teacher network; providing hints to a student network by a plurality of knowledge bridges between the teacher network and the student network; and determining a loss function from outputs of the plurality of knowledge bridges and the student network.
    Type: Grant
    Filed: January 10, 2018
    Date of Patent: December 7, 2021
    Inventors: Jaeyoung Kim, Mostafa El-Khamy, Jungwon Lee
  • Publication number: 20210374608
    Abstract: A federated machine-learning system includes a global server and client devices. The server receives updates of weight factor dictionaries and factor strengths vectors from the clients, and generates a globally updated weight factor dictionary and a globally updated factor strengths vector. A client device selects a group of parameters from a global group of parameters, and trains a model using a dataset of the client device and the group of selected parameters. The client device sends to the server a client-updated weight factor dictionary and a client-updated factor strengths vector. The client device receives the globally updated weight factor dictionary and the globally updated factor strengths vector, and retrains the model using the dataset of the client device, the group of parameters selected by the client device, and the globally updated weight factor dictionary and the globally updated factor strengths vector.
    Type: Application
    Filed: January 13, 2021
    Publication date: December 2, 2021
    Inventors: Mostafa EL-KHAMY, Jungwon LEE, Weituo HAO, Lawrence CARIN, Nikhil MEHTA, Kevin J. LIANG
  • Patent number: 11164071
    Abstract: Disclosed herein is convolutional neural network (CNN) system for generating a classification for an input image. According to an embodiment, the CNN system comprises a sequence of neural network layers configured to: derive a feature map based on at least the input image; puncture at least one selection among the feature map and a kernel by setting the value of one or more elements of a row of the at least one selection to zero according to a pattern and cyclic shifting the pattern by a predetermined interval per row to set the value of one or more elements of the rest of the rows of the at least one selection according to the cyclic shifted pattern; convolve the feature map with the kernel to generate a first convolved output; and generate the classification for the input image based on at least the first convolved output.
    Type: Grant
    Filed: June 27, 2017
    Date of Patent: November 2, 2021
    Inventors: Mostafa El-Khamy, Yoo Jin Choi, Jungwon Lee
  • Publication number: 20210319326
    Abstract: A system and method for operating a neural network. In some embodiments, the neural network includes a variational autoencoder, and the training of the neural network includes training the variational autoencoder with a plurality of samples of a first random variable; and a plurality of samples of a second random variable, the plurality of samples of the first random variable and the plurality of samples of the second random variable being unpaired, the training of the neural network including updating weights in the neural network based on a first loss function, the first loss function being based on a measure of deviation from consistency between: a conditional generation path from the first random variable to the second random variable, and a conditional generation path from the second random variable to the first random variable.
    Type: Application
    Filed: May 28, 2020
    Publication date: October 14, 2021
    Inventors: Yoo Jin Choi, Jongha Ryu, Mostafa El-Khamy, Jungwon Lee, Young-Han Kim
  • Publication number: 20210312591
    Abstract: A method and apparatus are provided. The method includes generating a dataset for real-world super resolution (SR), training a first generative adversarial network (GAN), training a second GAN, and fusing an output of the first GAN and an output of the second GAN.
    Type: Application
    Filed: December 24, 2020
    Publication date: October 7, 2021
    Inventors: Haoyu REN, Amin KHERADMAND, Mostafa EL-KHAMY, Shuangquan WANG, Dongwoon BAI, Jungwon LEE
  • Publication number: 20210295173
    Abstract: A method and system are provided. The method includes receiving, at a generator, a random input, producing, at the generator, a synthetic output of the received random input, receiving, at a teacher network, the synthetic output, receiving, at a student network, the synthetic output, minimizing a maximum of a distance between an output of the teacher network and an output of the student network, and constraining the generator.
    Type: Application
    Filed: September 15, 2020
    Publication date: September 23, 2021
    Inventors: Yoo Jin CHOI, Jihwan CHOI, Mostafa EL-KHAMY, Jungwon LEE
  • Publication number: 20210287378
    Abstract: A system for disparity estimation includes one or more feature extractor modules configured to extract one or more feature maps from one or more input images; and one or more semantic information modules connected at one or more outputs of the one or more feature extractor modules, wherein the one or more semantic information modules are configured to generate one or more foreground semantic information to be provided to the one or more feature extractor modules for disparity estimation at a next training epoch.
    Type: Application
    Filed: May 27, 2021
    Publication date: September 16, 2021
    Inventors: Xianzhi Du, Mostafa El-Khamy, Jungwon Lee
  • Patent number: 11094072
    Abstract: A method and system for determining depth information of an image are herein provided. According to one embodiment, the method includes receiving an image input, classifying the input image into a depth range of a plurality of depth ranges, and determining a depth map of the image by applying depth estimation based on the depth range into which the input image is classified.
    Type: Grant
    Filed: September 18, 2019
    Date of Patent: August 17, 2021
    Inventors: Haoyu Ren, Mostafa El-Khamy, Jungwon Lee
  • Publication number: 20210224953
    Abstract: In a method for super resolution imaging, the method includes: receiving, by a processor, a low resolution image; generating, by the processor, an intermediate high resolution image having an improved resolution compared to the low resolution image; generating, by the processor, a final high resolution image based on the intermediate high resolution image and the low resolution image; and transmitting, by the processor, the final high resolution image to a display device for display thereby.
    Type: Application
    Filed: April 6, 2021
    Publication date: July 22, 2021
    Inventors: Mostafa El-Khamy, Jungwon Lee, Haoyu Ren
  • Publication number: 20210217145
    Abstract: A method and system for multi-frame contextual attention are provided. The method includes includes obtaining a reference frame to be processed, identifying context frames with respect to the reference frame, and producing a refined reference frame by processing the obtained reference frame based on the context frames.
    Type: Application
    Filed: April 28, 2020
    Publication date: July 15, 2021
    Inventors: Mostafa EL-KHAMY, Ryan SZETO, Jungwon LEE
  • Patent number: 11055866
    Abstract: An electronic device and method are herein disclosed. The electronic device includes a first camera with a first field of view (FOV), a second camera with a second FOV that is narrower than the first FOV, and a processor configured to capture a first image with the first camera, the first image having a union FOV, capture a second image with the second camera, determine an overlapping FOV between the first image and the second image, generate a disparity estimate based on the overlapping FOV, generate a union FOV disparity estimate, and merge the union FOV disparity estimate with the overlapping FOV disparity estimate.
    Type: Grant
    Filed: March 26, 2019
    Date of Patent: July 6, 2021
    Inventors: Mostafa El-Khamy, Xianzhi Du, Haoyu Ren, Jungwon Lee
  • Patent number: 11043976
    Abstract: A method, system, and non-transitory computer-readable recording medium of decoding a signal are provided. The method includes receiving signal to be decoded, where signal includes at least one symbol; decoding signal in stages, where each at least one symbol of signal is decoded into at least one bit per stage, wherein Log-Likelihood Ratio (LLR) and a path metric are determined for each possible path for each at least one bit at each stage; determining magnitudes of the LLRs; identifying K bits of the signal with smallest corresponding LLR magnitudes; identifying, for each of the K bits, L possible paths with largest path metrics at each decoder stage for a user-definable number of decoder stages; performing forward and backward traces, for each of the L possible paths, to determine candidate codewords; performing a Cyclic Redundancy Check (CRC) on the candidate codewords; and stopping after a first candidate codeword passes the CRC.
    Type: Grant
    Filed: February 11, 2019
    Date of Patent: June 22, 2021
    Inventors: Mostafa El-Khamy, Jinhong Wu, Jungwon Lee, Inyup Kang
  • Publication number: 20210174082
    Abstract: A method for computing a dominant class of a scene includes: receiving an input image of a scene; generating a segmentation map of the input image, the segmentation map being labeled with a plurality of corresponding classes of a plurality of classes; computing a plurality of area ratios based on the segmentation map, each of the area ratios corresponding to a different class of the plurality of classes of the segmentation map; and outputting a detected dominant class of the scene based on a plurality of ranked labels based on the area ratios.
    Type: Application
    Filed: February 17, 2021
    Publication date: June 10, 2021
    Inventors: Qingfeng Liu, Mostafa El-Khamy, Rama Mythili Vadali, Tae-ui Kim, Andrea Kang, Dongwoon Bai, Jungwon Lee, Maiyuran Wijay, Jaewon Yoo
  • Patent number: 11024037
    Abstract: A system for disparity estimation includes one or more feature extractor modules configured to extract one or more feature maps from one or more input images; and one or more semantic information modules connected at one or more outputs of the one or more feature extractor modules, wherein the one or more semantic information modules are configured to generate one or more foreground semantic information to be provided to the one or more feature extractor modules for disparity estimation at a next training epoch.
    Type: Grant
    Filed: June 25, 2019
    Date of Patent: June 1, 2021
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Xianzhi Du, Mostafa El-Khamy, Jungwon Lee
  • Publication number: 20210124985
    Abstract: A computer vision (CV) training system, includes: a supervised learning system to estimate a supervision output from one or more input images according to a target CV application, and to determine a supervised loss according to the supervision output and a ground-truth of the supervision output; an unsupervised learning system to determine an unsupervised loss according to the supervision output and the one or more input images; a weakly supervised learning system to determine a weakly supervised loss according to the supervision output and a weak label corresponding to the one or more input images; and a joint optimizer to concurrently optimize the supervised loss, the unsupervised loss, and the weakly supervised loss.
    Type: Application
    Filed: May 11, 2020
    Publication date: April 29, 2021
    Inventors: Haoyu Ren, Mostafa El-Khamy, Jungwon Lee, Aman Raj
  • Patent number: 10970820
    Abstract: In a method for super resolution imaging, the method includes: receiving, by a processor, a low resolution image; generating, by the processor, an intermediate high resolution image having an improved resolution compared to the low resolution image; generating, by the processor, a final high resolution image based on the intermediate high resolution image and the low resolution image; and transmitting, by the processor, the final high resolution image to a display device for display thereby.
    Type: Grant
    Filed: November 22, 2019
    Date of Patent: April 6, 2021
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Mostafa El-Khamy, Jungwon Lee, Haoyu Ren
  • Publication number: 20210089807
    Abstract: Some aspects of embodiments of the present disclosure relate to using a boundary aware loss function to train a machine learning model for computing semantic segmentation maps from input images. Some aspects of embodiments of the present disclosure relate to deep convolutional neural networks (DCNNs) for computing semantic segmentation maps from input images, where the DCNNs include a box filtering layer configured to box filter input feature maps computed from the input images before supplying box filtered feature maps to an atrous spatial pyramidal pooling (ASPP) layer. Some aspects of embodiments of the present disclosure relate to a selective ASPP layer configured to weight the outputs of an ASPP layer in accordance with attention feature maps.
    Type: Application
    Filed: January 30, 2020
    Publication date: March 25, 2021
    Inventors: Qingfeng Liu, Mostafa El-Khamy, Dongwoon Bai, Jungwon Lee
  • Publication number: 20210083809
    Abstract: Apparatuses (including user equipment (UE) and modern chips for UEs), systems, and methods for UE downlink Hybrid Automatic Repeat reQuest (HARQ) buffer memory management are described. In one method, the entire UE DL HARQ buffer memory space is pre-partitioned according to the number and capacities of the UE's active carrier components. In another method, the UE DL HARQ buffer is split between on-chip and off-chip memory so that each partition and sub-partition is allocated between the on-chip and off-chip memories in accordance with an optimum ratio.
    Type: Application
    Filed: December 1, 2020
    Publication date: March 18, 2021
    Inventors: Mostafa EL-KHAMY, Arvind YEDLA, Sang-Hyuck HA, Hyunsang CHO, Inyup KANG
  • Publication number: 20210065340
    Abstract: A system and method for processing an input video while maintaining temporal consistency across video frames is provided. The method includes converting the input video from a first frame rate to a second frame rate, wherein the second frame rate is a faster frame rate than the first frame rate; generating processed frames of the input video at the second frame rate; and aggregating the processed frames using temporal sliding window aggregation to yield a processed output video at a third frame rate.
    Type: Application
    Filed: April 7, 2020
    Publication date: March 4, 2021
    Inventors: Mostafa EL-KHAMY, Ryan SZETO, Jungwon LEE
  • Patent number: 10938420
    Abstract: Method for decoding signal includes receiving signal, where signal includes at least one symbol; decoding signal in stages, where each at least one symbol of signal is decoded into at least one bit per stage, wherein Log-Likelihood Ratio (LLR) for each at least one bit at each stage is determined, and identified in vector LAPP; performing Cyclic Redundancy Check (CRC) on LAPP, and stopping if LAPP passes CRC; otherwise, determining magnitudes of LLRs in LAPP; identifying K LLRs in LAPP with smallest magnitudes and indexing K LLRs as r={r(1), r(2), . . . , r(K)}; setting Lmax to maximum magnitude of LLRs in LAPP or maximum possible LLR quantization value; setting v=1; generating {tilde over (L)}A(r(k))=LA(r(k))?Lmaxvksign[LAPP(r(k))], for k=1, 2, . . .
    Type: Grant
    Filed: February 11, 2019
    Date of Patent: March 2, 2021
    Inventors: Mostafa El-Khamy, Jinhong Wu, Jungwon Lee, Inyup Kang