Patents by Inventor Abdelrahman N. Mahmoud

Abdelrahman N. Mahmoud has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20190283762
    Abstract: Vehicle manipulation uses cognitive state engineering. Images of a vehicle occupant are obtained using imaging devices within a vehicle. The one or more images include facial data of the vehicle occupant. A computing device is used to analyze the images to determine a cognitive state. Audio information from the occupant is obtained and the analyzing is augmented based on the audio information. The cognitive state is mapped to a loading curve, where the loading curve represents a continuous spectrum of cognitive state loading variation. The vehicle is manipulated, based on the mapping to the loading curve, where the manipulating uses cognitive state alteration engineering. The manipulating includes changing vehicle occupant sensory stimulation. Additional images of additional occupants of the vehicle are obtained and analyzed to determine additional cognitive states. Additional cognitive states are used to adjust the mapping. A cognitive load is estimated based on eye gaze tracking.
    Type: Application
    Filed: June 2, 2019
    Publication date: September 19, 2019
    Applicant: Affectiva, Inc.
    Inventors: Rana el Kaliouby, Abdelrahman N. Mahmoud, Panu James Turcot, Andrew Todd Zeilman, Taniya Mishra
  • Publication number: 20190268660
    Abstract: Techniques are disclosed for vehicle video recommendation via affect. A first media presentation is played to a vehicle occupant. The playing is accomplished using a video client. Cognitive state data for the vehicle occupant is captured, where the cognitive state data includes video facial data from the vehicle occupant during the first media presentation playing. The first media presentation is ranked, on an analysis server, relative to another media presentation based on the cognitive state data which was captured for the vehicle occupant. The ranking is determined for the vehicle occupant. The cognitive state data which was captured for the vehicle occupant is correlated, on the analysis server, to cognitive state data collected from other people who experienced the first media presentation. One or more further media presentation selections are recommended to the vehicle occupant, based on the ranking and the correlating.
    Type: Application
    Filed: May 10, 2019
    Publication date: August 29, 2019
    Inventors: Rana el Kaliouby, Abdelrahman N. Mahmoud, Panu James Turcot
  • Publication number: 20190197330
    Abstract: Cognitive state-based vehicle manipulation uses near-infrared image processing. Images of a vehicle occupant are obtained using imaging devices within a vehicle. The images include facial data of the vehicle occupant. The images include visible light-based images and near-infrared based images. A classifier is trained based on the visible light content of the images to determine cognitive state data for the vehicle occupant. The classifier is modified based on the near-infrared image content. The modified classifier is deployed for analysis of additional images of the vehicle occupant, where the additional images are near-infrared based images. The additional images are analyzed to determine a cognitive state. The vehicle is manipulated based on the cognitive state that was analyzed. The cognitive state is rendered on a display located within the vehicle.
    Type: Application
    Filed: March 1, 2019
    Publication date: June 27, 2019
    Inventors: Abdelrahman N. Mahmoud, Rana el Kaliouby, Seyedmohammad Mavadati, Panu James Turcot
  • Publication number: 20190152492
    Abstract: Techniques are described for cognitive analysis for directed control transfer for autonomous vehicles. In-vehicle sensors are used to collect cognitive state data for an individual within a vehicle which has an autonomous mode of operation. The cognitive state data includes infrared, facial, audio, or biosensor data. One or more processors analyze the cognitive state data collected from the individual to produce cognitive state information. The cognitive state information includes a subset or summary of cognitive state data, or an analysis of the cognitive state data. The individual is scored based on the cognitive state information to produce a cognitive scoring metric. A state of operation is determined for the vehicle. A condition of the individual is evaluated based on the cognitive scoring metric. Control is transferred between the vehicle and the individual based on the state of operation of the vehicle and the condition of the individual.
    Type: Application
    Filed: December 28, 2018
    Publication date: May 23, 2019
    Applicant: Affectiva, Inc.
    Inventors: Rana el Kaliouby, Abdelrahman N. Mahmoud, Taniya Mishra, Andrew Todd Zeilman, Gabriele Zijderveld
  • Publication number: 20190133510
    Abstract: Mental state analysis uses sporadic collection of affect data within a vehicle. Mental state data of a vehicle occupant is collected within a vehicle on an intermittent basis. The mental state data includes facial image data and the facial image data is collected intermittently across a plurality of devices within the vehicle. The mental state data further includes audio information. Processors are used to interpolate mental state data in between the collecting which is intermittent. Analysis of the mental state data is obtained on the vehicle occupant, where the analysis of the mental state data includes analyzing the facial image data. An output is rendered based on the analysis of the mental state data. The rendering includes communicating by a virtual assistant, communicating with a navigation component, and manipulating the vehicle. The mental state data is translated into an emoji.
    Type: Application
    Filed: December 3, 2018
    Publication date: May 9, 2019
    Applicant: Affectiva, Inc.
    Inventors: Rana el Kaliouby, Abdelrahman N. Mahmoud, Seyedmohammad Mavadati
  • Publication number: 20190110103
    Abstract: Content manipulation uses cognitive states for vehicle content recommendation. Images are obtained of a vehicle occupant using imaging devices within a vehicle. The one or more images include facial data of the vehicle occupant. A content ingestion history of the vehicle occupant is obtained, where the content ingestion history includes one or more audio or video selections. A first computing device is used to analyze the one or more images to determine a cognitive state of the vehicle occupant. The cognitive state is correlated to the content ingestion history using a second computing device. One or more further audio or video selections are recommended to the vehicle occupant, based on the cognitive state, the content ingestion history, and the correlating. The analyzing can be compared with additional analyzing performed on additional vehicle occupants. The additional vehicle occupants can be in the same vehicle as the first occupant or different vehicles.
    Type: Application
    Filed: December 6, 2018
    Publication date: April 11, 2019
    Applicant: Affectiva, Inc.
    Inventors: Rana el Kaliouby, Abdelrahman N. Mahmoud, Panu James Turcot, Andrew Todd Zeilman, Gabriele Zijderveld
  • Publication number: 20190073547
    Abstract: Personal emotional profile generation uses cognitive state analysis for vehicle manipulation. Cognitive state data is obtained from an individual. The cognitive state data is extracted, using one or more processors, from facial images of an individual captured as they respond to stimuli within a vehicle. The cognitive state data extracted from facial images is analyzed to produce cognitive state information. The cognitive state information is categorized, using one or more processors, against a personal emotional profile for the individual. The vehicle is manipulated, based on the cognitive state information, the categorizing, and the stimuli. The personal emotional profile is generated by comparing the cognitive state information of the individual with cognitive state norms from a plurality of individuals and is based on cognitive state data for the individual that is accumulated over time. The cognitive state information is augmented based on audio data collected from within the vehicle.
    Type: Application
    Filed: October 29, 2018
    Publication date: March 7, 2019
    Applicant: Affectiva, Inc.
    Inventors: Rana el Kaliouby, Abdelrahman N. Mahmoud, Seyedmohammad Mavadati, Gabriele Zijderveld
  • Publication number: 20180330178
    Abstract: Disclosed embodiments provide cognitive state evaluation for vehicle navigation. The cognitive state evaluation is accomplished using a computer, where the computer can perform learning using a neural network such as a deep neural network (DNN) or a convolutional neural network (CNN). Images including facial data are obtained of a first occupant of a first vehicle. The images are analyzed to determine cognitive state data. Layers and weights are learned for the deep neural network. Images of a second occupant of a second vehicle are collected and analyzed to determine additional cognitive state data. The additional cognitive state data is analyzed, and the second vehicle is manipulated. A second imaging device is used to collect images of a person outside the second vehicle to determine cognitive state data. The second vehicle can be manipulated based on the cognitive state data of the person outside the vehicle.
    Type: Application
    Filed: May 9, 2018
    Publication date: November 15, 2018
    Applicant: Affectiva, Inc.
    Inventors: Rana el Kaliouby, Abdelrahman N. Mahmoud, Seyedmohammad Mavadati, Panu James Turcot
  • Patent number: 10074024
    Abstract: Mental state analysis is performed by obtaining video of an individual as the individual interacts with a computer, either by performing various operations, such as driving a vehicle or being a passenger in a vehicle, or by consuming a media presentation. The video is analyzed to determine eye-blink information on the individual, such as eye-blink rate or eye-blink duration. The blink-rate information is compensated for a context. Blinking for a group of people of which the individual is a part is evaluated. Mental states of the individual are inferred for the blink event based on the blink event, the blink duration of the individual, the difference in blinking between the individual and the remainder of the group, and the blink-rate information that was compensated. The blink-rate information and associated mental states can be used to modify an advertisement, a media presentation, a digital game, or vehicle controls.
    Type: Grant
    Filed: August 7, 2017
    Date of Patent: September 11, 2018
    Assignee: Affectiva, Inc.
    Inventors: Rana el Kaliouby, Abdelrahman N. Mahmoud, Seyedmohammad Mavadati, Panu James Turcot
  • Publication number: 20180189581
    Abstract: Disclosed embodiments provide for vehicle manipulation using convolutional image processing. The convolutional image processing is accomplished using a computer, where the computer can include a multilayered analysis engine. The multilayered analysis engine can include a convolutional neural network (CNN). The computer is initialized for convolutional processing. A plurality of images is obtained using an imaging device within a first vehicle. A multilayered analysis engine is trained using the plurality of images. The multilayered analysis engine includes multiple layers that include convolutional layers hidden layers. The multilayered analysis engine is used for cognitive state analysis. The evaluating provides a cognitive state analysis. Further images are analyzed using the multilayered analysis engine. The further images include facial image data from one or more persons present in a second vehicle. Voice data is collected to augment the cognitive state analysis.
    Type: Application
    Filed: March 2, 2018
    Publication date: July 5, 2018
    Inventors: Panu James Turcot, Rana el Kaliouby, Abdelrahman N. Mahmoud, Seyedmohammad Mavadati
  • Publication number: 20180157923
    Abstract: Vehicle cognitive data is collected using multiple devices. A user interacts with various pieces of technology to perform numerous tasks and activities. Reactions can be observed and cognitive states inferred from reactions to the tasks and activities. A first computing device within a vehicle obtains cognitive state data which is collected on an occupant of the vehicle from multiple sources, wherein the multiple sources include at least two sources of facial image data. A second computing device generates analysis of the cognitive state data which is collected from the multiple sources. A third computing device renders an output which is based on the analysis of the cognitive state data. The cognitive state data from multiple sources is tagged. The cognitive state data from the multiple sources is aggregated. The cognitive state data is interpolated when collection is intermittent. The cognitive state analysis is interpolated when the cognitive state data is intermittent.
    Type: Application
    Filed: February 1, 2018
    Publication date: June 7, 2018
    Applicant: Affectiva, Inc.
    Inventors: Rana el Kaliouby, JR., Abdelrahman N. Mahmoud, Seyedmohammad Mavadati, Panu James Turcot
  • Publication number: 20180143635
    Abstract: Vehicle manipulation is performed using occupant image analysis. A camera within a vehicle is used to collect cognitive state data including facial data, on an occupant of a vehicle. A cognitive state profile is learned, on a first computing device, for the occupant based on the cognitive state data. The cognitive state profile includes information on absolute time. The cognitive state profile includes information on trip duration time. Voice data is collected and the cognitive state data is augmented with the voice data. Further cognitive state data is captured, on a second computing device, on the occupant while the occupant is in a second vehicle. The further cognitive state data is compared, on a third computing device, with the cognitive state profile that was learned for the occupant. The second vehicle is manipulated based on the comparing of the further cognitive state data.
    Type: Application
    Filed: January 19, 2018
    Publication date: May 24, 2018
    Applicant: Affectiva, Inc.
    Inventors: Gabriele Zijderveld, Rana el Kaliouby, Abdelrahman N. Mahmoud, Seyedmohammad Mavadati
  • Publication number: 20170337438
    Abstract: Mental state analysis is performed by obtaining video of an individual as the individual interacts with a computer, either by performing various operations, such as driving a vehicle or being a passenger in a vehicle, or by consuming a media presentation. The video is analyzed to determine eye-blink information on the individual, such as eye-blink rate or eye-blink duration. The blink-rate information is compensated for a context. Blinking for a group of people of which the individual is a part is evaluated. Mental states of the individual are inferred for the blink event based on the blink event, the blink duration of the individual, the difference in blinking between the individual and the remainder of the group, and the blink-rate information that was compensated. The blink-rate information and associated mental states can be used to modify an advertisement, a media presentation, a digital game, or vehicle controls.
    Type: Application
    Filed: August 7, 2017
    Publication date: November 23, 2017
    Applicant: Affectiva, Inc.
    Inventors: Rana el Kaliouby, JR., Abdelrahman N. Mahmoud, Seyedmohammad Mavadati, Panu James Turcot
  • Publication number: 20110263946
    Abstract: A digital computer and method for processing data indicative of images of facial and head movements of a subject to recognize at least one of said movements and to determine at least one mental state of said subject is provided. The outputting instructions for providing to a user information relating to at least one said mental state. A further processing data reflective of input from a user, and based at least in part on said input, confirming or modifying said determination and generating with a transducer an output of humanly perceptible stimuli indicative of said at least one mental state.
    Type: Application
    Filed: April 22, 2010
    Publication date: October 27, 2011
    Inventors: Rana el Kaliouby, Rosalind W. Picard, Abdelrahman N. Mahmoud, Youssef Kashef, Miriam Anna Rimm Madsen, Mina Mikhail