Abstract: A dialogue system that dialogues with a user includes an execution device configured to execute a detection process and a recognition process. The recognition process is a process of recognizing that the user talks to the dialogue system when a predetermined state is detected by the detection process. The detection process is a process of detecting the predetermined state using, as an input, an output signal of a camera that images the user. The predetermined state includes a state in which a mouth of the user moves.

Abstract: A dialogue system includes a storage device and an execution device. Scenario data stored in the storage device define a response sentence corresponding to a state and a transition condition for transition to a different state. The execution device executes a text data generation process, a determination process, a scenario response process, a chat process, a storage process, and a return process. The text data generation process is a process of converting a voice of a user into text data. The determination process is a process of determining whether the transition condition is satisfied. The scenario response process is a process of operating a speaker so as to make a response when the transition condition is satisfied. The chat process is a process of operating the speaker so as to make a different response when the transition condition is not satisfied.

Abstract: A face image processing device, includes: an image coordinate system coordinate value derivation unit detecting an x-coordinate value and a y-coordinate value in an image coordinate system at a feature point of an organ of a face of a person in an image, and estimating a z-coordinate value, so as to derive three-dimensional coordinate values in the image coordinate system; a camera coordinate system coordinate value derivation unit deriving three-dimensional coordinate values in a camera coordinate system from the three-dimensional coordinate values in the image coordinate system derived by the image coordinate system coordinate value derivation unit; and a parameter derivation unit applying the three-dimensional coordinate values in the camera coordinate system derived by the camera coordinate system coordinate value derivation unit to a predetermined three-dimensional face shape model to derive a model parameter of the three-dimensional face shape model in the camera coordinate system.

Abstract: A line-of-sight measurement device includes: an imaging unit that images a face of a subject; a light illumination unit that illuminates light to an eye of the subject; a camera coordinate system eyeball center coordinate calculation unit that estimates coordinates of an eyeball center, from a face image imaged by the imaging unit; a pupil center calculation unit that estimates coordinates of an apparent pupil center, from a pupil center position on the face image; an eyeball position orientation estimation unit that calculates an optical axis vector toward the pupil center from the eyeball center on the basis of the coordinates of the eyeball center and the apparent pupil center; a corneal reflection image calculation unit that obtains coordinates of a corneal reflection image on the basis of the coordinates of the eyeball center, the optical axis vector, and a predetermined eyeball model; and an image coordinate calculation unit that estimates image coordinates of a corneal reflection image on the face imag

Abstract: An occupant monitoring device includes: an acquisition unit that acquires a captured image obtained by imaging a region in which there is a probability that a face of an occupant is present in a vehicle; a determination unit that determines whether the captured image acquired by the acquisition unit corresponds to a first image including the face a part of which is hidden by an accessory or a second image including the face a part of which is hidden by a non-accessory object other than the accessory; and a processing unit that detects face information regarding the face of the occupant based on the captured image in different modes according to a determination result in the determination unit, and monitors a state change of the face of the occupant based on a detection result.

Abstract: A face model parameter estimation device includes: an image coordinate system coordinate value derivation unit detecting x-coordinate and y-coordinate values in an image coordinate system at a feature point of an organ of a face in an image and estimating a z-coordinate value to derive three-dimensional coordinate values in the image coordinate system; a camera coordinate system coordinate value derivation unit deriving three-dimensional coordinate values in a camera coordinate system from the three-dimensional coordinate values in the image coordinate system; a parameter derivation unit applying the three-dimensional coordinate values in the camera coordinate system to a three-dimensional face shape model to derive a position and posture parameter of the three-dimensional face shape model in the camera coordinate system; and an error estimation unit estimating a position and posture error between the position and posture parameter and a true parameter and a shape deformation parameter.

Abstract: An occupant modeling device includes: an acquisition section acquiring an image by imaging a region where there is a probability that a face of an occupant is present; a model fitting section generating a model of the face based on a first image acquired by the acquisition section; a tracking section adapting the model to a second image acquired after the first image; a determination section determining correctness of a facial part position included in the second image to which the model is adapted, by using learned information obtained through learning based on correct information and incorrect information regarding the facial part position; and a processing section determining whether a process in the tracking section is to be continuously executed or a process in the model fitting section is to be executed again according to a determination result in the determination section.

Abstract: A face image processing device, includes: an image coordinate system coordinate value derivation unit detecting an x-coordinate value and a y-coordinate value in an image coordinate system at a feature point of an organ of a face of a person in an image, and estimating a z-coordinate value, so as to derive three-dimensional coordinate values in the image coordinate system; a camera coordinate system coordinate value derivation unit deriving three-dimensional coordinate values in a camera coordinate system from the three-dimensional coordinate values in the image coordinate system derived by the image coordinate system coordinate value derivation unit; and a parameter derivation unit applying the three-dimensional coordinate values in the camera coordinate system derived by the camera coordinate system coordinate value derivation unit to a predetermined three-dimensional face shape model to derive a model parameter of the three-dimensional face shape model in the camera coordinate system.

Abstract: An occupant monitoring device includes: an acquisition unit that acquires a captured image obtained by imaging a region in which there is a probability that a face of an occupant is present in a vehicle; a determination unit that determines whether the captured image acquired by the acquisition unit corresponds to at least a first image not including a feature portion of the face of the occupant but including at least a part of a body of the occupant or a second image not including the body of the occupant; and a processing unit that monitors a state change of the face of the occupant based on the captured image, and switches whether or not an occupant parameter set for monitoring the state change of the face of the occupant is to be reset according to a determination result in the determination unit.

Abstract: An apparatus for determining a visual confirmation target, the apparatus includes a gaze detection portion detecting a gaze direction of a driver for a vehicle, a vehicle information acquisition portion, an image acquisition portion acquiring a captured image, a gaze region extraction portion extracting a gaze region of the driver within the captured image based on a detection result of the gaze direction, a candidate detection portion recognizing objects included in the captured image, generating a top-down saliency map based on the captured image and the vehicle information, and detecting an object having saliency in the top-down saliency map among the recognized objects as a candidate for a visual confirmation target, and a visual confirmation target determination portion determining a visual confirmation target on a basis of an extraction result of the gaze region and a detection result of the candidate for the visual confirmation target.

Abstract: A line-of-sight measurement device includes: an imaging unit that images a face of a subject; a light illumination unit that illuminates light to an eye of the subject; a camera coordinate system eyeball center coordinate calculation unit that estimates coordinates of an eyeball center, from a face image imaged by the imaging unit; a pupil center calculation unit that estimates coordinates of an apparent pupil center, from a pupil center position on the face image; an eyeball position orientation estimation unit that calculates an optical axis vector toward the pupil center from the eyeball center on the basis of the coordinates of the eyeball center and the apparent pupil center; a corneal reflection image calculation unit that obtains coordinates of a corneal reflection image on the basis of the coordinates of the eyeball center, the optical axis vector, and a predetermined eyeball model; and an image coordinate calculation unit that estimates image coordinates of a corneal reflection image on the face imag

Abstract: A corneal-reflection-based gaze detection section calculates a time series of a three-dimensional gaze vector in a camera coordinate system from a time series of facial images. A face position-and-orientation estimation section estimates a time series of a three-dimensional position and orientation of a face. An eyeball-center-coordinates transformation section calculates a time series of a three-dimensional position of the eyeball center in a coordinate system of a three-dimensional facial model. A fixed parameter calculation section calculates for use as a fixed parameter a three-dimensional position of the eyeball center in the three-dimensional facial-model coordinate system.

Abstract: An eyeball information detection device includes: an acquisition unit configured to acquire a captured image obtained by imaging a region in which there is a probability that a face of an occupant is present in a vehicle by an imaging device provided in proximity to an illumination device that emits light to the region; and a detection unit configured to output eyeball information that corresponds to the captured image by receiving the captured image acquired by the acquisition unit as an input based on a learned model generated by learning a learning image including information similar to the captured image acquired by the acquisition unit and a correct value of the eyeball information on an eyeball of the occupant appearing in the learning image, by machine learning.

Abstract: A passenger detection device includes: an input unit that receives a captured image of an interior space of a vehicle imaged by an imaging device and a load measured by a load sensor provided on a seat provided in the interior space of the vehicle; a determination unit that determines that a passenger is seated on the seat, in a case where the load is equal to or more than a predetermined threshold value; a detection unit that obtains a face determination degree indicating a certainty of a face of the passenger in the captured image from the captured image; and a threshold value changing unit that changes the threshold value according to the face determination degree.

Abstract: An apparatus for determining a visual confirmation target, the apparatus includes a gaze detection portion detecting a gaze direction of a driver for a vehicle, a vehicle information acquisition portion, an image acquisition portion acquiring a captured image, a gaze region extraction portion extracting a gaze region of the driver within the captured image based on a detection result of the gaze direction, a candidate detection portion recognizing objects included in the captured image, generating a top-down saliency map based on the captured image and the vehicle information, and detecting an object having saliency in the top-down saliency map among the recognized objects as a candidate for a visual confirmation target, and a visual confirmation target determination portion determining a visual confirmation target on a basis of an extraction result of the gaze region and a detection result of the candidate for the visual confirmation target.

Abstract: An eyeball information detection device includes: an acquisition unit configured to acquire a captured image obtained by imaging a region in which there is a probability that a face of an occupant is present in a vehicle by an imaging device provided in proximity to an illumination device that emits light to the region; and a detection unit configured to output eyeball information that corresponds to the captured image by receiving the captured image acquired by the acquisition unit as an input based on a learned model generated by learning a learning image including information similar to the captured image acquired by the acquisition unit and a correct value of the eyeball information on an eyeball of the occupant appearing in the learning image, by machine learning.

Abstract: An occupant modeling device includes: an acquisition section acquiring an image by imaging a region where there is a probability that a face of an occupant is present; a model fitting section generating a model of the face based on a first image acquired by the acquisition section; a tracking section adapting the model to a second image acquired after the first image; a determination section determining correctness of a facial part position included in the second image to which the model is adapted, by using learned information obtained through learning based on correct information and incorrect information regarding the facial part position; and a processing section determining whether a process in the tracking section is to be continuously executed or a process in the model fitting section is to be executed again according to a determination result in the determination section.

Abstract: An occupant monitoring device includes: an acquisition unit that acquires a captured image obtained by imaging a region in which there is a probability that a face of an occupant is present in a vehicle; a determination unit that determines whether the captured image acquired by the acquisition unit corresponds to at least a first image not including a feature portion of the face of the occupant but including at least a part of a body of the occupant or a second image not including the body of the occupant; and a processing unit that monitors a state change of the face of the occupant based on the captured image, and switches whether or not an occupant parameter set for monitoring the state change of the face of the occupant is to be reset according to a determination result in the determination unit.

Abstract: An occupant monitoring device includes: an acquisition unit that acquires a captured image obtained by imaging a region in which there is a probability that a face of an occupant is present in a vehicle; a determination unit that determines whether the captured image acquired by the acquisition unit corresponds to a first image including the face a part of which is hidden by an accessory or a second image including the face a part of which is hidden by a non-accessory object other than the accessory; and a processing unit that detects face information regarding the face of the occupant based on the captured image in different modes according to a determination result in the determination unit, and monitors a state change of the face of the occupant based on a detection result.

Abstract: A passenger information detection device includes: an acquisition unit that acquires an image imaged by an imaging device that is provided in an interior space of a vehicle to image a passenger seated on a seat and a detection value of a load sensor provided on the seat; a first calculation unit that calculates first information that is information on a face of the passenger from the image; and a second calculation unit that calculates second information that is information on a body size of the passenger based on the first information and the detection value.