Patents by Inventor Kazuhiro Nakadai

Kazuhiro Nakadai has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 10839823
    Abstract: A sound source separating device includes: a signal acquiring unit that acquires the sound signal including mixed sounds from a plurality of sound sources; a start information acquiring unit that acquires start information representing a start timing of at least one sound source among the plurality of sound sources; and a sound source separating unit that separates a specific sound source from the sound signal by setting a binary mask controlling presence of the sound source using a variable of “0” and “1” and using a Markov chain for the activation on the basis of the start information and decomposing the spectrogram generated from the sound signal into the base spectrum and the activation through non-negative matrix factorization using the set binary mask S.
    Type: Grant
    Filed: February 13, 2020
    Date of Patent: November 17, 2020
    Assignee: HONDA MOTOR CO., LTD.
    Inventors: Kazuhiro Nakadai, Yuta Kusaka, Katsutoshi Itoyama, Kenji Nishida
  • Publication number: 20200294520
    Abstract: An acoustic signal processing device calculates a signal waveform that a microphone receives when at least one of a sound source and the microphone is moving. The acoustic signal processing device includes a coefficient calculation unit configured to model a steering coefficient gk,m representing how much an amplitude of a sound source signal emitted at an mth discrete time, where m is an integer between 1 and M and M is a length of the sound source signal, is transferred to an amplitude of a signal that the microphone receives at a kth discrete time, where k is an integer between 1 and K and K is a length of a recording signal, using N-order Fourier series expansion where N is an integer of 1 or more, and a recording signal calculation unit configured to calculate the signal waveform that the microphone receives using the modeled steering coefficient gk,m.
    Type: Application
    Filed: March 5, 2020
    Publication date: September 17, 2020
    Inventors: Kazuhiro Nakadai, Hirofumi Nakajima
  • Publication number: 20200296508
    Abstract: A sound source localization device includes: a sound receiving unit that includes two or more microphones; and a sound source localization unit that transforms a sound signal received by each of the microphones into a frequency domain, models a steering vector through Fourier series expansion of an N-th (here, N is an integer equal to or larger than “1”) order for the transformed sound signal of the frequency domain for each of the microphones, calculates a steering vector of an arbitrary angle using the modeled steering vector, and performs localization of a sound source using the calculated steering vector of the arbitrary angle.
    Type: Application
    Filed: March 4, 2020
    Publication date: September 17, 2020
    Inventors: Kazuhiro Nakadai, Hirofumi Nakajima
  • Publication number: 20200293857
    Abstract: A CNN processing device includes: a kernel storage unit configured to store kernels used in a convolution operation; a table storage unit configured to store a Fourier base function used in the convolution operation; and a convolution operation unit configured to model an element g in coefficients G of the kernels in a convolutional neural network (CNN) using N-order (N is an integer equal to or greater than 1) Fourier series expansion and to perform a convolution operation on processing target information that is information on a processing target through a CNN method using the kernels and the Fourier base function.
    Type: Application
    Filed: March 4, 2020
    Publication date: September 17, 2020
    Inventors: Kazuhiro Nakadai, Hirofumi Nakajima
  • Publication number: 20200275200
    Abstract: A sound source localization device includes an acquisition unit configured to acquire acoustic signals of M channels (M is an integer equal to or greater than one), a phase difference information calculator configured to perform a short-time Fourier transform on the acoustic signals of M channels and to convert a time domain into a frequency domain including phase information, and an estimator configured to input phase information of the acoustic signals subjected to the short-time Fourier transform to a deep learning machine and to perform sound source localization of the acoustic signals using the deep learning machine where input follows a von Mises distribution.
    Type: Application
    Filed: February 20, 2020
    Publication date: August 27, 2020
    Inventors: Kazuhiro Nakadai, Shungo Masaki, Ryosuke Kojima, Osamu Sugiyama, Katsutoshi Itoyama, Kenji Nishida
  • Publication number: 20200273480
    Abstract: A sound source separating device includes: a signal acquiring unit that acquires the sound signal including mixed sounds from a plurality of sound sources; a start information acquiring unit that acquires start information representing a start timing of at least one sound source among the plurality of sound sources; and a sound source separating unit that separates a specific sound source from the sound signal by setting a binary mask controlling presence of the sound source using a variable of “0” and “1” and using a Markov chain for the activation on the basis of the start information and decomposing the spectrogram generated from the sound signal into the base spectrum and the activation through non-negative matrix factorization using the set binary mask S.
    Type: Application
    Filed: February 13, 2020
    Publication date: August 27, 2020
    Inventors: Kazuhiro Nakadai, Yuta Kusaka, Katsutoshi Itoyama, Kenji Nishida
  • Publication number: 20200275224
    Abstract: A microphone array position estimation device includes an estimation unit that estimates a position X of a microphone array for maximizing a simultaneous probability P(X,S,Z) of X, Y, and Z through repeated estimation of S and X when the position of the microphone array constituted by M (M is an integer of 1 or greater) microphones is set to X (=(X1T, . . . , XMT)T, T indicates a transposition), spectrums of sound source signals output by the N (N is an integer of 1 or greater) sound sources are set to S (a set related to all of n, f, and t of Snft, f is a frequency bin, and t is a frame index), and spectrums of recorded signals collected by the microphone array are set to Z (a set related to all of f and t of Zft).
    Type: Application
    Filed: February 13, 2020
    Publication date: August 27, 2020
    Inventors: Kazuhiro Nakadai, Katsuhiro Dan, Katsutoshi Itoyama, Kenji Nishida
  • Patent number: 10748544
    Abstract: A voice processing device includes: a sound source localization unit configured to determine a direction of each sound source on the basis of voice signals of a plurality of channels; a sound source separation unit configured to separate signals for respective sound sources indicating components of respective sound sources from the voice signals of the plurality of channels; a speech section detection unit configured to detect a speech section in which the number of speakers is 1 from the signals for respective sound sources; and a speaker identification unit configured to identify a speaker on the basis of the signals for respective sound sources in the speech section.
    Type: Grant
    Filed: March 23, 2018
    Date of Patent: August 18, 2020
    Assignee: HONDA MOTOR CO., LTD.
    Inventors: Kazuhiro Nakadai, Tomoyuki Sahata
  • Patent number: 10741172
    Abstract: A conference system includes an utterance indication processing unit configured to display text information representing utterance content of each speaker on a display unit of each of one or more terminals, and a notification unit configured to notify a speaker of a request to slow down a speech rate of the speaker.
    Type: Grant
    Filed: March 23, 2018
    Date of Patent: August 11, 2020
    Assignee: HONDA MOTOR CO., LTD.
    Inventors: Takashi Kawachi, Kazuhiro Nakadai, Tomoyuki Sahata, Syota Mori, Yuki Uezono, Kyosuke Hineno, Kazuya Maura
  • Patent number: 10674261
    Abstract: A transfer function generation apparatus includes: a modeling part that models, using a function which uses an arrival direction of a sound source as a non-discrete argument, a plurality of acoustic transfer functions to a microphone from sound sources present in a plurality of directions and that stores the modeled function; and a transfer function generation part that generates a transfer function of an arbitrary direction by using the modeled and stored function.
    Type: Grant
    Filed: August 16, 2019
    Date of Patent: June 2, 2020
    Assignee: HONDA MOTOR CO., LTD.
    Inventors: Kazuhiro Nakadai, Hirofumi Nakajima
  • Patent number: 10622008
    Abstract: An audio processing apparatus includes a first-section detection unit configured to detect a first section that is a section in which the power of a spatial spectrum in a sound source direction is higher than a predetermined amount of power on the basis of an audio signal of a plurality of channels, a speech state determination unit configured to determine a speech state on the basis of an audio signal within the first section, a likelihood calculation unit configured to calculate a first likelihood that a type of sound source according to an audio signal within the first section is voice and a second likelihood that the type of sound source is non-voice, and a second-section detection unit configured to determine whether or not a second section in which power is higher than average the power of a speech section is a voice section on the basis of the first likelihood and the second likelihood within the second section.
    Type: Grant
    Filed: June 27, 2016
    Date of Patent: April 14, 2020
    Assignee: HONDA MOTOR CO., LTD.
    Inventors: Keisuke Nakamura, Kazuhiro Nakadai
  • Publication number: 20200077218
    Abstract: An audio processing device includes: a sound source localizing unit configured to determine a localized sound source direction, which is a direction of a sound source, on the basis of audio signals of a plurality of channels acquired from M (here, M is an integer equal to or greater than “3”) sound receiving units of which positions are different from each other; and a sound source position estimating unit configured to, for each set of two sound receiving units, estimate a midpoint of a segment perpendicular to both of half lines directed in estimated sound source directions, which are directions from the sound receiving units to an estimated sound source position of the sound source, as the estimated sound source position.
    Type: Application
    Filed: August 22, 2019
    Publication date: March 5, 2020
    Inventors: Kazuhiro Nakadai, Daniel Patrik Gabriel
  • Publication number: 20200077187
    Abstract: An acoustic signal processing device includes an acoustic signal processing unit configured to calculate a spectrum of each acoustic signal and a steering vector having m elements on the basis of m acoustic signals converted into m digital signals by sampling m analog signals representing sounds collected by m microphones (m is an integer of 1 or more and M or less, and M is an integer of 2 or more), and to estimate a sampling frequency ?m in the sampling on the basis of the spectrum, the steering vector, and a sampling frequency ?ideal that is a predetermined value.
    Type: Application
    Filed: August 28, 2019
    Publication date: March 5, 2020
    Inventors: Katsutoshi Itoyama, Kazuhiro Nakadai
  • Publication number: 20200077185
    Abstract: A transfer function generation apparatus includes: a modeling part that models, using a function which uses an arrival direction of a sound source as a non-discrete argument, a plurality of acoustic transfer functions to a microphone from sound sources present in a plurality of directions and that stores the modeled function; and a transfer function generation part that generates a transfer function of an arbitrary direction by using the modeled and stored function.
    Type: Application
    Filed: August 16, 2019
    Publication date: March 5, 2020
    Inventors: Kazuhiro Nakadai, Hirofumi Nakajima
  • Publication number: 20200066023
    Abstract: An acoustic scene reconstruction device includes: a sound source localization and separation unit configured to perform sound source localization and sound source separation from a collected sound signal; an identification unit configured to identify a kind of a sound source contained in the sound signal; an analysis processing unit configured to estimate a position of the sound source based on a result obtained through the sound source localization and the sound source separation and a result obtained through the identification, select a separation sound and generate visualization information; and a visualization processing unit configured to generate an image corresponding to the sound source is displayed at the estimated position of the sound source by using the visualization information and the separation sound and generate a sound in which the separation sound is reproduced at the estimated position of the sound source.
    Type: Application
    Filed: August 9, 2019
    Publication date: February 27, 2020
    Inventor: Kazuhiro Nakadai
  • Patent number: 10390130
    Abstract: A sound processing apparatus includes an acquisition unit configured to acquire sound signals collected by a microphone array, a sound source localization unit configured to determine a sound source direction on the basis of the sound signals acquired by the acquisition unit, and a sound source identification unit configured to identify a type of sound source on the basis of a sound model indicating a dependence relationship between sound sources, in which the sound model is represented by a probabilistic model expression including sound source localization as an element.
    Type: Grant
    Filed: June 12, 2017
    Date of Patent: August 20, 2019
    Assignee: HONDA MOTOR CO., LTD.
    Inventors: Kazuhiro Nakadai, Ryosuke Kojima
  • Patent number: 10354295
    Abstract: A reception system includes: a visitor recognition unit that recognizes a visitor; a receiving person recognition unit that recognizes a receiving person that corresponds to the visitor; a receiving person contact information storage unit that stores contact information of the receiving person; a notification unit that notifies the receiving person of a visit of the visitor at the contact information of the receiving person stored by the receiving person contact information storage unit; and a receiving person selection unit that selects a substitute receiving person associated with the receiving person in a case where the receiving person is absent when the notification unit notifies the receiving person at the contact information of the receiving person, wherein the notification unit notifies the substitute receiving person selected by the receiving person selection unit when the receiving person is absent.
    Type: Grant
    Filed: March 22, 2017
    Date of Patent: July 16, 2019
    Assignee: HONDA MOTOR CO., LTD.
    Inventors: Naoaki Sumida, Hiroshi Kondo, Asuka Shiina, Shunichi Yamamoto, Kazuhiro Nakadai, Keisuke Nakamura
  • Patent number: 10356520
    Abstract: A sound source localization unit determines a localized sound source direction that is a direction to a sound source on the basis of acoustic signals of a plurality of channels acquired from M (M is an integer equal to or greater than 3) sound pickup units being at different positions, and a sound source position estimation unit determines an intersection of straight lines to an estimated sound source direction, which is a direction from the sound pickup unit to an estimated sound source position of the sound source for each set of the two sound pickup units, classifies a distribution of intersections into a plurality of clusters, and updates the estimated sound source positions so that an estimation probability that is a probability of the estimated sound source positions being classified into clusters corresponding to the sound sources becomes high.
    Type: Grant
    Filed: September 4, 2018
    Date of Patent: July 16, 2019
    Assignee: HONDA MOTOR CO., LTD.
    Inventors: Kazuhiro Nakadai, Daniel Patryk Gabriel, Ryosuke Kojima
  • Patent number: 10283115
    Abstract: A separation unit separates voice signals of a plurality of channels into an incoming component in each incoming direction, a selection unit selects a statistic corresponding to an incoming direction of the incoming component separated by the separation unit from a storage unit which stores a predetermined statistic and a voice recognition model for each incoming direction, an updating unit updates the voice recognition model on the basis of the statistic selected by the selection unit, and a voice recognition unit recognizes a voice of the incoming component separated using the voice recognition model.
    Type: Grant
    Filed: June 15, 2017
    Date of Patent: May 7, 2019
    Assignee: HONDA MOTOR CO., LTD.
    Inventors: Randy Gomez, Kazuhiro Nakadai
  • Patent number: 10245732
    Abstract: A reception system includes: an imaging unit that captures an image; a reception information storage unit that stores an image of a visitor who visited in the past and information of a receiving person whom the visitor visited in the past in an associated manner; and an action performing unit that generates a question to the visitor based on a scenario regarding a response to the visitor and acquires information via a response to the question from the visitor, wherein the action performing unit changes a response presented to the visitor based on the scenario depending on whether or not the reception information storage unit stores an image that corresponds to the image captured by the imaging unit.
    Type: Grant
    Filed: March 27, 2017
    Date of Patent: April 2, 2019
    Assignee: HONDA MOTOR CO., LTD.
    Inventors: Hiroshi Kondo, Naoaki Sumida, Asuka Shiina, Shunichi Yamamoto, Kazuhiro Nakadai, Keisuke Nakamura