Abstract: Disclosed are a device and a method for providing automatically focusing on audio and video. An operation method of an electronic device comprises registering objects of interest, capturing video, displaying the video on a display unit, inferring at least one object of interest included in the video among the objects of interest, adjusting a zoom of the video being captured by controlling the capturing unit based on a position of the at least one object of interest and performing audio focusing by adjusting activity of each of the multiple microphones based on the position of the at least one object of interest. Accordingly, in capturing video by means of the electronic device, the satisfaction with capturing results can be enhanced by emphasizing the audio of interest and by video focusing.

Abstract: In some embodiments, an apparatus includes a memory configured to store data and a controller coupled to the memory. The controller is configured to receive, from a computing device coupled to the apparatus, one or more frames of a digital video. The controller is also configured to analyze one or more components of the memory. The controller is further configured to determine a set of states for the one or more components of the memory based on the analysis of the one or more components of the memory. The controller is further configured to determine a first encoding rate for the digital video from a plurality of encoding rates based on the set of states for the one or more components of the memory. The controller is further configured to encode the digital video based on the first encoding rate and to store the encoded digital video in the memory.

Abstract: The present invention eliminates meaningless searching for an object, and increases the probability that an image the user likes can be obtained. An image capturing apparatus comprises an image capturing device configured to capture an object image, an object detection unit configured to detect an object from image data captured by the image capturing device, a state detection unit configured to detect information pertaining to a state in which the image capturing apparatus is being held, and a control unit configured to control a range in which the object detection unit searches for an object, on the basis of state information of the image capturing apparatus detected by the state detection unit.

Abstract: The present disclosure generally relates to embodiments for video communication interfaces for automatically adjusting a displayed representation of a field-of-view of a camera in response to detecting a change in a scene.

Abstract: The present disclosure relates to an artificial intelligence (AI) system utilizing a machine learning algorithm such as deep learning, etc. and an application thereof. In particular, a controlling method of an electronic apparatus includes obtaining a user voice of a first user, converting the voice of the first user into a first spectrogram, obtaining a second spectrogram by inputting the first spectrogram to a trained model through an artificial intelligence algorithm, converting the second spectrogram into a voice of a second user, and outputting the converted second user voice. Here, the trained model is a model trained to obtain a spectrogram of a style of the second user voice by inputting a spectrogram of a style of the first user voice. In particular, at least part of the controlling method of the electronic apparatus uses an artificial intelligence model trained according to at least one of machine learning, a neural network or a deep learning algorithm.

Abstract: An imaging device includes a processor including hardware. The processor is configured to implement controlling a focus position of an objective optical system configured to form an image of a subject on an image sensor, acquiring L×N images per second captured by the image sensor, and combining acquired M images into one extended depth of field image to extend a depth of field, and outputting L extended depth of field images per second. The processor sets one of the M images as a reference image, performs positioning of the other image or images of the M images with respect to the reference image, and combines the thus positioned M images into the one extended depth of field image.

Abstract: A system and method is disclosed for identifying demographics of an audience (e.g., customers, shoppers and the like) using machine learning and delivering relative content to the audience. In one embodiment, a processing unit receives images and/or audio of an audience and uses a machine learning logic to identify audience demographical characteristics. Demographical characteristics are used to select and deliver audio entertainment, and/or audio information, and/or visual entertainment and/or visual information to the audience that is relevant to the audience. In other embodiments, demographic information is used to analyze audience or customer demographics based on time periods (hour, day, week, etc.), location(s), and/or point of sale data.

Abstract: The present disclosure relates to a device and method of providing audio focusing on multiple objects of interest, the method includes: capturing a video: displaying the video on the display unit; obtaining multiple objects of interest from the video on the basis of a user's input; setting importance of each of the multiple objects of interest; obtaining location information of each of the multiple objects of interest; and allocating audio focusing to the multiple objects of interest on the basis of the importance and the location information of each of the multiple objects of interest, whereby it is possible to provide audio focusing on multiple objects of interest during the video capturing of the electronic device, thereby improving the satisfaction with the video capturing result.

Abstract: Method of performing acoustic zooming starts with microphones capturing acoustic signals associated with video content. Beamformers generate beamformer signals using the acoustic signals. Beamformer signals correspond respectively to tiles of video content. Each of the beamformers is respectively directed to a center of each of the tiles. Target enhanced signal is generated using beamformer signals. Target enhanced signal is associated with a zoom area of video content. Target enhanced signal is generated by identifying the tiles respectively having at least portions that are included in the zoom area, selecting beamformer signals corresponding to identified tiles, and combining selected beamformer signals to generate target enhanced signal. Combining selected beamformer signals may include determining proportions for each of the identified tiles in relation to the zoom area and combining selected beamformer signals based on the proportions to generate the target enhanced signal.

Abstract: Method of performing acoustic zooming starts with microphones capturing acoustic signals associated with video content. Beamformers generate beamformer signals using the acoustic signals. Beamformer signals correspond respectively to tiles of video content. Each of the beamformers is respectively directed to a center of each of the tiles. Target enhanced signal is generated using beamformer signals. Target enhanced signal is associated with a zoom area of video content. Target enhanced signal is generated by identifying the tiles respectively having at least portions that are included in the zoom area, selecting beamformer signals corresponding to identified tiles, and combining selected beamformer signals to generate target enhanced signal. Combining selected beamformer signals may include determining proportions for each of the identified tiles in relation to the zoom area and combining selected beamformer signals based on the proportions to generate the target enhanced signal.

Abstract: A system configured to improve spatial coverage of output audio and a corresponding user experience by performing upmixing and loudspeaker beamforming to stereo input signals. The system can perform upmixing to the stereo (e.g., two channel) input signal to extract a center channel and generate three-channel audio data. The system may then perform loudspeaker beamforming to the three-channel audio data to enable two loudspeakers to generate output audio having three distinct beams. The user may interpret the three distinct beams as originating from three separate locations, resulting in the user perceiving a wide virtual sound stage despite the loudspeakers being spaced close together on the device.

Abstract: Disclosed are a method for playing sound and a multi-screen terminal. The method includes: detecting, by a multi-screen terminal, a current opening/closing angle or angles between the respective display screens; and selecting, by the multi-screen terminal according to the detected current opening/closing angle or angles, a group of prestored audio drive parameters, and outputting the group of audio drive parameters to a power amplification module of the multi-screen terminal; or, determining whether the current opening/closing angle or angles and the current audio drive parameters are respectively prestored opening/closing angle or angles and audio drive parameters corresponding to an optimal sound field playing effect, and if not, giving a prompt as to whether to adjust the opening/closing angle or angles between the respective display screens and/or the current audio drive parameters.

Abstract: Systems and methods can be implemented to include a speaker system with microphone room calibration in a variety of applications. The speaker system can be implemented as a smart speaker. The speaker system can include a microphone array having multiple microphones, one or more optical sensors, one or more processors, and a storage device comprising instructions. The one or more optical sensors can be used to determine distances of one or more surfaces to the speaker system. Based on the determined distances, an algorithm to manage beamforming of an incoming voice signal to the speaker system can be adjusted or selected one or more microphones of the microphone array can be turned off, with an adjustment of an evaluation of the voice signal to the microphone array to account for the one or more microphones turned off. Additional systems and methods are disclosed.

Abstract: A method includes receiving an input signal comprising an original domain signal and creating a first window data set and a second window data set from the signal, wherein an initiation of the second window data set is offset from an initiation of the first window data set, converting the first window data set and the second window data set to a frequency domain and storing the resulting data as data in a second domain different from the original domain, performing complex spectral phase evolution (CSPE) on the second domain data to estimate component frequencies of the first and second window data sets, using the component frequencies estimated in the CSPE, sampling a set of second-domain high resolution windows to select a mathematical representation comprising a second-domain high resolution window that fits at least one of the amplitude, phase, amplitude modulation and frequency modulation of a component of an underlying signal wherein the component comprises at least one oscillator peak, generating an ou

Abstract: The present disclosure relates to a device and method of providing automatic audio focusing, the method includes: registering objects of interest; capturing a video; displaying the video on a display; recognizing at least one object included in the video; inferring at least one object of interest included in the video from the recognized at least one object; identifying distribution of the at least one object of interest in the video; and performing audio focusing on the at least one object of interest by adjusting activity of each of multiple microphones included in a microphone array on the basis of the distribution of the at least one object of interest in the video, whereby it is possible to emphasize voice of the object of interest during the video capturing of the electronic device, thereby improving the satisfaction with the video capturing result.

Abstract: A method and system is provided for refocusing images captured by a plenoptic camera. In one embodiment the plenoptic camera is in processing with an audio capture device. The method comprises the steps of determining direction of a dominant audio source associated with an image; creating an audio zoom by filtering out all other audio signals except those associated with said dominant audio source; and performing automatic refocusing of said image based on said created audio zoom.

Abstract: Systems and methods are described for determining a material of an object using sound. Exemplary methods employ a head-mounted display (HMD). In an embodiment, the method includes determining, using a depth camera function of the HMD, a distance to an object; emitting, using a speaker of the AR HMD, a generated sound signal; and responsive to emitting the sound signal, detecting a reflected sound signal. Relative to the emitted sound signal, the HMD determines attenuation levels for a temporal portion of the reflected sound signal for at least two frequency ranges of the reflected sound signal, the temporal portion of the reflected sound signal corresponding to a computed round-trip travel time of the sound signal traveling the determined distance to and from the object. Based upon the determined attenuation levels, the determines a material corresponding to the attenuation levels.

Abstract: Hyperlapse results are generated from wide-angled, panoramic video. A set of wide-angled, panoramic video data is obtained. Video stabilization is performed on the obtained set of wide-angled, panoramic video data. Without user intervention, a smoothed camera path is automatically determined using at least one region of interest that is determined using saliency detection and semantically segmented frames of stabilized video data resulting from the video stabilization. A set of frames is determined to vary the velocity of wide-angled, panoramic rendered display of the hyperlapse results.

Abstract: An audio processing apparatus includes a transform unit that transforms time series audio data obtained from first and second microphones into first and second frequency spectrum data; a driving noise computation processing unit that computes a subtraction amount of the driving noise for each of frequencies from the first and second frequency spectrum data obtained by the transform unit; a generating unit that, on the basis of the first and the second frequency spectrum data obtained by the transform unit and the driving noise subtraction amount obtained by the driving noise computation processing unit, generates left and right channel frequency spectrum data in which the driving noise is respectively suppressed; and an inverse transform unit that inverse-transforms the left and right channel frequency spectrum data generated by the generating unit into left and right channel time series audio data, respectively.

Abstract: A camera system capable of capturing images of an event in a dynamic environment includes two microphones configured to capture stereo audio of the event. The microphones are on orthogonal surfaces of the camera system. Because the microphones are on orthogonal surfaces of the camera system, the camera body can impact the spatial response of the two recorded audio channels differently, leading to degraded stereo recreation if standard beam forming techniques are used. The camera system includes tuned beam forming techniques to generate multi-channel audio that more accurately recreates the stereo audio by compensating for the shape of the camera system and the orientation of microphones on the camera system. The tuned beam forming techniques include optimizing a set of beam forming parameters, as a function of frequency, based on the true spatial response of the recorded audio signals.

Abstract: An apparatus configured to: determine a viewing angle associated with at least one apparatus camera; determine from at least two audio signals at least one audio source orientation relative to an apparatus; and generate at least one spatial filter including at least a first orientation range associated with the viewing angle and a second orientation range relative to the apparatus.

Abstract: The present invention provides a technique that simplifies operations for making appropriate print settings. Accordingly, a terminal apparatus serving as an information processing apparatus according to the present invention acquires, from a printing apparatus, information indicating the sheet type and size of the sheets set in each paper feed tray. Then, the terminal apparatus determines whether the acquired information includes a sheet type that matches a sheet type suitable for an attribute of information selected by a user as a print target. If it is determined that the information includes a sheet type that matches the suitable sheet type, the terminal apparatus generates print data including information designating a paper feed tray containing sheets of the matching sheet type, and transmits the generated print data to the printing apparatus.

Abstract: The spectral response of an omnidirectional microphone is used as a reference. This reference is compared to the spectral response of each directional microphone to develop scale factors that are applied to the directional microphone spectral response to perform spectral equalization. The outputs of the omnidirectional microphone and the directional microphones are decomposed into a series of sub-bands and the comparison and equalization is done for each sub-band. The equalized sub-bands are then converted into a time domain signal for further processing by the conference phone or video conference system.

Abstract: The voice recorder includes a right-and-left pair of microphones. Each microphone is fixed to a holder that is rotatable about a rotation axis, and a sound-collection axis of each microphone is tilted to the rotation axis. Turning each holder by 180 degrees allows each microphone to change over between an inward position where the sound-collecting axes SAX of the microphones intersect and an outward position where the sound-collecting axes do not intersect. Each microphone is placed offset from the rotation axis on each holder in such a direction that the sound-collecting axes are spaced apart from each other in a height-direction (Z-direction) while the microphones are in the inward position.

Abstract: An image pickup device which captures sound and a moving image prevents deterioration in a reproduction quality. A scene change detector detects a frame at the time of a scene change from among a plurality of frames imaged at a predetermined frame rate as a detection frame. A frame rate converting unit converts a frame rate of the frame imaged outside a detection to a lower frame rate. A video reproduction time setting unit sets a reproduction time when reproduction is performed at the lower frame rate as a video reproduction time. An audio reproduction time setting unit sets an audio reproduction time at constant intervals for sounds recorded at constant intervals outside the detection period and sets an audio reproduction time in synchronization with the video reproduction time corresponding to the detection frame relative to sound recorded in the detection period.

Abstract: Embodiments herein relate generally to changing spatial audio fields that are defined for audio sources. In the embodiments, the spatial audio fields are indicated to a user performing audio mixing, for instance by displaying them as polygons on a touch screen. The spatial audio fields move as the related audio sources move, and/or as the position of a notional consumer changes. Apparatus of the embodiments is configured to detect whether at any time (initially, or after movement) there is overlapping of two spatial audio fields. If an overlap is detected, this is indicated to a user performing audio mixing The apparatus then responds to a user input (e.g. a gesture on the touch screen) by detecting the nature of the user input and then moving or sizing one or both of overlapping spatial audio fields and such that overlapping is avoided or reduced.

Abstract: Methods and apparatus are provided providing users with the ability to create and produce multimedia devices. In one aspect of the present invention, users are provided with the capability to easily and seamlessly create slideshows using multiple forms of graphic elements instead of just still pictures. In another aspect of the present invention, users are provided with the capability to create and modify the DVD menu that is required for DVDs to function properly on conventional DVD players. In still another aspect of the present invention, users are provided with an intuitive graphic interface that simply and clearly explains the trade offs the user must make in deciding which mode to record the DVD.

Abstract: A multi-MEMS module is specified which can be produced expediently and enables a smaller design. The module comprises a housing having an interior and a first and a second opening, a first MEMS chip and a second MEMS chip. The first MEMS chip is acoustically coupled to the first opening. The second MEMS chip is acoustically coupled to the second opening.

Abstract: Provided is a video processing method and apparatus. The video processing method includes acquiring an input video including a plurality of video frames and audio frames; dividing the input video into one or more sections; determining a representative video frame from among the plurality of video frames with respect to each of the one or more sections; and acquiring a slide video that includes the representative video frames.

Abstract: A method and system for capturing, sharing, viewing, and/or displaying one or more video videos. A user of a computing device performs a gesture involving contacting a touch sensitive display. In response, a video segment is captured while the user maintains contact with the touch sensitive display. Upon releasing contact with the touch sensitive display, recording of the video segment is ceased. In one or more embodiments of the invention, the user may then record one or more additional video segments to be included in a video vignette.

Abstract: According to the present invention, a recording apparatus includes an apparatus body having a front surface and a rear surface that constitute a front and a back, a first microphone which is placed in the front surface of the apparatus body and which has a predetermined directivity, a second microphone which is provided in the rear surface facing the front surface of the apparatus body and which has a directivity that is narrower than that of the first microphone, and a recording processing unit which switches a microphone for use in recording processing between the first microphone and the second microphone to perform the recording processing in accordance with the state of the apparatus body.

Abstract: There is provided a display apparatus. A reception unit receives a captured image from an image capturing apparatus. A display unit displays the captured image. A detection unit detects an attitude of the display apparatus. A transmission unit, while the display unit is displaying the captured image, transmits attitude information to the image capturing apparatus in response to an instruction from a user. The attitude information indicates the attitude detected by the detection unit.

Abstract: Example embodiments disclosed herein relate to user experience oriented audio signal processing. There is provided a method for user experience oriented audio signal processing. The method includes obtaining a first audio signal from an audio sensor of an electronic device; computing, based on the first audio signal, a compensation factor for an acoustic path from the electronic device to a listener and applying the compensation factor to a second audio signal outputted from the electronic device. Corresponding system and computer program products are disclosed.

Abstract: When a microphone array apparatus sound-picks up voice in a prescribed sound volume or higher, which is output from a sound source, and sends voice data on the voice to voice processing apparatus, a sound source direction detection unit causes sound source marks, each of which indicates a directivity direction, to be displayed on a display, and urges a user to make a selection among the sound source marks and to input camera information. A voice processing apparatus transmits the camera information that is input, and the directivity direction, to the microphone array apparatus. The microphone array apparatus stores the camera information and the directivity direction, as a preset information table, in a storage unit. Accordingly, where a positional relationship between the camera and the microphone array is unclear, directionality is formed in a determined image capture position, and voice in the predetermined image capture position is output clearly.

Abstract: A display apparatus transmits a picture acquisition request for getting picture information to an external image apparatus connected through a predetermined interface to the display apparatus from the external image apparatus at predetermined intervals and gets a plurality of pieces of picture information from the external image apparatus to be displayed. The plurality of pictures may be switched at predetermined intervals, for example, to be displayed, so that the plurality of pictures may be displayed in a so-called slide show manner. A plurality of pictures for thumbnail may be produced from the plurality of pieces of picture information and be arranged together to be displayed in one picture screen of a display device.

Abstract: There is provided a server including a reception section which receives, from a client terminal, present position information showing a position of the client terminal, and direction information showing an orientation of the client terminal, a retrieval section which retrieves sensory data to which detection position information is added corresponding to a position in a vicinity of an axial line extending in a direction shown by the direction information from the position of the client terminal, and a transmission section which transmits the sensory data retrieved by the retrieval section to the client terminal.

Abstract: A method of controlling a mobile terminal, and which includes displaying an image on a touchscreen of the mobile terminal; receiving an audio synthesis command for synthesizing audio with the image; saving at least one audio candidate in association with the image, based on the received audio synthesis command; and displaying an indicator on the touchscreen indicating the at least one audio candidate is saved with the image.

Abstract: An apparatus configured to: determine a viewing angle associated with at least one apparatus camera; determine from at least two audio signals at least one audio source orientation relative to an apparatus; and generate at least one spatial filter including at least a first orientation range associated with the viewing angle and a second orientation range relative to the apparatus.

Abstract: An imaging apparatus comprising a photographing unit which performs photographing and a sound collection unit which collects voice, comprising determining whether a condition to enable a voice command function which executes predetermined processing according to a voice associated with the predetermined processing in a case where a selfie mode, which is an operation mode for photographing, by an operator, the operator himself with the photographing unit, is set, enabling the voice command function in a case where the selfie mode is not set, and changing to enable the voice command function according to a change from a state in which the condition to enable the voice command function is not satisfied to a state in which the condition is satisfied with the voice command function in a disable state in a case where the selfie mode is set.

Abstract: Techniques for processing directionally-encoded audio to account for spatial characteristics of a listener playback environment are disclosed. The directionally-encoded audio data includes spatial information indicative of one or more directions of sound sources in an audio scene. The audio data is modified based on input data identifying the spatial characteristics of the playback environment. The spatial characteristics may correspond to actual loudspeaker locations in the playback environment. The directionally-encoded audio may also be processed to permit focusing/defocusing on sound sources or particular directions in an audio scene. The disclosed techniques may allow a recorded audio scene to be more accurately reproduced at playback time, regardless of the output loudspeaker setup. Another advantage is that a user may dynamically configure audio data so that it better conforms to the user's particular loudspeaker layouts and/or the user's desired focus on particular subjects or areas in an audio scene.

Abstract: One example device includes a camera; a display device; a memory; and a processor in communication with the memory to receive audio signals from two or more microphones or a far-end device; receive first location information and second location information, the first location information for a visual identification of an audio source of the received audio signals and the second location information identifying a direction of arrival from the audio source; receive a first adjustment to a first portion of a UI to change either a visual identification or a coordinate direction of a direction focus; in response to the first adjustment, automatically perform a second adjustment to a second portion of the UI to change the other of the visual identification or the coordinate direction of the direction focus; and process the audio signals to filter sounds outside the direction focus, or emphasize sounds within the direction focus.

Abstract: A photographing apparatus and a method thereof includes acquiring a depth map of an image currently captured by the photographing apparatus, if auto focusing (AF) of the photographing apparatus is performed, calculating depth information of an area in which the AF has been performed, and outputting a guide audio having a volume adjusted according to the calculated depth information to inform a user of a state of auto focusing with respect to a desired area without checking a viewfinder.

Abstract: A method for image photographing in an apparatus having a camera, according to the present disclosure is provided. The method includes identifying a face area by focusing on a subject in an automatic shot mode, automatically and continuously shooting the subject if the identified face area is directed to the front toward the camera, setting a best photo by generating and displaying thumbnails of the continuously shot images of the subject, and by displaying an identifier on a thumbnail of at least one image satisfying a predetermined condition, and by displaying an identifier on a thumbnail of image satisfying a predetermined condition, and storing the at least one image satisfying the predetermined condition if storing of images is requested.

Abstract: An operating area determination method and system are provided. In the operating area determination method, a plurality of depth maps of a target scene is generated at several time points. At least two specific depth maps among the depth maps are selected and compared to identify a moving object in the target scene, and a position of the moving object in the target scene is defined as a reference point. A standard point in the target scene is obtained according to the reference point and a specific depth corresponding to the reference point. An effective operating area in the target scene is determined according to the reference point and the standard point for controlling an electronic apparatus.

Abstract: A display control apparatus comprises a recording unit which records a still image and a moving image associated with the still image on a recording medium; a setting unit which sets whether to set the associated moving image as a start image to be displayed first at a start of image reproduction; and a control unit which controls to, when the associated moving image has been set as the start image, display the associated moving image from images recorded on the recording medium at the start of image reproduction, and controls to, when the associated moving image has not been set as the start image, display an image based on another condition irrelevant to whether the image is the associated moving image, from images recorded on the recording medium at the start of image reproduction.

Abstract: The invention concerns a device for multimedia data retrieval, said multimedia data being associated with an active component, said device being characterized in that, depending on an external event, some of the active components trigger an action that make the user aware of the multimedia data associated with said active components.

Abstract: Embodiments of the invention provide an apparatus for noise cancellation of an optical image stabilizer, which includes a motion sensor configured to sense a motion, a microphone configured to acquire a sound, a speaker configured to output a sound, and a controller. The controller is configured to determine whether at least one predetermined condition is satisfied based on at least one signal input from the motion sensor and the microphone, when it is determined that the at least one predetermined condition is satisfied, to periodically operate the optical image stabilizer to store one or more noises generated from an actuator within the optical image stabilizer and driving information of the actuator in a memory, and then, when the optical image stabilizer is normally operated, to acquire the one or more noises corresponding to the driving information of the actuator from the memory, and to generate a canceling noise and to output the generated canceling noise through the speaker.

Abstract: There is provided a server including a reception section which receives, from a client terminal, present position information showing a position of the client terminal, and direction information showing an orientation of the client terminal, a retrieval section which retrieves sensory data to which detection position information is added corresponding to a position in a vicinity of an axial line extending in a direction shown by the direction information from the position of the client terminal, and a transmission section which transmits the sensory data retrieved by the retrieval section to the client terminal.

Abstract: An imaging device includes an imaging unit configured to generate image data, an image data analyzing unit configured to analyze the image data to determine an age group or a sex of an image of a person included in the image data, a voice data generating unit configured to generate voice data, a voice data analyzing unit configured to analyze the voice data, a shooting condition information generating unit configured to generate shooting condition information based on a result of an analysis by the voice data analyzing unit and the age group or the sex of the image of the person determined by the image data analyzing unit, an image data recording unit, and a recording controller configured to record the image data and the shooting condition information in the image data recording unit.

Abstract: A controller controls a first changing unit to intermittently drive an optical unit of an image pick-up unit, and while the image pick-up parameter is being changed, reduces the noise based on an audio signal obtained by a microphone unit in a period before or after a period the optical unit is being driven.