Abstract: An imaging device includes: a capturing unit that captures a motion picture at one of a plurality of frame rates; a buffer that temporary stores the motion picture captured by the capturing unit; a signal generation unit that generates a signal used for specifying a core frame that is to be subjected to a resolution enhancement processing, while the capturing unit captures the motion picture at a first frame rate; a capturing control unit that controls the capturing unit to capture the motion picture, after the core frame specified by the signal, at a second frame rate that is higher than the first frame rate; a reference frame determination unit that determines a frame stored in the buffering unit at the second frame rate as a reference frame; and an image processing unit that performs the resolution enhancement processing by converting the core frame into a high-resolution frame by reference to the reference frame determined by the reference frame determination unit.

Abstract: A method and apparatus for reproducing metadata provides a data structure in which metadata whose preferential reproduction is desired can be selected and reproduced. The metadata relevant to a moving image includes a stream data structure including one or more access units each being a data unit which can be independently processed, and each of the access units includes first data to specify an effective period defined with respect to a time axis of the moving image, object area data describing a spatio-temporal region in the moving image, second data including one of or both of data to specify a display method relevant to the spatio-temporal region and data to specify a processing to be performed when the spatio-temporal region is specified, and third data to specify, in a case where one or more access units exist on a same screen in the moving image at a time of reproduction of the metadata, reproduction priority of each of the access units.

Abstract: An image sensor has a plurality of photo detectors each having a spectral sensitivity. An image data input unit inputs image data having a position, a spectral sensitivity, and a measured signal value of each photo detector in an area of the image sensor. A parameter estimation unit estimates each parameter of a plurality of spectral intensity functions using the image data. Each spectral intensity function represents an intensity of light corresponding to a spectrum in the area. An intensity estimation unit estimates the intensity of light in the area using the plurality of spectral intensity functions with each parameter. An output unit outputs the intensity of light for each photo detector.

Abstract: In the case where metadata refers to relevant content on a network, since the relevant content on the network can not be necessarily permanently held, it is desirable that a time limit for making reference can be set. Besides, in order to provide relevant content which can be referred to only in a limited period for promoting sales, or in order to provide relevant content which can be referred to only in a limited period such as, for example, Christmas Day every year for raising the entertainingness of the metadata, it is desirable that a reference time limit of the relevant content can be set. Then, in order to set an effective time limit of an access unit, a flag to specify the effective time limit and the effective time limit are described in each access unit.

Abstract: There is provided a structure of metadata which can express an object of a complicated shape. There are added vcr_shape_num indicating the number of sub-planar areas constituting an object area, vcr_rule_code indicating that an integration system to integrate the sub-planar areas and to determine a final object area is written, vcr_rule_length to indicate the length of data of the integration system, and data vcr_rule to indicate the integration system, and thereafter, vcr_subregion_data of data of each sub-planar area appearing in vcr_rule is described continuously by the number indicated by vcr_shape_num.

Abstract: A super-resolution device and method for setting at least one of a plurality of pixels included in image data as target pixels, the image data including pixels arranged in a screen and pixel values representing brightness, an area including the target pixel and peripheral pixels as a target area, and an area for searching pixel value change patterns in the target pixel area; calculating a difference between a first change pattern and second change pattern; comparing a difference between the first and second change patterns; calculating a pixel value of a super-resolution image having a number of pixels larger than a number of pixels included in the image data on the basis of a decimal-accuracy-vector, an extrapolated vector, and pixel values obtained from the image data.

Abstract: A structure of metadata relevant to a moving image, in which metadata editing is easily performed, comprising a stream data structure including one or more access units each capable of being independently processed; each of said access units has: first data for a reference time point within an access unit time for describing the metadata so as to cause the access unit time to correspond to a moving image time; an object area data describing a spatio-temporal region in the moving image; and second data for specifying a display method relevant to the spatio-temporal region and/or a processing when the spatio-temporal region is designated. The moving image time is to be made to correspond to the access unit time on basis of the reference time point in the first data, so that a period during which at least said second data may be called is specified.

Abstract: An apparatus includes unit acquiring, from an image source, an image represented by pixel values indicating brightness levels, unit setting, as a reference frame, one frame, unit sequentially setting, to be target pixels, each of pixels in one or more frames, unit setting target-image regions including the target pixels, unit searching the reference frame for similar-target-image regions similar to each of the target-image regions in a change pattern of pixel values, unit selecting, from the similar-target-image regions, corresponding points corresponding to each of the target pixels, unit setting sample values concerning brightness at the corresponding points to a pixel value of a target pixel corresponding to the corresponding points, and unit computing pixel values in a high-resolution image corresponding to the reference frame, based on the sample values and the corresponding points, the high-resolution image containing a larger number of pixels than pixels in the reference frame.

Abstract: An image resolution increasing method includes reducing an input image, calculating a first feature vector having a feature quantity of a first block of a reduced image, extracting a high-frequency component image from the input image, storing pairs each having the first feature vector and a second block of the high-frequency component image that is located at the same position as the first block as a look-up table, enlarging the input image, calculating a second feature vector having a feature quantity of a third block of an object in the input image, searching the look-up table for the first feature vector similar to the second feature vector, and adding a fourth block of the look-up table which pairs with the first feature vector and a fifth block of the temporal enlarged image that is located at the same position as the third block.

Abstract: A resolution enhancing method of a video includes reducing a training video, extracting a high-frequency component from the training video, calculating a first feature vector including a feature amount of a first spatio-temporal box in the reduced video, storing pairs of the first feature vectors and second spatio-temporal boxes in the high-frequency component videos at the same positions as those of the first spatio-temporal boxes, expanding an input video, retrieving a first feature vector similar to a second feature vector including a feature amount of a third spatio-temporal box of an object of the input video to be processed, as an element, and adding a second spatio-temporal box making a pair with the retrieved first feature vector to a fourth spatio-temporal box in the expanded video at the same position as that of the third spatio-temporal box in order to generate an output video.

Abstract: An imaging device includes: a capturing unit that captures a motion picture at one of a plurality of frame rates; a buffer that temporary stores the motion picture captured by the capturing unit; a signal generation unit that generates a signal used for specifying a core frame that is to be subjected to a resolution enhancement processing, while the capturing unit captures the motion picture at a first frame rate; a capturing control unit that controls the capturing unit to capture the motion picture, after the core frame specified by the signal, at a second frame rate that is higher than the first frame rate; a reference frame determination unit that determines a frame stored in the buffering unit at the second frame rate as a reference frame; and an image processing unit that performs the resolution enhancement processing by converting the core frame into a high-resolution frame by reference to the reference frame determined by the reference frame determination unit.

Abstract: There is provided a structure of metadata in which an operation at a time of object designation can be changed according to the kind of an input interface used for input or a method of an input operation. The metadata relevant to a moving image has a stream data structure including one or more access units each being a data unit which can be independently processed, and each of the access units includes first data to specify an effective period defined with respect to a time axis of the moving image, object area data describing a spatio-temporal region in the moving image, plural second data each specifying processing to be performed when the spatio-temporal region is designated, and third data to cause the respective second data to correspond to kinds of input interfaces or operation methods by which a user designates the object.

Abstract: In order to assist a viewer in understanding the contents of a moving image, it is desired to plainly represent the contents of metadata to the viewer. The metadata relevant to the moving image has a stream data structure including one or more access units each being a data unit which can be independently processed. Each of the access units includes first data to specify an effective period defined with respect to a time axis of the moving image, object area data describing a spatio-temporal region in the moving image, and balloon data to display relevant data of the object by a balloon. The balloon data includes text data to be displayed in the inside of the balloon and to express the contents of the object and position specifying data to specify a display position of the balloon.

Abstract: For a reproduction device in which a usable memory size has an upper limit, this invention discloses a novel structure of metadata, which can suppress the number of seeks of a CD or a DVD occurring at a time of reproduction of metadata of a moving image including a large number of scripts and hyper texts (HTML) each having a small data size, and the number of times of file request transmission/reception through a large number of networks. In the metadata of the moving image, in the case where access units AU#2 and AU#3 require EF#1, and access units AU#2 to AU#4 require EF#2, the EF#1 and the EF#2 are embedded in front of the AU#2, and a script in the AU uses a file name, a file identifier, or a virtual buffer offset in order to specify the EF to which reference is made. Embedding of a file and identification of the file are realized by causing the EF and the AU to have file identification information respectively and by specifying the file identification information from the script.

Abstract: There is provided an invention in which in a moving image hypermedia including moving image data and its metadata, when a viewer specifies an object, it is easy to confirm that the specification has been successfully made, and the specified object can be easily confirmed. Metadata relevant to a moving image has a stream data structure including one or more access units each being a data unit which can be independently processed, and each of the access units includes first data to specify an effective period defined with respect to a time axis of the moving image, object area data describing a spatio-temporal region in the moving image, second data to specify a processing to be performed when the spatio-temporal region is specified, third data including a display method for causing the spatio-temporal region to be recognized, and fourth data to specify, in a case where an event relevant to the spatio-temporal region is generated by a cursor operated by a user, calling the third data.

Abstract: Although it is necessary to display a mouse cursor in order to specify an object area in a moving image, there is a fear that a standard cursor spoils the atmosphere of the moving image or the intention of a producer. Then, a data structure is desired in which the shape of the cursor can be freely specified according to the contents of the moving image or the object. In metadata of the moving image, the object area appearing in the moving image is divided into plural data (access units), and the access unit is made to be capable of specifying the cursor shape, and the cursor shape in the object area is made freely changeable.

Abstract: Since there is a case where reproduction of metadata is limited according to the processing power of a reproduction apparatus or the designation from a user, the invention provides a data structure in which metadata whose preferential reproduction is desired can be selected and reproduced.

Abstract: In preparation for a case where a user indicates an area where an object does not exist, it is desired that an action for that case is also described in metadata. Then, layer value data indicating order of overlap of object area data on a screen or discrimination data intrinsic to an object area is described in each of access units. The access unit in which the layer value data or the discrimination data is a specific value, corresponds to “area where an object does not exist” at that time.

Abstract: There is provided a structure of metadata which can express an object of a complicated shape. There are added vcr_shape_num indicating the number of sub-planar areas constituting an object area, vcr_rule_code indicating that an integration system to integrate the sub-planar areas and to determine a final object area is written, vcr_rule_length to indicate the length of data of the integration system, and data vcr_rule to indicate the integration system, and thereafter, vcr_subregion_data of data of each sub-planar area appearing in vcr_rule is described continuously by the number indicated by vcr_shape_num.

Abstract: In the case where metadata refers to relevant content on a network, since the relevant content on the network can not be necessarily permanently held, it is desirable that a time limit for making reference can be set. Besides, in order to provide relevant content which can be referred to only in a limited period for promoting sales, or in order to provide relevant content which can be referred to only in a limited period such as, for example, Christmas Day every year for raising the entertainingness of the metadata, it is desirable that a reference time limit of the relevant content can be set. Then, in order to set an effective time limit of an access unit, a flag to specify the effective time limit and the effective time limit are described in each access unit.