Abstract: Disclosed is an input device and method for interacting with motion pictures incorporating content-based haptic response. Content data relating to the motion picture is stored in a content data storage device while motion picture data is stored in a prerecorded image data storage device. A viewer input device is provided so that a viewer (end-user) can advance and reverse the frames on a display screen while substantially simultaneously, the content data is accessed and braking commands are sent to the viewer input device. The result is that the viewer is provided with haptic responses to the viewer's input through the viewer input device.

Abstract: Arbitrary input sounds are analyzed and the coefficients of a low-dimensional representation, such as LPC or MFCC, are determined as a measure of the timbre of the sounds. The coefficients can be employed in different ways to control output events, such as the generation of synthesized sounds. In one approach, the individual coefficients are mapped to the control parameters of a sound synthesizer, to enable highly complex output sounds to be generated in response to simple input sounds. In another approach, pattern recognition techniques are employed with respect to the coefficients, to classify the input sounds. Each classification is mapped to a control parameter, to cause different events to occur in response to the respective input sounds. In one example, the sounds of different musical instruments are generated in dependence upon the classification of the input sounds. These analysis techniques have low latency, and thereby allow events to be controlled without perceptible delay.

Abstract: A music synthesizer has one or more sensors that generate a respective plurality of sensor signals, at least one of which is an audio frequency sensor signal. Electronic circuitry, such as a specialized circuit or a programmed digital signal processor or other microprocessor, implements a physical model. The electronic circuitry includes an excitation signal input port for continuously receiving the audio frequency sensor signal as well as a control signal port for continuously receiving a control signal corresponding to the audio frequency sensor signal. The-control signal can have much lower bandwidth than the audio frequency sensor signal.

Abstract: A computer input system and method is described which includes a spatial reference member having a body with a first surface and an opposing second surface. The spatial reference member separates a first spatial region including the first surface from a second spatial region including the second surface. A physical object is movable within the first spatial region. Optical indicia tracks the physical object through the body. The optical indicia is observable from the second spatial region. An optical detector positioned in the second spatial region tracks the optical indicia and develops signals which may serve as inputs to a computer. The object may emit a signal which is tracked by the optical detector.

Abstract: An affect-based method of communication between robots is provided by displaying a visual facial expression indicative of a simulated emotional state on a display device of a first robot, and viewing the visual facial expression using a camera on a second robot. The simulated emotional state may be one of happiness, anger, or sadness, for example. The second robot determines the simulated emotional state based upon the visual facial expression. The second robot processes the simulated emotional state to redefine its own simulated emotional state, and to display a visual facial expression indicative thereof. The visual facial expression allows a human observer to discern the simulated emotional state of the robot. Optionally, the robots further communicate affect using audio tones.

Abstract: An attention manager presents information to a person in the vicinity of a display device in a manner that engages at least the peripheral attention of the person. The information is embodied by one or more sets of content data (e.g., video or audio data). Each set of content data is formulated by a content provider and made available for use by content display systems. Upon appropriate activation, each content display system displays images corresponding to the sets of content data in accordance with predetermined scheduling information. The attention manager makes use of "unused capacity" of the display device and the person's attention, providing information to the person that the person might not otherwise expend adequate energy to obtain.

Abstract: A signal mapping system maps sensor signals into control signals that control the operation of a music synthesizer. A "one to many" mapping technique is used, allowing at least some of the sensor signals to each be mapped into numerous music synthesizer control signals. Physical gestures by a user are mapped into a large set of music synthesizer control signals, some of which continuously vary in value as the user moves through the gestures. Signal mapping functions are used to map the sensor signals into note number and velocity values for at least one voice to be generated by the music synthesizer. The note number and velocity values are sent to the music synthesizer as note-on events when predefined note-on and note-off trigger conditions, defined with respect to specified ones of the sensor signals, are satisfied. Other ones of the signal mapping functions are used to generate asynchronous control signals that are sent to the music synthesizer independent of the note-on and note-off events.

Abstract: A personal object or obstacle detector apparatus with enhanced stereo imaging capability includes a sonar system having dual receivers with field lobes positioned based on psychoacoustic considerations such as interaural intensity difference (IID) and interaural time difference (ITD). Receiver field pattern shapes and directions are selected to match the natural IID and ITD associated with human hearing. In one embodiment, dual receivers are used with a common transmitter while another embodiment utilizes two transmitter/receiver pairs with the transmitter and receiver collocated.

Abstract: A video image, characterized by a plurality of pixels, is compressed by deriving at least one parameter based upon principal component analysis. An image value for each video image is then determined for the one parameter. The image value associated with each video image is compared to an image value associated with another video image to determine a difference. Any one of the image values of each video image and another video image is discarded in the event the difference is below a threshold. The remaining video images are then stored. Using this method of compression, it becomes possible to search a video database to find a match between an inquiring video scene with a video database comprising a plurality of video scenes with each video scene having a plurality of video images.

Abstract: A control instrument for a system for generating acoustic output which includes a processor for receiving user input signals and control the acoustic output in response to the user input signals. The control instrument includes an instrument body, and at least one sensor element carried by the instrument body. The sensor element generates user input signals upon tactile actuation of the sensor element by a user. The user input signals indicate the location of contact and amount of force applied to the sensor element by the user. A music synthesis system and a sound processing system each including the control instrument are also provided.

Abstract: The invention enables the display of an image to be paused, then, at the end of the pause, resumed at an accelerated rate until a time at which the content of the display corresponds to the content that would have been displayed had the image been displayed at the normal display rate without the pause, at which time display of the image at the normal display rate resumes. The invention can be used with display systems that display pre-recorded images (such as are found on video or audio cassettes, or video or audio compact discs, for example) or with display systems that display images based upon display data that is only momentarily available to the display system (such as occurs in the display of television or radio broadcasts). The invention can be used with either analog or digital display systems. Further, the invention can be used with any type of image display, such as, for example, audio displays, video displays or audiovisual displays.

Abstract: A system and method for wide angle imaging create a high resolution image using a convex primary mirror concentrically positioned relative to a concave secondary mirror and one or more detectors spherically juxtaposed. The radii of the primary and secondary mirrors are related by the square of the "golden ratio" to reduce low order aberrations. A fiber optic faceplate coupled to each detector corrects field curvature of the image which may then be detected with a conventional flat detector, such as a CCD camera.

Abstract: Two or more microphones are mounted in an environment that contains an equal or lesser number of distinct sound sources. Acoustic energy from each source, with its attendant echoes and reverberation, impinges on each microphone. Using direction-of-arrival information, a first module attempts to extract the original source signals as if the acoustic environment were anechoic. Any residual crosstalk between the channels, which may be caused by echoes and reverberation, is removed by a second module. The first and second modules may be implemented using existing technology.

Abstract: Existing media signals are processed to create new media content by defining content representations for the existing media and establishing functional dependencies between the representations. The content representations constitute different data types which determine the kinds of operations that can be performed and dependencies that can be established. Among the types of transformation that can be achieved are synchronization, substitution resequencing temporal compression and dilation, and the creation of parametric special effects. The content representations and their functional dependencies are combined to construct a functional dependency network which causes the desired transformations to occur on input media signals. The inputs to the functional dependency network are parametrically specified by media data types to construct a template that can be used to create adaptive media productions.

Abstract: A computer input system and method is described which includes a spatial reference member having a body with a first surface and an opposing second surface. The spatial reference member separates a first spatial region including the first surface from a second spatial region including the second surface. A physical object is movable within the first spatial region. Optical indicia tracks the physical object through the body. The optical indicia is observable from the second spatial region. An optical detector positioned in the second spatial region tracks the optical indicia and develops signals which may serve as inputs to a computer. The object may emit a signal which is tracked by the optical detector.

Abstract: An improved music generation system that facilitates artistic expression by non-musician and musician performers in both individual and group performance contexts. Mappings are provided between 1) gestures of a performer as indicated by manipulation of a user input device, 2) displayed motion of a graphic object, and 3) global features of a musical segment. The displayed motions and global features are selected so as to reinforce the appearance of causation between the performer's gestures and the produced musical effects and thereby assist the performer in refining his or her musical expression. The displayed motion is isomorphically coherent with the musical segment in order to achieve the appearance of causation. The global features are segment characteristics perceivable to human listeners. Control at the global feature level in combination with isomorphic visual feedback provides advantages to both non-musicians and musicians in producing artistic effect.

Abstract: A constraint-based graphics system employs different examples of an image to define the constraints of the system. The examples are grouped into subsets which can be interpolated with one another, according to a user-specified input that determines the relative proportion of each example image. An animation can be created by defining a sequence of such interpolated images. Alternatively, a user can directly manipulate observable components of an image to define a particular state for the image. Automatic transformations are defined to provide registration of an image within an overall scene by shifting the frame of reference for an image so that unnatural movements do not occur as the animation proceeds through a sequence of states. The structure of the system permits an animation to be divided into distinct components that can be combined with complementary components of other animations, to provide new results.

Abstract: A hierarchical memory for use in a programmable gate array integrated circuit comprises an interconnect structure having a plurality of interconnect nodes electrically connected in a tree configuration. The interconnect nodes include a root node which receives a multi-bit address word indicative of a selected memory location. The hierarchical memory further includes a plurality of memory cells electrically connected to the interconnect structure to form leaf nodes of the tree. Each of the memory cells contains at least one memory location for storing binary data. The interconnect structure is traversed from the root node to a memory cell containing the selected memory location based upon the multi-bit address word, wherein the interconnect structure provides a communication path for accessing the selected memory location from the root node.

Abstract: The invention enables the apparent display rate of an audiovisual display to be varied. The invention can modify an original set of audio data in accordance with a target display rate, then modify a related original set of video data to conform to the modifications made to the audio data set, such that the modified audio and video data sets are synchronized. When the modified audio and video data sets so produced are used to generate an audiovisual display, the audiovisual display has an apparent display rate that approximates the target display rate. The target display rate can be faster or slower than a normal display rate at which an audiovisual display system generates an audiovisual display from the original sets of audio and video data. The target display rate can be established solely by a user instruction, by analysis of the audiovisual data, or by modification of a user-specified nominal target display rate based upon analysis of the audiovisual data.

Abstract: A method and a system for audio communication between a plurality of users at a plurality of sites utilizes a set of audio input sensors at each site. Each set of audio input sensors binaurally senses an auditory space in proximity thereto. A metaphorical representation of each of the sites is provided. Each metaphorical representation has a position which is variable within a metaphorical space. The metaphorical representations can be based upon, for example, a physical metaphor, a visual metaphor, an auditory metaphor, or a textual metaphor. The auditory space sensed at each site is combined to form at least one synthetic auditory space. The at least one synthetic auditory space is formed in dependence upon the position of each metaphorical representation within the metaphorical space. A binaurally perceivable auditory environment is produced at one or more sites based upon the at least one synthetic auditory space.