Abstract: A method and apparatus for the control and manipulation of a virtual environment (such as virtual objects therein) in response to a music signal. The music is either interpreted directly to effect the control and manipulation. Alternatively, a control track corresponding to an audio signal (such as a music signal) is prerecorded, played back with the audio signal, and the control track is processed to control and manipulate the virtual world (or to control some other process of a computer system) as the audio signal is playing. In preferred embodiments, a computer creating a virtual world interprets the music, the control track, or both, and uses the resulting information to modify, create, and or control objects in the virtual environment. Preferred embodiments of the inventive system include apparatus for delaying input music to compensate for lag introduced by the system components, such as delay required to implement processing of control tracks corresponding to the input music.