MCM

Motion Capture Music (MCM) was originally proposed as a student project in Information and Computer Science. The idea was to make software that would allow one to make musical use of the data from UCI's Vicon 8 motion capture system.

The data generated by the Vicon 8-camera motion capture system is normally intended for use in 3D animation and in biomechanics studies. With the introduction of the Vicon RT realtime capability, however, the Vicon system has potential for realtime control of music. MCM is intended to be a first step in that direction.

The Vicon system captures three-dimensional information about the precise location of points on a body in a space surrounded by eight cameras outfitted with stroboscopic lights. Special markers attached to the body reflect the flashing light, and the images from the eight cameras are used to derive x,y,z Cartesian coordinates for each marker. (A typical configuration might use thirty or more markers on the body, tracked by the cameras at a rate of 30, 60, or 120 frames per second.) The frames of x,y,z data for the markers can be stored in a file (for future use as input to an animation program or for motion analysis), or, with the Vicon RT system, can be sent directly to CG software to control an animation.

MCM allows a user to specify which coordinates and markers are of interest, and how the values should be mapped into music control data in the MIDI format. Once the user has made these specifications, MCM receives data from a Vicon realtime system (or Vicon's RTEmulator software), translates it as indicated, and transmits the appropriate MIDI data, which can be used for direct control of a synthesizer or which can be used as input to some other program for further processing.

The initial design document for MCM (available here as a PDF document) was written by UCI students Cayci Suitt and Gene Wie, under the supervision of Information and Computer Science professor Andre van der Hoek, according to specifications by Music professor Christopher Dobrian. The software implementation was begun by Suitt and Wie, and is being continued by two other UCI students, Mark Magpayo and Maybelle Tan. The Vicon system itself uses the Windows NT/2000 operating system, and MCM is also intended for use in Windows, but is being made as platform-neutral as possible in order that it be easily portable to other operating systems as may be needed in the future. For that reason, the user interface is written in Java and the data-mapping portion is written in C++. The Vicon RT engine acts as a TCP/IP server, so it can communicate with practically any operating system.

A version of MCM for Macintosh, called MCMMax, has been written by Dobrian and researcher Frederic Bevilacqua using the Max/MSP/Jitter programming environment. Because Max/MSP/Jitter already has support for MIDI, audio synthesis, and video processing, Vicon data can be directly applied to any of these media with ease. You can read more details about MCMMax, and see a few brief video examples of some basic mappings of mocap data to MIDI and MSP.