Music Programming

An Introductory Essay (1988)


Chris Dobrian

What is a computer program?

A program is simply a set of instructions given to a computer to carry out a certain task or tasks.

The instructions are typed into the computer by a programmer in a special language that the computer will understand. The language may be a numerical code which tells the computer fairly directly which circuits to turn on and off (this is termed a "low level" language), or the language may closely resemble a normal human language which will not directly control the computer's operation, but will be translated into low level terms and then to binary code that the computer can understand. A programming language such as Basic or HyperTalk is a "high level" language which is interpreted or evaluated by other programs and is thus translated into low level instructions.

The user of the program is usually unaware of the precise instructions, and knows only the result of using the program. The user sets the program into action (by pushing a button, clicking a mouse, typing a command, etc.). The program may require the user to supply information necessary to perform its task; or the program may give the user choices as to exactly how the task is to be performed.

Another term for a program is a "routine". Yet another nearly synonymous word is algorithm. An algorithm is any set of rules by which an input is transformed into a different output. Here are three examples of algorithms contained in a single sentence:

  1. "When the starting gun is fired, run forward as fast as you can until you cross the finish line."
  2. "When the command 'FtoC' is received, ask 'What is the temperature in degrees Fahrenheit?', then wait until a number is received, subtract 32 from the number, then multiply the result by 5, then divide the result by 9, then print out 'Here is the temperature in degrees Celsius: ', then print out the result."
  3. "When a key is pressed down, play the corresponding note until either the key is released or the sustain pedal is in the up position, whichever comes later."

The first example is an algorithm of extreme simplicity. The firing of the starting gun is the input (the starting command), the running of the race is the output (and the crossing of the finish line is the stopping command). A robot could perform this algorithm with only three abilities: the ability to detect the firing of the starting gun, the ability to run forward, and the ability to detect the crossing of the finish line.

Of course, the simplicity of this algorithm also limits its usefulness. What if the track is curved, for example? The robot would need a way to detect the angle of the curve, and alter its direction and its speed appropriately. Another thing to note is that once the algorithm/robot is launched by the user (the one giving the starting command), its behavior is in no way modifiable until it reaches the finish line, since the user cannot interact with the algorithm in any way.

Oversimplicity and lack of interaction are problems of primary interest to authors of algorithms. Consider, for example, the grave consequences of the lack of interaction in the following similar algorithm: "When the button is pressed, fire the missile at Moscow." The absence of a command to abort the flight of the missile is one of the many problems with this algorithm.

Indeed, it might be argued that the first example is not an algorithm at all, since it does not actually transform its input. The input (the firing of the starting gun) is merely a command to start other processes. This seems to be the main distinction between an algorithm and any other type of program; an algorithm transforms input information into a different output, according to some set of (possibly modifiable) rules.

The second example is an algorithm for transforming Fahrenheit temperature into Celsius temperature. This algorithm is fairly clever in that when it receives its starting command, it asks the user for the input information that it needs. Its input can be virtually any number, which will always be transformed according to exactly the same arithmetic rules. Thus, the output may be different, depending on the input, but the relation between the two will always be the same. However, even though this algorithm seems to converse with its user, it is not dynamically interactive because the user has no way to influence its main operation; the rules and the operations remain exactly the same once the input is received.

Algorithms or other programs which have a set operation regardless of external forces are termed black box algorithms. This is because the "inner workings" are shielded from the user (and vice versa) much like a black box with nothing on the outside except an input jack and an output jack. The user may know what kind of output will be obtained with different inputs, but the algorithm is neither reprogrammable nor affected by external forces.

The third example transforms the input, the depression of a key on a synthesizer, into a sonic output. This algorithm has an interactive component in that it's performance depends on two user controls: the key and the sustain pedal. Thus the algorithm's performance, specifically the duration of the note, can be different every time it is invoked, controlled at will by the user.

From these examples it is evident that many things work according to a routine, a set of rules: a piano or other keyboard mechanism, a runner, a computer, and many other aspects of human and mechanical behavior that we may not even be aware of. In the case of a computer, the challenge for the programmer is to state a set of rules concisely and completely in the language (whether high or low level) which will be understood by the computer. This involves two interrelated steps: determining the rules which govern a desired behavior, and stating those rules in the appropriate language.

A program may perform certain activities without needing any input information, or it may require the input of certain information in order to govern its activities, or it may transform input data into different output data, or it may be dynamically alterable by data input during the course of its operations. Among the activites it may perform is the activity of starting other programs! Or providing data to other programs, or algorithmically transforming the data it receives from other programs which it started...etc. Thus a program may interact with other programs or even contain other programs of its own (known as "subroutines" or "functions") which it will operate as if it were the user.

Here is a simple human example of a function: Suppose a woman has the job of underlining the number "28" every time it occurs in a column of numbers. In short, her job consists of two repeated tasks: finding the number "28" and underlining it. However, she doesn't want to have to consciously think what method she uses to find and underline. Nor would a programmer want to rewrite these instructions over and over. Therefore, repeated tasks are often stored in programs (and quite possibly in the brain, as well) as "subroutines" called on by the main program. So a function might be created and called "find it" which performs the following tasks: "To find the number '28' look at the first digit of a number; if it is not '2', go directly on to the next number, but if it is '2', then look at the second digit of the number; if it is '8' and there are no other digits, then conclude that the number is '28' and say 'Found it.'". Similarly, a function might be created called "underline it": "To underline the number put the pencil at the bottom left corner of the number and draw a straight line to the bottom right corner of the number." (In fact, the "underline it" function might even call on another function, "draw a straight line" to perform its task.) Once the knowledge of how to "find it" and "underline it" is stored away in some readily accessible part of the memory, all the woman or the program needs to know is "Start at the beginning, 'find it' and 'underline it' until the end." Whenever "find it" or "underline it" are encountered, the auxiliary (more detailed) function takes over until it has completed its work, then the main program takes over again.

Functions are made more generally useful if they can perform different, or at least varying, tasks. In the above example "find it" finds the number "28". But it would be more useful if it could find any number required of it. Suppose we make a new function "find this ()" which performs the task: "Look at the number, and if it's equal to whatever is inside the parentheses, say 'Found it,' otherwise go to the next number and try again." Then if we said "find this (29)" the function would find "29" because it is what's inside the parentheses. The input information is known as an argument to the function. In some programming languages there is virtually no distinction between "programs" and "functions" or between "input data" and "arguments".

One thing that computer programs and people often need to do is make a test in order to gain information. For, example, in order to boil water the stove must be turned on, but there's no point in turning it on if it's already on. So in order to know what to do, one may first need run a test to find the answer to the question "Is the stove on?" By testing the position of the knob ("Is the knob pointing to 'off'?") or other means such as testing the color or temperature of the burner, one can answer the earlier test "Is the stove on?" The answer to that test will in turn tell whether to turn it on.

Computers, because of the binary nature of their operation, are best at answering test questions which have a "yes or no" or "true or false" answer like "Is the stove on?" This is because eventually such a question can be boiled down, at a lower level of the computer's operation, into "Is the electrical circuit 'open' or 'closed'?" However, many questions which don't seem to have a "yes or no" answer can be solved using such tests. "I'm thinking of a number between 1 and 100. What is it?" is not a "yes or no" question, but it can be answered by running "yes or no" tests. One way of testing is to ask, "Is it 1?", "Is it 2?", etc. until a "yes" answer is received. But this is a very inefficient way to test, because on the average it would require about 50 tests in order to answer the original question. A much more efficient method is to run tests like "Is it less than 50?", then "Is it less than 25?", etc.

Conversely, some apparently simple "yes or no" questions are extremely difficult to test. For example, "Is it a good piece of music?" may require an enormous number of tests and subtests (even if one knows which tests to run) before the original question can be answered. Of course some may feel that the answer to this question can be found by running one simple test, "Do I like it?" These people would generally make efficient programmers.

Why use computer programs to make music?

Although we may not always be aware of the fact, we use tests, algorithms, and routines in our thinking constantly. Often the use of an extremely convoluted predetermined algorithm makes our life easier and more productive than constantly devising new solutions to problems. Here's an example: A pianist is sightreading and sees the four-note chord C-E-G-B. What she does not and cannot do, if the tempo of the music is at all quick, is read each of the four notes, find each one on the keyboard, and determine the appropriate finger with which to depress the keys.

Here is a highly unscientific approximation of what might actually pass quickly through her mind: "A four-note stack of thirds starting on C with no accidentals. [This information comes from a visual test. The location of the bottom note is known to be C, and the formation of the dots is stored in memory as a stack of thirds or a seventh chord. The algorithm is something like "If it's a four-note stack of thirds it is a seventh chord, if there are no flats or sharps it's all on white keys, if it starts on C it's a C major seventh chord, if it's a C major seventh chord my fingers should form themselves to play such a chord." The algorithm involves subroutines developed by the study of music theory and the practice of piano technique.] I feel no black key to the left of my hand. [The test being "If I feel no black key to the left of my hand, then the leftmost finger of my hand is on either C or F," derived from a knowledge of the keyboard layout.] I just played a C with the leftmost finger of my hand. [Therefore it must be C, not F.]" In fact, if the knowledge of what was just played were more prominent in the pianist's mind than the feeling in her hands, her mind may never even get around to caring whether she feels a black key to the left of her hand. She would just keep her hand in place over the C she had just played and form a C major seventh chord.

By using algorithms she has transformed the input data of four black dots, having the significance "play the notes C, E, G, and B simultaneously", into "keep your hand in place and configure your hand for a C major seventh chord". This is much more productive than transforming the input data into "find the note C, find the note E, find the note G, find the note B, figure out how to play all four at once". Practice and study have developed "functions" in her mind which increase its productivity.

Presumably, if one can define and program useful functions used by composers and instrumentalists, one can enhance their work with the use of a computer.

By extension, by defining and programming new functions (as opposed to merely imitating functions which humans already perform) one may enhance the composer's or instrumentalist's operations in ways previously unheard of, actually expanding the number of abilities at that person's disposal. This is exciting when you stop to think how much of what is considered musical is based on what humans can physically achieve. When such limitations are overcome, the realm of what is considered musical may be vastly enlarged.

By defining his or her intentions in a very strict language for a computer, a composer or performer is required to specify precisely the methods used. This may result in a better awareness of one's own behavior, what is essential to one's work, and what is illusory or superfluous. Here again, by defining and programming new functions (new ways of composing, new ways of interpreting a performer's gestures) entirely new musical experiences may be discovered.

Furthermore, because analog sound waves can be expressed in terms of numerical samples, mathematical functions and other types of numerical processing can be fertile ground for actually synthesizing new sounds.

With relatively limited resources--a personal computer and a MIDI instrument (and any other generator of MIDI data, if desired)--one can perform any kind of algorithmic processing on any data sent to the sound generator of the MIDI instrument. This includes all aspects of the sound over which the instrument provides control (usually pitch, loudness, timbre, modulation, envelope, even spatial location) as well as musical aspects which the program may also provide such as rhythm, delay, reordering of pitches, or any other processing of the data imaginable). The data may originate from a computer program, or it may originate from some other MIDI controller and be processed by the computer program. Thus data may be provided by a composer and/or by a performer.

Computer processing of data which occurs so fast as to cause no perceptible delay in the transmission of the data to the instrument is called realtime processing. Efficient programming at all levels is vital in order to prevent the computer from doing unnecessary work which may slow it down. For this reason, if for no other, simple algorithms are generally advisable.

Why not use computer programs to make music?

Art is, almost by definition, ambiguous and vague. It is frequently extremely complex, defying definition in the simple algorithmic terms referred to earlier. As Vinko Globokar once remarked, with only slight exaggeration, "Anything which can be taught about composition is necessarily too simplistic to be of value."

Sound synthesists and music programmers often add complexity to algorithms in an effort to make their effect more interesting. Unfortunately, even complexity can be simplistic, by being predictable, so unpredictable as to be unengaging, or simply by being, for whatever reason, uninteresting. It is known that one of the main reasons that the sound of an acoustic instrument like the flute is so attractive is because the sound contains a vast and complex variety of subtle noises and variations which go virtually (but not totally!) undetected by our ears, in addition to the more obvious noises and variations that we do detect. Synthesists have tried to add noise, jitter, vibrato, pitch inflection, etc. to synthesized sounds in an effort to simulate the complexity of acoustic sounds. Unfortunately, for the most part, the precise nature of the variations which make acoustic sounds rich and attractive remains undiscovered. One is tempted to suspect that these "microscopic" but all-important aspects of acoustic sound are so complex, or complex in such an unusual way, as to be undefinable in precise terms.

Similarly, the relationship between a virtuoso musician and his or her instrument is replete with an overwhelming amount and degree of nuance, built up over years of intense listening and practicing. So much of the nuance of the instrumentalist/instrument relationship is developed without being linguistically defined, by listening, imitating, feeling. Furthermore, it seems to be largely stored not in some intellectually explicable way, but in laboriously developed muscular reflexes, almost as if the brain is entirely bypassed. In playing a single brief note, a violinist combines bow angle, bow speed, bow pressure, bow placement, bow attack, finger placement, finger movement, finger pressure, in addition to whatever totally involuntary muscular movements may be caused by nervousness, coffee consumption, humidity, unknown electrical discharges in the brain, etc.--and all of these factors are changing from millisecond to millisecond, (more correctly, probably much, much faster than that) modified by the brain in interactive response to the sound being produced, the sound others are producing, the acoustics of the room, etc. Is it any wonder, then, that virtuoso performers of acoustic instruments are dismayed by the lack of response of a synthesizer (the sound of which is already vastly inferior to his or her ear) when he or she must manipulate with the foot (one of the less sensitive bodily extensions) a pedal which has only 128 possible gradations, and the resulting effect is (for example) a simplistic strictly regular vibrato of a low pass filter.

Efforts by programmers to add complexity to the instrumentalist/instrument interaction in the case of synthesizers has suffered a fate similar to that of synthesists' efforts to add complexity to sound materials. The complexities are so numerous, undefined, interconnected, and depend on so many variable, constantly changing factors, that an attempt to reproduce them is invariably simplistic. When a large amount of complexity is introduced, it may exceed the ability of the instrumentalist to control it (since such control was previously largely subconscious), it may be complexity which does not actually add to the performer's expressive or musical control, and it may simply be complexity which is not sonically or musically engaging because of aesthetic taste.

Similarly, the relationship between a composer (or improviser) and the musical materials he or she manipulates lies largely in the difficult-to-express-in-language domains of aesthetics, "taste", style, and imagination. The difficulty of expressing the "why" and "how" of even a very simple passage of music is stupefying. For efforts to answer even the simple question "Why one pitch and not another?" witness the thousands of books on harmony, counterpoint, composition, and analysis. There are, quite literally, as many ways of composing music as there are composers. Apparent similarities (to say nothing of universalities) of style and method are often superficial or even illusory. What is more, a single composer may use widely divergent styles and methods, even within a single work.

Thus, to define an algorithm of general utility to composers is more difficult than to define an algorithm of utility to one composer. And it is more difficult to define a composer's general methods or intentions than it is to define isolated aspects of his or her music. In the case of all but the most systematic of composers, algorithmic description of composition will require explicitly stating that which is normally left unstated, that which is performed according to unstated, or even unknown, rules. If one subscribes to Globokar's statement, such description will involve so much simplification as to diminish the quality of the resultant music or be of virtually no help to a composer. What's more, with regard to compositional algorithms of "general" utility, one may argue that virtually no aspect of music is common to all cultures, or even to different periods of the same culture, and therefore no algorithm could nor should attempt to be "generally useful".

Use computer programs to make music?

When new technology is introduced to society, society generally takes a certain time to develop the use of it fully. (Many would argue, for example, that the full potential of television has never been remotely approached, although its usage is certainly more developed than when it first appeared.) This time lag is all the more pronounced with new technology so general in purpose as the computer, which is programmable to perform an almost unlimited variety of tasks.

The first inclination is to use the new technology to duplicate already existent functions (test scoring, for example). This may be in order to demonstrate the usefulness of the technology, or it may be to eliminate the traditional (perhaps tedious, dangerous, or otherwise undesirable) method of performing the function. The second way of using technology is to perform previously unperformable but desired functions (telecommunication, for example). A third, less frequent, use of technology is to discover new, previously unconceived functions. For example, the idea of performing surgery without incision, by reflecting concentrated beams of light through fine, flexible cylinders inserted through an orifice in the body, would likely never have existed without the prior invention of lasers and fiberoptics.

So far, a large amount of the work done in computer music has involved the first way of using technology, trying to make computers behave similarly to humans. In the making of music, however, the only activities which could really be termed tedious are technical instrumental practice (scales, arpeggi, etc.) and music copying. While it is unlikely that computers will help people become virtuosi without practicing (although the possibility may one day warrant consideration), many admirable attempts have been made to reduce the tedium, and to improve the speed and quality, of music copying. Attempts to duplicate other aspects of human musicmaking have proven less successful, and are, in any case, rather pointless since the other aspects of musicmaking--composing, rehearsing, interpreting, improvising, listening--are mostly enjoyable human activities.

Some excellent work has also been done with the second use of technology, enlarging the capabilities of musicians. MIDI sequencers enable a recording artist to correct wrong notes, speed up the performance, etc. By complex calculations performable only by computers, one can give the illusion of recorded sound flying about through space--an idea dreamed of by the revolutionary thinker Edgard Varèse long before the development of modern computers. Other composers have used computers to realize their conception of music unperformable by humans or as a tool to develop compositional ideas which would require amounts of calculation unthinkable without the use of a computer. A composer who imagines such novel music, and feels that it can be defined, or better understood, using an algorithm, may be well advised to write a computer program to test or realize the imagined music.

It is worth considering that music, however essentially inexplicable it may be to us, invariably uses (or at least evokes) linguistic elements of some sort, and may even be a language. (Contrary to the common cliché, however, it is not a "universal language", any more than the sum total of all vocal sounds can be said to be a universal spoken language. It is extremely dependent on aesthetics, tastes, and other explicit and implicit cultural associations, all of which are in turn dependent on society and the individual. Even though media and telecommunications are increasing the awareness of music of other cultures, most individuals are still no closer to knowing all music than they are to knowing all languages.) Therefore, it is interesting to consider the relationship between music and languages, including programming languages. Whether musical phenomena can be or should be expressed in terms of computer programs depends entirely on the musical intentions and desires of each individual.

It would certainly be a mistake to discard the enormous potential of computer programs in musicmaking simply because of "technophobia" or a blanket assumption that a machine cannot perform musical tasks. These are obsolete thoughts, disproven by the facts. Lingering romanticism in thinking about artistic endeavors makes it unacceptable to believe that an artwork rich in meaning can be reduced to an array of "yes/no", "on/off", "0/1" answers. Yet we have already been amazed by the number of complex activities which can indeed be so expressed. The Beethoven string quartet coming from your compact disc player is just a bunch of 0's and 1's.

The third use of technology, to inspire discovery of previously unconceived ideas and functions, is no less possible in music than in any other field. As has been pointed out, the concept of what is "musical" is largely limited by what is "possible". As performers' instrumental technique has expanded, so has musical taste widened to accept it. In fact, musical technique has in turn expanded at times as a result of some revolutionary composer's demands. Technology (possiblilty) and discovery (imagination) have a symbiotic relationship. Certainly, the use of computers to expand our abilities and realize our dreams--or even to create brand new abilities and dreams--is the more forward-thinking, productive, and desirable direction for technology in music. Creative endeavor, just like scientific endeavor, requires careful consideration of what goals are to be pursued.

Christopher Dobrian