Robert Gauldie

Art to Music: Fourteen Songs for the Stations of the Cross Voiced by Computer

Art and music are objects of man’s utmost diligence and industry. We are awash with art and music. At one level, Western culture could be accurately described as the culture of continuous exposure to music and image.
                Words and writing, another level of Western culture, are a something that exists as a link between an image and a sound. When we invented words they were to express, and to attach, a meaning to images and sounds. If, as it is claimed, the letter A is derived from a pictograph of the Auroch, then perhaps the letter A once stood for sound like Au or Awe that was similar to the sound made by the Auroch. The letter A is today but a building block code used to make many words in which the visual letter A is far more significant than the sound of the letter A and any connotation of A with Auroch is entirely lost outside of philology.
                When we “write” music we express sound in another idealized notation that represents sounds as notes, or combinations of notes, which are sustained over a certain period of time. Images, words and music are all inter-related in their intent, their purpose: even if they all evolved into different notations, or different coded marks. Transliteration between words and music should be possible, and in some cases (i.e. onomatopoeia) remains so. Our modern notations, however, for art, words and music are each highly evolved and specialised to the point where our words, images and sound-combinations have become not only codified in themselves, but their code is also expressed in yet another coded form. A simple, onomatopoeic transliteration between art and music is no longer possible. The brain now translates between image and music via the word-code.
                The relationships between sound, word and image are a human relationship. The physiology of the complete relationship is unknown, although the physiologies of some of the parts are well understood. Sound is received as a disturbance in the air. The inner ear turns the sound into its component frequencies and transmits this sorted frequency package to the brain. The brain recognizes the sorted package as music, sound or noise, but how it does so is unknown. The brain can not only understand the sorted package, but also the brain can make up words that specify the kind of sound that enters the ear. The eye sees objects. We know that what the eye sees is also turned into frequencies because frequencies are the only message format that nerves can transmit. Our nervous system is frequency modulated, which means that our inner perceptions are all in coded form. How the brain handles all of this coded information relating to sight and sound and decodes it into another code that creates words is completely unknown. We can, however, be reasonably sure that the method is fairly simple in principle but has been amplified by brute force, the relentless pressure of evolution. I say this because the simplest of invertebrates have surprisingly complex behaviors. For example, Erythropsis cornuta is a single-celled dinoflagellate that has a large ocellus, a light sensitive organ, complete with a lens. Such dinoflagellates use their light organs to direct complex behaviour yet they have no nervous system at all. For vertebrates like ourselves, seeing and hearing are very ancient physiological functions. Haikouichthys is a chordate from 550 million years ago that has eyes and inner ears. Most invertebrate phyla are not as old as Haikouichthys.
                Physiologically, at least in the broad sense, I can say that our capacity for words sits between our capacity for sound, and so music; and an image, and so art. Words, then, are the something in between sound and image that produces a coded response to both art and music that makes two different inputs user compatible and user comparable. Thus, an internally generated code is used to link the coded information that the brain receives from both the eye and the ear. The transliteration that occurs in the brain is, thereby, a transposition of two coded messages from the eye and from the ear.
                I am going to step out of the context of the brain’s functions into the functions offered by the computer. I will use the computer to divide an image into a sequence of time signatures. Each time signature will be read as a package of colour frequencies. The package of colour frequencies from each time signature will be played as sound by a synthesizer that I will instruct in various ways. Thus, each time signature may range from a sustained note when the colour frequency is constant over the time signature, to a chord when the colour frequency is complex over the time signature. In effect, I am substituting the computerized transformation of image to frequency, and frequency to sound, for some part of the transposition by the brain of the two coded messages received by the eye and the ear.
                Given this background, I need to consider what effect the sounds that I make using the computer will have on the listener. The transliteration that occurs in the brain is from the sound, to a code-complex, which is linked to whatever it is in the brain that controls our emotion. To consider the effects of the sounds that I make, I need to examine the composition of the sounds produced, both in the sense of composition as content and composition as the act of creating content.
                Composition in the artistic sense, whether image or sound, proceeds in certain ways. Reginald Brindle-Smith (Musical Composition, Oxford. 1986; pp. 7-8) has some pertinent comments to make about composition in general and musical composition in particular in the following two quotes:
I would say that any branch of art—be it music, painting, sculpture, literature, drama or poetry—which abandons formal principles to the point of formlessness is doomed to failure: form allows a sense of completeness in a work of art, one that can be grasped by us and held in our memories. The fact that much modern art is quite unmemorable—and modern music particularly so—is perhaps a result of formal weakness, and is certainly the reason for its perishable quality.

• • • • • • •

The real importance of musical form is of a psychological nature: music is an emotive message; and good form ensures that the message is convincing, unified and complete.

                Form and continuity are the bare minimum required to distinguish music from sound. Using the computer to give voice to a painting raises the issue of pre-adapted musical form. I could, for example, take a well-known piece of music, such as a Bach fugue, and use a computer-derived modulation of the Bach fugue that corresponded to frequency patterns derived by a common algorithm used on each painting. In this context, Bach provides the music and the input of the painting via the computer is simply a modulation, a change in pitch of a known and still recognizable form of music. If I chose to use only the painting, without a prior musical form, I need to decide on the internal musical structure of the sound that will be created. Will it have form, and therefore, be music? Or, will it be formless, and therefore, just a type of noise?
                The minimum musical effect from a painting would be achieved scanning the painting pixel by pixel and playing a note corresponding to a colour-to-note look-up table. Even at this minimal level, a note has many connotations. Typically, the means by which the note is produced confers certain qualities above simple pitch. The same note played on different instruments retains qualities typical of the instrument. When the computer synthesizes a note is it to be a pure tone, or piano note, or a violin note? In addition, music sounded note by note can be rather tiresome. Instead of pixel-by-pixel, note-by-note, I can sound a whole line of scanned pixels as a chord.
                Chords have a higher music-emotion content then notes. A piece of music is often an arrangement of successive single notes and chords that is designed to produce a certain emotional response. I could, therefore, instruct the computer to translate an image according to a pre-arranged pattern of notes and chords that is designed to produce a certain type of emotional appeal. This approach, however, like the Bach fugue modulation, minimizes the direct contribution of the painting to the computer-produced sense of music.
                I have decided to use the fourteen Stations of the Cross that I painted for a Wellington Church, St. Andrew’s on the Terrace, in 1987 for the images to be voiced by computer. My reasons for choosing the Stations of the Cross at St. Andrew’s on the Terrace are, firstly, that I like the paintings. Secondly, I painted them; any distempers that my voicings may produce will not offend the artist. In balancing the form, continuity and emotional appeal of the computer produced voicing for the Stations of the Cross I have decided on the following format.
                Firstly, I will follow a chord-like format, sounding the net frequency of the colours of each line as a chord. A line will extend from the top to the bottom, reading from left to right. This is how the western-adapted eye-brain reads images.
                Secondly, the tempo of each chord will be constant from painting to painting. The choice of tempo is arbitrary, being that which is pleasing to my ear.
                Thirdly, the voicing of the synthesizer will be set constant for all fourteen paintings. I will use the QuickTime synthesizer because of its easy accessibility. I will use the software SoundOfAnImage to generate MIDI files for sound. MIDI files can be played back on a wide range of programs.
                SoundOfAnImage is a low-input image to sound translator. There are more sophisticated translators available. As sophistication increases, so does the input command structure become more complex. Increasing the complexity of command requires increasing input from the user who, thus, assumes the role of composer rather than transliterator. Some composer input is necessary to bridge the gap between formlessness and music. By using SoundOfAnImage I can minimize the composer input and maximize the painting input.
                The tone and timbre settings for SoundOfAnImage for all fourteen voicing are as follows:
                Instrument #1: Aah Choir
                Instrument #2: Oboe
                Instrument #3: Cello
                Tempo is 4.5 beats to a bar as defined by a bar length of 1 to 10 as indicated on the tempo scale of SoundOfAnImage.
                The computerised translation of image to music has an invisible component in the sense that we do not perceive the computer’s coded activities in the same sense of image-words-music that the brain’s code requires. We cannot look into the workings of the computer in the way that we are compelled to perceive the inner workings of our own brain. Even if I were to scroll the computer code in time with the music like a player-piano, the emotional content would still be quite different to that of the inner word-code of the brain. Image-to-music is simply a re-coded message from one code to another code, which leaves out the word-code connecting step that is the difference between translating and understanding. I can enrich the computer translation by introducing an artificial word-code. It can be any word-code, even a randomised word-code, which provides a focus for the brain’s need for the intermediate internal code that provides a sense of understanding. I can, however, give such a word-code the appearance of the emotional appeal of the inner word-code of the brain even without its substance.
                I have constructed a word-code based on the titles of each of the Stations of the Cross. The letters of the title of each Station are randomized and reconstructed in the same letter number format as the words in the original title. I have then listed fourteen emotive words, each of which is substituted for one word in the random word code for each Station. Each line that is so devised is then re-folded to make the first word in the next line of a word-code poem. Each word-code poem has as many lines as there are words in the title of each Station. The word-code poems retain a structure, a direction and an emotional content; that is why they can continue to be referred to as poems. Each poem represents a meaningful code whose meaning, like the unknown brain process, is unknown.
                The three components: image, word-code-poem and music complete the voicing of the Stations of the Cross by computer. The texts of the word-code-poems are as follows:

Station of the Cross # 1

Jesus is condemned to death

Usaeo nd tnehesscm di denial
Denial usaeo nd tnehesscm di
Di denial usaeo nd tnehesscm
Tnehesscm di denial usaeo nd
Nd tnehesscm di denial usaeo

Station of the Cross # 2

Jesus is made to bear his cross

Bsedt is caih sa deny moJ suoes
Suoes bsedt is caih sa deny moJ
MoJ suoes bsedt is caih sa deny
Deny moJ suoes bsedt is caih sa
Sa deny moJ suoes bsedt is caih
Caih sa deny moJ suoes bsedt is
Is caih sa deny moJ suoes bsedt

Station of the Cross # 3

Jesus falls for the first time under his cross

Srrfe iuesl rfo hot ficnes fear threu sas dimtj
Dimtj srrfe iuesl rfo hot ficnes fear threu sas
Sas dimtj srrfe iuesl rfo hot ficnes fear threu
Threu sas dimtj srrfe iuesl rfo hot ficnes fear
Fear threu sas dimtj srrfe iuesl rfo hot ficnes
Ficnes fear threu sas dimtj srrfe iuesl rfo hot
Hot ficnes fear threu sas dimtj srrfe iuesl rfo
Rfo hot ficnes fear threu sas dimtj srrfe iuesl
Iuesl rfo hot ficnes fear threu sas dimtj srrfe

Station of the Cross # 4

Jesus meets his afflicted mother

Etlat ieotf edm riehemcsf unjust
Unjust etlat ieotf edm riehemcsf
Riehemcsf unjust etlat ieotf edm
Edm riehemcsf unjust etlat ieotf
Ieotf edm riehemcsf unjust etlat

Station of the Cross # 5

Simon the Cyrenean helps Jesus to carry his cross

TysnJ ioa hopeless ctrsr iheeo eC erSep onr rshyc
Rshyc tysnJ ioa hopeless ctrsr iheeo eC erSep onr
Onr rshyc tysnJ ioa hopeless ctrsr iheeo eC erSep
ErSep onr rshyc tysnJ ioa hopeless ctrsr iheeo eC
EC erSep onr rshyc tysnJ ioa hopeless ctrsr iheeo
Iheeo eC erSep onr rshyc tysnJ ioa hopeless ctrsr
Ctrsr iheeo eC erSep onr rshyc tysnJ ioa hopeless
Hopeless ctrsr iheeo eC erSep onr rshyc tysnJ ioa
Ioa hopeless ctrsr iheeo eC erSep onr rshyc tysnJ

Station of the Cross # 6

Veronica wipes the face of Jesus

EsoJeffV teasc eou love sc wihei
Wihei esoJeffV teasc eou love sc
Sc wihei esoJeffV teasc eou love
Love sc wihei esoJeffV teasc eou
Eou love sc wihei esoJeffV teasc
Teasc eou love sc wihei esoJeffV

Station of the Cross # 7

Jesus falls for the second time

NiJfr ldtoo icu eeh eemafs hate
Hate niJfr ldtoo icu eeh eemafs
Eemafs hate niJfr ldtoo icu eeh
Eeh eemafs hate niJfr ldtoo icu
Icu eeh eemafs hate niJfr ldtoo
Ldtoo icu eeh eemafs hate niJfr

Station of the Cross # 8

Jesus speaks to the daughters of Jerusalem

JrhJs unfair gh std omeaeroes us tketefspl
Tketefspl JrhJs unfair gh std omeaeroes us
Us tketefspl JrhJs unfair gh std omeaeroes
Omeaeroes us tketefspl JrhJs unfair gh std
Std omeaeroes us tketefspl JrhJs unfair gh
Gh std omeaeroes us tketefspl JrhJs unfair

Station of the Cross # 9

Jesus falls for the third time

Jedif hthfs msu oel itasr pain
Pain Jedif hthfs msu oel itasr
Itasr pain Jedif hthfs msu oel
Oel itasr pain Jedif hthfs msu
Msu oel itasr pain Jedif hthfs
Hthfs msu oel itasr pain Jedif

Station of the Cross # 10

Jesus is stripped of his garments

Egtif sa despairs ss Jrp hsiremnp
Hsiremnp egtif sa despairs ss Jrp
Jrp hsiremnp egtif sa despairs ss
Ss Jrp hsiremnp egtif sa despairs
Despairs ss Jrp hsiremnp egtif sa
Sa despairs ss Jrp hsiremnp egtif

Station of the Cross # 11

Jesus is nailed to the cross

Diech on eossau se Jts cruel
Cruel diech on eossau se Jts
Jts cruel diech on eossau se
Se Jts cruel diech on eossau
Eossau se Jts cruel diech on
On eossau se Jts cruel diech

Station of the Cross # 12

Jesus dies on the cross

EsJus rage sr tsn ooeie
Ooeie esJus rage sr tsn
Tsn ooeie esJus rage sr
Sr tsn ooeie esJus rage
Rage sr tsn ooeie esJus

Station of the Cross # 13

Jesus is taken down from the cross

Etass es fhsit enuw rnks ood final
Final etass es fhsit enuw rnks ood
Ood final etass es fhsit enuw rnks
Rnks ood final etass es fhsit enuw
Enuw rnks ood final etass es fhsit
Fhsit enuw rnks ood final etass es
Es fhsit enuw rnks ood final etass

Station of the Cross # 14

Jesus is placed in the sepulchre

Shcdp su futile Ja nie uesrphlis
Uesrphlis shcdp su futile Ja nie
Nie uesrphlis shcdp su futile Ja
Ja nie uesrphlis shcdp su futile
Futile Ja nie uesrphlis shcdp su
Su futile Ja nie uesrphlis shcdp

Editor's Note. The music described in the piece above was going to be included with the paintings on the previous page, but technical difficulties were encountered in the transfer of the files. I was going to get Bob to load them to his website & pull them in from there but, sadly, before that could be done, Bob passed away suddenly.

Even though I was unable to complete the project as Bob envisaged it, the parts I have published clearly demonstrate the breadth & depth of his talents & interests. He will be sorely missed.
previous page     contents     next page



Post a Comment

<< Home