For two years, I worked closely with Dr. Catherine Ryu of Michigan State University’s Department of Linguistics and Languages in creating a unique audio database of all monosyllables in Mandarin across the various tonal levels; the Tone Perfect: Multimodal Database for Mandarin Chinese.  The goal of this project is to have an easily accessible repository for learners and researchers to use and analyse, better responding to the demands of a language where pitch inflection can dramatically alter the meaning of a word.

As we were getting ready to unveil the initial database of six speakers and a total of 9,860 audio files, Dr. Ryu asked about the possibility of creating a piece out of them. We had discussed this before during the recording process, and in prototyping an analysis engine. As the analysis engine worked by through analyzing pitch changes over time by looking back every X samples and comparing that to the current pitch level, I was struck by how closely that approximated granulation. So, after classes had let out and the grading was finished, I began the task of creating granular instruments out of the samples.

By and large, they were a miserable and uninteresting failure. It didn’t matter if the speaker was male or female, or what the syllable or tone was – they all sounded the same.

But, I did discover that there were other ways to create interesting samples. Through time stretching and layering, interesting textural pads were created. Replacing the excitation source in a Karplus-Strong implementation created an interesting resonance in what should be a plucked string sound. And a network of reverb and resonator modules created a hazy background when multiple samples were sequenced in close proximity to one another, with selection determined by first-order Markov chains. The resulting piece uses eight primary samples to create the six instruments used, with an additional 124 sounds randomly triggered to create the background texture.

The aim of Lingua Incognita is to explore the musical possibilities of speech while also simulating the confusion of the novice language learner. As far as I’m aware, there is no semantic meaning in the arrangement of the syllables as presented. It’s up to the listener to decide on the meaning or lack thereof.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.