Biting off more than I can chew: Songification

Standard

So I had a thought: the human voice has been picked over to find sonic commonalities (at least within cultures) of expression. For instance, when someone gets vibrantly angry, they tend to get louder and their muscles tense up all over – including their vocal tract. This translates to a change in how their voice sounds. These changes are relative to a person’s own baseline, but the voice changes predictably.

I had another thought: verbal performance of poetry is awfully close to music. It has a sense of pitch, rhythm, flow, tension, and release. It’s structured differently than most popular music, but the pieces are there.

So then I had an idea: I should build an automated system that could take a recording of any (Western, English-language) meaningful verbal performance of a poem and turn it inso music that reflects or supports the aesthetics and affective nature of the performance. The system would effectively deconstruct not the meaning of the words, but the feeling of the voice performing the poem and turn that into music. Patterns that suggest anger should generate more tense music. Patterns that suggest ennui should generate more detached music. You get the idea.

I’ve spent more time on this than I really should, so I’m going to share my failures with you and move on. Actually, that’s too harsh and I should give myself a little more credit. Besides, who doesn’t like reading a blog post where the author talks about how great they are?

Here’s where I’ve succeeded:

  • making some objectively creepy music (Happy Halloween?)
  • the system does respond reasonably well to building tension
  • recording of a poem goes in, unique music comes out

Here’s where I’ve failed:

  • I wanted the system to be fully automated. I’m not sure if fully automated is possible, but I do believe that nearly fully automated is possible.
  • While the music is unique, there’s too much homogeny between the poems
  • I used a between-poems design, when maybe I should have used a within-poems design.

I think my biggest stumbling block here is that I got so lost in my proof-of-concept stage while using a typical DAW (Logic 9), I never got into using more specialized and arguably more suited tools like MaxMSP. I went through several iterations of the prototype before I decided to cut my losses and move on. Perhaps the problem is that I am using the wrong tools.

So as I mentioned, I wanted this system to be fully automated. With no user input other than the source poem, I wanted meaningful musical output. My system ended up needing massaging, kind of like Brian Eno & Peter Chilver’s Bloom app. I had to set the musical scales used for each poem instead of this choice being driven from data in the sound file. This is a technological constraint of Logic, or at least it is a constraint of my understanding of Logic. I couldn’t figure out how to get plugins to send control data to each other (with a few exceptions). MaxMSP can do this.

The homogeny is a also a product of the system not being able to manipulate itself, or to have the pieces interrelate other than a simple A -> B fashion. Data flowed from one plugin the next, then through a series of busses to be routed and mixed. Some interesting evens still happen, and the music is emotive, but it’s more like baby’s first steps than functioning elegantly. Additionally, I chose poems with speakers that have similar voices. This was a choice I debated but stuck with. I didn’t want to risk the aesthetic responses to be caused by vastly different voices, but by vastly different poems.

What I meant above about a “between-poems” and “within-poems” design is that I’m starting to wonder if I should have done my proof-of-concept on multiple different performances of a single poem. That way I could have learned if the system could generate meaningful output in that setting, before expecting it to work on several different poems. I think I not only dove in the deep end, but I decided to do it at midnight. In a quarry. With a new moon.

Anyway, enough running myself down. Have I got you excited to listen to these recordings yet? Yeesh. The three poems I chose are “Ursonate” by Kurt Scwitters, “Do Not Go Gentle Into That Good Night” by Dylan Thomas, and “May I feel said he” by EE Cummings. Jaap Blonk performed “Ursonate,” Anthony Hopkins performed “Do Not Go Gentle…,” and Tom Hiddleston performed “May I feel said he.”

I envisioned “Ursonate” as my control – the poem contains no words, just mouth sounds. Blonk gives a stirring, bizarre, and emphatic performance. It’s vocal olympics. Amazing stuff! Definite thanks to Norbert Herber for suggesting this poem and performance. In a sort of backwards way, this proves my system works to some degree. The music does not sound like the poem outright, but it has the flow of the energy of Blonk’s performance.

Here’s the version of “Ursonate” I used as input:

And here is the output my system generated:

I chose this Dylan Thomas poem partially because it is a favorite of mine, but also because of its quiet rage and it’s understated climatic moments. Here is Hopkins’ performance of Dylan Thomas:

Here is the output my system generated:

I chose EE Cummings because I feel like his poems lend themselves to being spoken aloud. Also, it is gentle, playful, and sweet. Hiddleston’s performance is muted, much like Hopkins’, but there’s menaing and feeling that I wanted to tease apart. Finally, here is the EE Cummings poem performed by Hiddleston. (Do ignore the video content, please):

And last but not least, my system’s output:

I feel like “Ursonate” demonstrates that this is a responsive system, but perhaps I went too fine-grained for my stimuli and should have instead focused on some broader strokes instead of nuanced performances. More Shatner, and less uh, Shatner. Or maybe this is as far as it’ll ever get. All I know is that I’ll go crazy if I don’t put some distance between myself and this project for a while!

Moonage Daydream: Science Fiction, Human/Computer Interaction, and Feedback Systems

Standard

“Science fiction is the most important literature in the history of the world, because it’s the history of ideas, the history of our civilization birthing itself. […] Science fiction is central to everything we’ve ever done, and people who make fun of science fiction writers don’t know what they’re talking about.” – Ray Bradbury (Unknown, 1995)

Feedback systems are key to current human/computer interaction and machine intelligence paradigms. The ability for a system to self-regulate through feedback makes it less mechanized and more interactive. In media, science fiction as a genre can and does extrapolate contemporary technology and science to explore a plausible future from that foundation of knowledge. Through that mechanism, the authors and writers of science fiction explicitly or implicitly use feedback systems in their explanation of technology. Inherently, science fiction reflects or is a critique of the time period in which it is written. Using science fiction as a kind of anthropological tool for discussion of feedback systems, a greater appreciation of how attitudes about human/computer interaction and machine intelligence have changed over time.

Science fiction is now ubiquitous within American media. However, the genre is not particularly old and its popularity is relatively new. An agreed-upon founding work of the genre is Mary Shelley’s Frankenstein published in 1818. Science fiction has gone through distinct eras since then. It is important to appreciate the differences between the eras, as well as exemplars of each, when discussing feedback systems because the eras each have their own distinct mentality in terms of what defines the genre. That will also color the discussion of technology and feedback systems.

Overture: A Brief and Relevant History of Science Fiction

The genre of science fiction is widely regarded as founded in 1818 with the publishing of Mary Shelley’s Frankenstein (Alexander, 2014). Generally speaking, the genre uses advanced science and technology as plot devices, or a means to solve a problem, to earn its branch on the tree of speculative fiction, but that is not a complete definition. In fact, science fiction is challenging to define adequately. Robert Heinlein (1957) offered this: “a handy short definition of almost all science fiction might read: realistic speculation about possible future events, based solidly on adequate knowledge of the real world, past and present, and on a thorough understanding of the nature and significance of the scientific method.”

The history of science fiction can largely be described in three eras: Early Science Fiction, Golden Age Science Fiction, and New Wave Science Fiction / Soft Science Fiction / Contemporary Science Fiction. Early Science Fiction is the birth and formative years of the genre. Here, it is not uncommon for science fiction stories to be woven into other genres – not quite strong enough to stand on its own yet. Jules Verne’s work, for example, is often a romance story wrapped in science fiction. Verne’s The Green Ray (1883) is about a man and woman trying to observe an optical phenomenon visible immediately before or after sunset, but instead they gaze into each other’s eyes and fall in love. How saccharine! That is not to suggest that Early Science Fiction lacks merit or does not deserve further inspection. Several important literary works come from this era written by authors such as Shelley, Wells, Verne, and Butler.

Golden Age Science Fiction has three clear exemplars: Asimov, Clarke, and Heinlein. These authors are known as “The Big Three;” they were considered luminaries of the era (Freedman, 2000). Works from Golden Age Science Fiction generally strive to be plausible, at least in the understanding of reality and science at the time (Roberts, 2006). The works of Verne are perhaps more fanciful and romantic; whereas Asimov is perhaps known just as well for his nonfiction works of science as his science fiction (Seller and Jenkins, 2000). In a sense, science fiction grew up. Additionally, science fiction made its way into the various new media formats such as radio and movies.

Finally, there is the conglomeration of New Wave, Soft, and Contemporary Science Fiction which is much more splintered and complex. Science fiction is now a part of every popular medium. As if it were a pendulum swinging back and forth, science fiction is once again more interested in literary value, perhaps at the cost of scientific accuracy or plausibility (Roberts, 2006). Some members of the Golden Age adapt and grow with the shift – notably Heinlein. His later works have a tendency towards Soft Science Fiction (Heinlein 1973, 1980, 1984). Others nearly abandoned the genre – notably Asimov. Much like other mediums undergoing a period of post-modernism, science fiction is now able to self-reference and self-critique and dabble more strongly in farce (Moorcock, 1963).

Theme: The Importance of Feedback Systems

A feedback system is a part of a larger system. Feedback systems take information from the larger system and then reinsert that information back into the larger system at different point. This is typically “earlier” in the larger system, in a sense of the signal flow, which lends the name feedback. The information is fed into the system to a prior point, or “back.” Another common feature of feedback systems is that they sometimes transform or interpret the information they receive before inserting it back into the larger system. Through the use of feedback, systems are able to self-regulate (United States Naval Academy).

Feedback systems are a major component to the current paradigm of machine intelligence and human/computer interaction (Wiener, 1954). With the ability for feedback, a system is able to fight entropy of information, as well as act in a manner more like human intelligence than without feedback systems. Wiener (1954) uses the example of an automatic door’s pressure plate. It is not sufficient for any amount of pressure to activate the door, but instead only a human-like amount of pressure. To achieve this, a feedback system is used in the circuit and the door is now functions in a more intelligent way. By understanding how western society implicitly and explicitly discusses feedback systems in science fiction, it will be possible to explore how the conceptualization of human/computer interaction and machine intelligence changes over time.

Variations on a Theme: Feedback Systems in Early Science Fiction

As a product of the 19th century, Shelley’s Frankenstein has a vastly different set of technology than that of the present. Victor Von Frankenstein’s laboratory is described as an attic that is lit by a single candle. Present around the laboratory are various devices and chemicals (Shelley, 1818). What is not present in any explicit sense is a feedback system, but this makes sense. As of writing in the early 19th century, the Industrial Revolution was brand new and only recently had society begun to transition to use of machinery and electricity (Wikipedia, 2014a).

But what is present is a biological feedback system: the monster. Frankenstein assembles the monster from pieces of different people and seeks to reanimate them (Shelley, 1818). In other words, Frankenstein builds a biological feedback system. Frankenstein’s creation comes to life when hit with a jolt of electricity (Shelley, 1818). While Shelley is more likely exploring the ethics and dangers of creating life, feedback systems are the core of this story. The monster is a collection of feedback systems because, much like humans, it has senses that take in information. It then processes that information, and responds to its environment. Admittedly, this connection is somewhat tenuous. It takes a modern, broad conception of feedback systems and applies it in a manner that may not be consistent with the time period’s usage of the term.

Verne (1867), however, frequently wrote about machinery in a more explicit fashion. In From the Earth to the Moon, Verne describes the technology used to send an enterprising group of travellers to the moon. Perhaps the most salient example of a feedback system in From the Earth to the Moon is the space capsule’s oxygen production machine. While the machine itself is fanciful, a minor plot point is that the regulator system on the machine goes out of alignment and the crew feels oxygen-induced euphoria. Here, a feedback system is not only explicit but also a plot device. Failings of machinery are not uncommon in all kinds of science fiction, but here the feedback system is what fails, not the machine itself. This elevates the importance of the feedback system and the value of machine intelligence. Granted, this is a very crude intelligence, but it represents the pinnacle of technology of the time. For context, the modern thermostat was only invented a mere 30 or so years earlier (Nagengast, 2001), making the notion of self-regulating systems reasonably new.

H.G. Wells (1895), another luminary of early science fiction, put forth a tale of the exploration of time with The Time Machine. Wells never got into the workings of the aforementioned machine. The reader knows it has levers, a chair, and a control panel but not much else is known (Wells, 1895). Wells’ (1895) machine may not use a feedback-controlled system at all, but at best the feedback systems are implied. It is impossible to be sure, but given the nature of the Time Traveller’s interactions with the machine, it does not seem to have any feedback systems. The rate of passage of time is controlled by the position of the lever (Wells, 1895). This is similar to how a steam engine can operate. This anti-example serves to be just as interesting, however. The time machine is so central to the plot of the book and so fascinating of a concept that it earns its place as the title, and yet very little discussion of the machine itself happens. The de facto insinuation is that how the machinery works is not important, merely that it does function. Another possibility is that Wells had no notion of how to construct such a machine and so left it to imagination. This is understandable since time travel is still an elusive, if possible, technology. Wells (1895) instead focuses on evolution and social issues, and this is what makes The Time Machine science fiction as opposed to science fantasy.

In another novel by Wells, The War Of The Worlds (1897), technology takes an even more prominent role in the story: alien invaders from Mars attempt to destroy the human race, using wildly advanced technology. Fascinatingly, despite the major role technology plays in the story, Wells makes little attempt to expound on its functions. The Martian technology is foreign and fanciful, but Wells keeps the machinations of it nearly entirely opaque (1898). In fact, Verne, his contemporary takes a jab at Wells for that very reason (Jones, 1904). There is one standout example, though: Wells describes how Martian automobiles roll. Not on wheels, but muscle-like contractions around an axle (Wells, 1989), not unlike an advanced electroactive polymer technology seen today (Wikipedia, Electroactive Polymers, 2014).

Samuel Butler’s Erewhon (1872), is a deep meditation on machine intelligence and Darwinism seated in satire of the Victorian British Empire. In Butler’s story people of the fictional nation Erewhon have banned machines out of fear that they could become intelligent. Butler’s point was lost, perhaps because it was so far-fetched, but he was very seriously concerned, having read Darwin’s Origin of Species (1859) prior to writing his novel (Butler, 2004).

As shown through previous examples, when feedback systems are discussed in Early Science Fiction, they very well may be very important to the story. However, it also seems that given the timing of Early Science Fiction with the Industrial Revolutions, the wonder of machinery might have outpaced some authors’ ability to develop plausible systems that would merit explicit discussion – such as Wells’ time machine. As such, feedback systems play a more implicit or even absent role. It also implies that, with Butler’s notable exception, machines are not even conceived to be possibly intelligent, even on a basic level. Or at the very least, feedback systems were not associated with intelligence in the same way as contemporary western society.

Variations on a Theme: Feedback Systems in Golden Age Science Fiction

A common theme in Golden Age Science Fiction is machine intelligence, and human/computer interaction. Luminaries of this period include Asimov, Clarke, and Heinlein. Also, science fiction now exists in more mediums than before with the advent of radio, television, and movies. As such, Star Trek and 2001: A Space Odyssey deserve discussion. Generally speaking, Golden Age Science Fiction takes the science portion more seriously: some authors even digress for swaths of novels to discuss science before returning to the narrative.

In 1942, Heinlein wrote a short story titled “Waldo” in which telepresence was first imagined, as well as haptic feedback. A waldo was a device that mimicked the human hands but could be operated remotely. Additionally, they provided scaled visual feedback to the user, meaning if the waldos were smaller than real human hands, the operator would get a visual feed as if they were that much smaller (Heinlein, 1950. In the story, the protagonist constructs ever smaller waldos to eventually work on the cellular level (Heinlein, 1950). Waldos, however, were not a dummy pair of gloves. Instead, they transmitted the tactile sensations they received to the operator’s hands (Heinlein, 1950). This is an example of a highly explicit and central feedback system as a plot device.

Heinlein’s Starship Troopers (1959) was the first mention of powered exoskeleton infantry armor as it is conceived of today (Weiss, 2001). Heinlein describes the function of the suit in detail, outlining a series of feedback systems necessary to make it function as desired. The suit moves with the soldier without additional human power. Additionally, it greatly enhances the human’s strength and speed (Heinlein, 1959). The suit’s use of feedback systems regulate its movement so as to not burden or injure the human inside. Again, feedback systems are highly explicit and and a major feature of the plot.

Perhaps the most iconic computer of Golden Age Science Fiction is found in 2001: A Space Odyssey, which was a joint effort between filmmaker Kubrick and author Clarke. The film and book were written concurrently, though Clarke is given full credit for the novel (1968). The computer in question, HAL 9000, is intelligent by human standards (Clarke 1968, Kubrick 1968). HAL also possesses several explicit feedback systems beyond intelligence: audio / video inputs, control panels, and sensors (Clarke 1968, Kubrick 1968). This allows HAL to take input from the world it to inform its decision making process. Of course, in 2001, HAL ends up being an antagonist who goes insane and kills most of the crew (Clarke 1968, Kubrick 1968). Murderous inclinations aside, HAL possesses a series of highly explicit feedback systems as well as machine intelligence. The mode of human/computer interaction is conceptualized as being similar to talking with another human. HAL can have information put into it via control panels when convenient, but much of the interaction with HAL is done with colloquial verbal communication. HAL can go one step further than verbal communication: it can read lips, much to the chagrin of the human crew (Clarke 1968, Kubrick 1968).

Asimov (1942) also tackles machine intelligence and human/computer interaction in a big way with his introduction of The Three Laws of Robotics:

“A robot may not injure a human being, or through inaction, allow a human being to come to harm.

A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.

A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.” (Asimov, 1942, p85)

Not only does this form a short manifesto on machine intelligence, but also infers a series of feedback systems. For example, the first law is directly guided by feedback systems: if the robot is about to act in a manner that would harm a human, feedback from this information would halt the robot. Machine intelligence, as well as human/computer interaction, is implied in the second law. If a robot is to receive orders, humans must be able to give the order in some manner. Additionally, the robot must be able to interpret the orders.

Though not seated in literature, Star Trek is a major collection of works of science fiction, spanning decades and arguably eras of science fiction. Though it began after the Golden Age of Science Fiction, its heritage is seated deeply in the Golden Age with it’s strong regard for hard science fiction. Possible era-spanning aside, Star Trek in its two most recognizable configurations – the original television series and Star Trek: The Next Generation television series – contains numerous examples of human/computer interaction, feedback systems, and machine intelligence. Additionally, during the two decade gap between the original series and The Next Generation, there is a demonstrable difference in extrapolations of then-current technology.

The mode of human-computer interaction in the original series is centered around buttons and levers, with a comparatively limited use of screen-based displays. The computer was able to talk, but barring a few notable exceptions like the Enterprise’s elevator system, it did not respond to voice commands. In The Next Generation, screens are present everywhere, as well as a contextual graphical user interface on touchscreens (Memory Alpha, 2014). Additionally, the computer was highly responsive to voice commands of many different kinds, from food orders to tactical commands (Memory Alpha, 2014b).

Feedback systems can and do routinely play major functions in plots for several episodes. For example, “The Doomsday Machine” (Daniels, 1967) in the original series, features a run-amok planet-eating machine that is on a trajectory to eat several densely populated planets. The machine has a defense system no doubt run by feedback systems which causes it to attack the most imminent threat posed to it (Daniels, 1967). While the mechanism is not described or explored, the outcome of the function of the planet-eating machine makes the feedback system inevitable.

The holodeck, an eminent technology of The Next Generation, uses a series of feedback systems to function. A holodeck technology is an immersive virtual reality where a computer-controlled simulation creates a material, though ephemeral, reality around the people on the holodeck (Memory Alpha, 2014a). In the simulation, there are no specialized interfaces or devices that a person must use to interact with the simulated environment. Though the deck is of finite size, it can trick the people on the holodeck into making it seem nearly limitless through optical illusions and other psychophysical tricks (Memory Alpha, 2014a). The use of psychophysical tricks implies a feedback system, but it is both nuanced and vague. Clearly, the holodeck needs to respond not just of the position of the people in the holodeck, but where they are looking and moving. Without eye tracking, optical illusions, such as those that imply depth along the z-axis, would be impossible with current understandings of optics. To track eyes, a feedback system would be necessary.

The holodeck is also a model of human/computer interaction due to the fact that people on the holodeck require no special equipment to interact with the simulated reality. From some perspectives, such as Ascott (1967), Krueger, (1977) and Wiener (1954), this is the ultimate form of human/computer interaction because it is seamless and effectively indistinguishable from reality. Through a series of complex feedback systems, the mode of human/computer interaction is changed substantially.

Variations on a Theme: Feedback Systems in New Wave Science Fiction, Soft Science Fiction and the Present

New Wave Science Fiction is a primarily literary movement, where science fiction writers begin experimenting with the form and function of the literature itself (Moorcock, 19363). Shea and Wilson’s The Illuminatus! trilogy (1984) is an example of New Wave Science Fiction because it is absurdist, and written in a non-linear fashion. The scientific portion of the novel is reduced to nearly a farce, and serves as framework for the narrative as opposed to the substance of it. The book explores a series of conspiracy theories, using time travel to tie it all together (Shea & Wilson, 1984).

Similarly, Dick’s A Scanner Darkly (1977) features seemingly possible technology, but it too is used mostly to make a setting for the narrative to unfold as opposed. The protagonist takes a fictional drug and has a variety of experiences while on the drug, blurring reality (Dick, 1977). Again, the scientific aspects – the formulation of the drug, for example – are not particularly relevant to the story. The main scientific interaction could be described as the implicit neurophysiology changes in the characters caused by the drug.

Whedon, a prominent writer and producer of science fiction content from the end of the 20th century to the present, represents a decidedly different tone in his relationship to technology, human/computer interaction and machine intelligence. In Firefly (Whedon, 2002), technology seems similar to present day technology with the noted exception of faster-than-light travel, starships, and thlike. Typical keyboards, levers, and buttons are used regularly on interfaces (Unknown, 2014). In Buffy The Vampire Slayer, Whedon (1997) has story arcs dealing with machine intelligence, but the “how” is glossed over almost entirely, much in the way that Wells never explained how his Martian invaders might function (Jones, 1904). This suggests a major shift away from being interested in how technology works to how people work or Soft Science Fiction.

Much like Whedon, Scalzi also diminishes the role of technology in his stories. For example, in Red Shirts (S2012), a major plot device is that the technology on board the starship works only because the author specifically wrote it that way. In Old Man’s War (2005), technology is discussed to some extent, but again the people are the larger focus. For example, the faster-than-light system that the ship uses is explained to be actually jumping through dimensions, however the characters’ experience with being young again is explored in depth (Scalzi, 2005). This developing theme of interest in people over science fiction is a defining trait of Soft Science Fiction (Sterling, 2014). Soft science refers to branches of science that deal with humanity, like sociology, psychology, political science, and linguistics (Sterling, 2014).

Coda: Analysis of Feedback Systems in Science Fiction

With exceptions, Golden Age Science Fiction is the most explicit in terms of feedback systems. Ideally, some type of content analysis would bolster these case studies, but the culture and values of the eras seem coherent. And the changes in tone can be found mirrored in texts discussing feedback systems in human/computer interaction and machine intelligence.

Early Science Fiction has no corresponding human/computer interaction text. Understandable, given that the modern computer will not exist for decades yet and as such no theory existed to discuss human interactions with them. However, given the context of the time in which it was written: during or immediately after the Industrial Revolutions, machinery was just beginning to become relevant to the daily lives of many people in society (Wikipedia, 2014a). In fact, society as a whole changed drastically and technology, in the layperson’s sense, was present now more than ever. And the possible wonders of self-regulating systems were only just starting to be explored. But overall, reactions to the major societal shift seem to be the central topic at hand. Only a few authors had the gall to suggest machine intelligence was possible: Butler was mocked somewhat mercilessly for suggesting machines could be intelligent at all, let alone surpass human intelligence (Butler, 2004).

Golden Age Science Fiction has some stunning parity with human/computer interaction literature. Wiener (1954) is speaking of human/computer interaction when he uses the term “cybernetics.” And to him, feedback systems are tantamount to not only interaction, but intelligent interaction (Wiener, 1954). Without feedback systems, machines cannot self-regulate or respond to conditions that would impact their function (Wiener, 1954). Clarke, Asimov, and Heinlein all recognize this fact through their stories. Asimov’s Three Laws, Clarke’s HAL 9000, and Heinlein’s waldos all rely on feedback systems to be able to function.

Perhaps because computers are no longer a novelty, or not a novelty in the same way they were during the Golden Age, New Wave, Soft and Contemporary Science Fiction generally seem to mirror more of Wells’ explication of technology. Technology is present, but how it works is of lesser importance. Just the same, a fair amount of explicit discussions of feedback systems can still occur. Science fiction is now mature enough to have its own nostalgia, it seems. And as such, there are many splinters and subgenres. It is too diverse a group to make sweeping statements, but it is safe to say that people are again more of the focus than the technology or how it might work.

And just the same, this theme appears in human/computer interaction literature. For example, Krueger (1977) writes more about the mode and meaning behind interaction than the technical function of it. Ascott (1967) describes the computer as a tool and how far it may go, but writes little about the uses of feedback to control the system. And this could mean that the unit of analysis has changed. Previously, the circuit was interesting, now the system of circuits and what they can do are interesting. Perhaps there is some poetic humor to be found between computer chip design and human/computer interaction: the chip designer spends their life making ever-smaller and powerful chips and as they do, the human/computer interaction researcher remembers less and less that the chips exist at all.

WORKS CITED

Alexander, L. Mary Shelley, Frankenstein. Retrieved September 27, 2014, from http://www.utm.edu/staff/lalexand/frankqst.htm

Ascott, R. (1967). Behaviorist Art and the Cybernetic Vision multiMedia: from Wagner to Virtual Reality (pp. 95 – 103). New York: W. W. Norton & Company, Inc.

Asimov, I. (1942). I, Robot Retrieved from http://nullfile.com/ebooks/%28ebook%29%20Asimov,%20Isaac%20-%20I,%20Robot.pdf

Butler, S. (1872). Erewhon. United Kingdom.

Butler, S. (2004). The Note-Books of Samuel Butler (H. F. Jones Ed.).

Clarke, A. C. (1968). 2001: A Space Odyssey: New American Library.

Daniels, M. (Writer). (1967). The Doomsday Machine [Television Broadcast]. In G. Roddenberry (Producer), Star Trek: Desilu.

Darwin, C. (1859). Origin of Species. United Kingdom: John Murray.

Dick, P. K. (1977). A Scanner Darkly: Doubleday.

Freedman, C. (2000). Critical Theory and Science Fiction: Doubleday.

Heinlein, R. (1950). Waldo: Doubleday.

Heinlein, R. (1957, 2012). Science Fiction: Its Nature, Faults and Virtues. Retrieved September 27, 2014, from http://www.loa.org/sciencefiction/biographies/heinlein_science.jsp

Heinlein, R. (1959). Starship Troopers. United States of America: G. P. Putnam’s Sons.

Heinlein, R. (1973). Time Enough For Love. United States of America: G. P. Putnam’s Sons.

Heinlein, R. (1980). The Number of the Beast. United States of America: Fawcett.

Heinlein, R. (1984). Job: A Comedy of Justice. United States of America: Ballantine Books.

Jones, G. (1904). Jules Verne At Home. Retrieved Septbember 27, 2014, from http://jv.gilead.org.il/evans/Gordon_Jones_interview_of_JV.html

Kreuger, M. (1977). Responsive Enironments. In R. Parker & K. Jordan (Eds.), multiMEDIA: from Wagner to Virtual Reality (pp. 104 – 120). New York: W. W. Norton & Company, Inc.

Kubrick, S. (Director). (1968). 2001: A Space Odyssey [Film].

Memory Alpha (2014). Library Computer Access and Retrieval System. Retrieved September 27, 2014, from http://en.memory-alpha.org/wiki/LCARS

Memory Alpha (2014a). Holodeck. Retrieved September 27, 2014, from http://en.memory-alpha.org/wiki/Holodeck

Memory Alpha (2014b). Computer. Retrieved September 27, 2014, from http://en.memory-alpha.org/wiki/Computer

Moorcock, M. (1963). Play With Feeling. New Worlds, 123 – 127.

Nagengast. (2001). An Early History of Comfort Heating. Retrieved September 27, 2014, from http://www.achrnews.com/articles/an-early-history-of-comfort-heating

Roberts, A. (2006). The History of Science Fiction. New York: Palgrave Macmillian.

Scalzi, J. (2012). Redshirts: Tor Books.

Scalzi, J. (2005). Old Man’s War: Tor Books.

Seiler, E. J., & Jenkins, J. H. (2000, February 24, 2000). Isaac Asimov FAQ, Part 1/4. Retrieved September 27, 2014, from http://www.faqs.org/faqs/books/isaac-asimov-faq/part1/

Shea, R., & Wilson, R. A. (1984). The Illuminatus! Trilogy: Dell Publishing.

Shelley, M. (1818). Frankenstein; Or, The Modern Promethius (1st ed.). London: Lackington, Hughes, Harding, Mavor & Jones.

Sterling, B. (2014). Science Fiction. Retrieved September 27, 2014, from http://www.britannica.com/EBchecked/topic/528857/science-fiction/235713/The-evolution-of-science-fiction

Unknown. (1995, March 24, 1995). Brown Daily Herald. Retrieved from http://en.wikiquote.org/wiki/Science_fiction

Verne, J. (1867). From the Earth to the Moon (Anonymous, Trans. Vol. 4). France: Pierre-Jules Hetzel.

Verne, J. (1883). The Green Ray. France: Pierre-Jules Hetzel.

Weiss, P. (2001). Dances With Robots. Science News Online. Retrieved September 27, 2014, from http://web.archive.org/web/20060116201552/http://www.sciencenews.org/articles/20010630/bob8.asp

Wells, H. G. (1895). The Time Machine. London: William Heinemann.

Wells, H. G. (1898). The War of the Worlds. London: William Heinemann.

Wiener, N. (1954). Cybernetics in History. In R. Packer & K. Jordan (Eds.), multiMEDIA: from Wagner to Virtual Reality (pp. 47 – 54). United States of America: W. W. Norton & Company, Inc.

Wikipedia. (2014a). Industrtial Revolution. Retrieved September 27, 2014, from http://en.wikipedia.org/wiki/Industrial_Revolution

Wikipedia. (2014). Electroactive Polymers. Retrieved September 2014, 2014, from http://en.wikipedia.org/wiki/Electroactive_polymers

Unknown (Producer). (September 27). 226826. [photo] Retrieved from http://images4.alphacoders.com/226/226826.jpg

United States Naval Academy Elements of Feedback Control. Retrieved September 27, 2014, from http://fas.org/man/dod-101/navy/docs/fun/part03.htm

Whedon, J. (Writer). (2002). Firefly [Television series]. In J. Whedon & T. Minear (Producer), Firefly: Fox.

Whedon, J. (Writer). (1997). Buffy The Vampire Slayer [Television Series]. In J. Whedon (Producer), Buffy The Vampire Slayer.

Transportation, Presence, and Flow in Bowie’s “TVC 15”

Video

David Bowie is a weird guy. I mean, let’s make no secrets about it. But let’s talk about the song I’ve linked above called “TVC 15.” (With visuals primarily from a movie he starred in called The Man Who Fell To Earth.) According to the ever-trustworthy Wikipedia, the song was inspired by Iggy Pop’s (drug-fuelled) fear that Bowie’s TV would swallow Iggy’s girlfriend. And so Bowie elaborated and expanded on the story so that the TV is holographic. The girl crawls in and the narrator thinks about doing the same himself. Kind of sounds like a Lt. Barclay episode of TNG.

Lt. Barclay, about to do something dumb with the holodeck.

So anyway, “TVC 15” is an interesting song because with subtly different interpretations of the lyrics, we can find the theories transportation, presence, and flow. First, here are the lyrics:

Up every evening 'bout half eight or nine
  I give my complete attention
  to a very good friend of mine
  He's quadraphonic, he's a,
  he's got more channels
  So hologramic,
  oh my TVC15 one five
  I brought my baby home, she
  She sat around forlorn
  She saw my TVC15, baby's gone, she
  She crawled right in, oh my
  She crawled right in my
  So hologramic,
  oh my TVC15
  Oh, so demonic,
  oh my TVC15
Maybe if I pray every, each night I sit there pleading
"Send back
  my dream test baby,
  She's my main feature"
  My TVC15, he just
  Stares back unblinking
  So hologramic,
  oh my TVC15
  One of these nights I may just
  Jump down that rainbow way, be with my baby, then
We'll spend some time together
  So hologramic, oh my TVC15
  My baby's in there someplace
  Love's rating in the sky
  So hologramic,
  oh my TVC15
Transition
  Transmission
  Transition
  Transmission

The reason it’s possible to poke at all three of these theories with one song is its ambiguity, and the power of interpretation: If we start with a literal interpretation, the happenings are science-fantasy: a girl literally crawls into a ‘hologramic’ TV. And, well, that doesn’t help us at all. So let’s call it metaphor – she doesn’t literally crawl into the TVC15, she just watches it nonstop. First, let’s focus on transportation. Transportation happens when the audience is drawn into the narrative of a mediated message. It’s not about technology, it’s about the storytelling itself. So right off the bat, this doesn’t seem like a good fit since the narrator touts the technological prowess of the TVC15. However, that could be a red herring or even the narrator could be mistaken. Though, now I’m speculating and supposing content that isn’t expressed in the song. But we’re slightly more interested in learning about these theories than accurate interpretations of these lyrics. For the sake of discussion, let’s pretend that the narrator is mistaken about the mechanism of the girl’s intense focus on the TVC15. She’s not captivated by the hologramic projections, or the rainbow waves. She’s taken by the stories presented on the TVC15, so much as she feels like she is a part of that world and the narrative becomes essentially real while she is in the moment. Though, as I said, with the information presented in the lyrics, transportation may not be a great fit to describe what the girl and the narrator are experiencing.

Next, let’s look at presencePresence describes an immersive state much like transportation, but is not solely about the narrative. It also has to do with the technology employed and the person’s ability to use the technology. Presence is when the person feels more than just in the narrative world, but a part of it and that they have a sense of agency. For example, a person could be less likely to feel presence when reading a book than playing a video game because a book is decidedly less interactive. But the media need not be interactive to induce presence: a 3D horror movie that makes the audience duck when the masked lunatic throws an ax towards them could be an instance of presence. If the audience didn’t think the ax portrayed by the movie was present, they wouldn’t move to avoid it. It feels like the girl and protagonist are in “TVC 15” might be experiencing presence because they are making choices with and within the mediated environment. She crawls in and gets lost in it. Had the lyric been about her getting pulled in, it wouldn’t suggest a choice but a compulsion. But here, she and the narrator have the ability to choose. Presence seems like a good fit.

Finally, let’s investigate flow. Flow is a positive experience of deep concentration. I often hear people talk about it as being “in the zone.” Let’s say a passionate birdwatcher – an ardent ornithologist, if you would like a little alliteration – sets out at 2pm to scope out the rare Buruk, a bird of prey. The birdwatcher gets set up with their binoculars and starts watching. Next they look up, it’s suddenly 6pm! Where has the time gone?

The suit is to… blend in? Maybe he’s pigeon watching.

Flow is more than just intense focus. It’s where an person’s skill closely matches the difficulty of the task. They are challenged, but not over taxed. But here we are talking about a watching, essentially, a high-tech TV. It’s practically a one-way street, right? A person has information presented to them and it just happens. Not so much, actually. The narrative and technical structure of the media needs to be comprehensible to the viewer for it to even have a chance for being enjoyable. Furthermore, flow is a state of losing not just sense of time but the sense of self. The girl “crawl[s] right in” the TVC15, which metaphorically could mean she’s lost herself.

Let’s reflect on some common experiences. Shows designed for very young children like Thomas The Tank Engine are sometimes difficult for a more mature mind to enjoy: the stories are incredibly simple and predictable. The pacing is slow. The dialog is basic and flat. That’s because as we mature, we learn more about how narratives, TV shows, and movies work. For example, we begin to develop an implicit understanding of what a montage means, whereas that could be very confusing to a very young mind. On the flip side of that, I am willing to admit that sometimes I see an art film that is far too complex both in narrative or technical prowess for me to appreciate. My skill is too far below the difficulty of the task for me to enjoy it, let alone become enthralled by it and enter into a state of flow.

So going back to “TVC 15,” it seems like the girl could just as easily be in a state of flow instead of experiencing presence. Personally, flow tips the scale in my mind. The reason being is that it accounts for the massive amount of time the girl spends watching. Presence doesn’t require long exposure, but it is a common ‘symptom’ of flow. We can assume that a regular TV wasn’t captivating enough of an experience – not challenging enough – but the TVC15 ‘hologramic’ display is what drew her in, causing her to lose herself in the narrative.

So there you have it: “TVC 15” just might be describing a state of flow.

Year-End Wrapup

Standard

Much like my social life, this blog had a bombastic start that petered out. But hey, it’s the summertime now and I’m learning to come back out of my shell a bit.

This past April, I was able to attend the Broadcast Educators of America conference. As my first academic conference, I’m not sure how much valuable insight I can offer in that regard. But I can offer a little advice:

  • Be prepared to eat over-priced, terrible food. Seriously, like $8 for a soggy sandwich made with only the cheapest ingredients.
  • You won’t be able to get to everything you want to.

I felt that from a scheduling standpoint, BEA was very well organized. There was minimal – if any – overlap within divisions. Most overlaps occurred when my interests were represented in different divisions and those were scheduled simultaneously. Obviously, no one could predict that.

One experience that I found to be surprising and strange was in a business meeting for the audio division. Each year, the audio division holds a contest for students and faculty to show off their projects. The focus is on news reporting, story telling, and advertisements. It is my understanding that other types of content are not explicitly excluded. That being said, a few music submissions are made each year and each year they are rejected.

This year, a music recording submission was not immediately rejected and went through several stages of the contest before being removed. Of course, the person whose entry was removed was not gracious and complained to the division and to HQ. So it boiled down to this: the audio division was debating in this meeting whether or not to include music submissions in the contest. I couldn’t fathom this. Music radio programming dwarfs news programming – Country music stations alone beat out all of news/talk listening.

The choice wasn’t necessarily ideological. It was based out of an inability to judge the material and an unfamiliarity with the core concepts of music production. Let me be clear, I am grateful that the group recognized these issues instead of plowing forward with hubris and ego. However, it still struck me as odd. It would be like a television division having no idea about what a TV show even is, really. People don’t tune into radio for advertisements or even the news necessarily. The product, to the customer, is the content. It baffled me to sit in with a group of radio experts that had no idea how to judge the content of a non-news/talk radio station.

The good news is that the group is working on solving this problem. And again, I am grateful that they didn’t think they could just wing it.

I hope the take away from this is not that I dislike BEA. Quite the opposite. It was an opportunity for me to engage with a lot of different educators on a lot of different levels. BEA has research, production, and pedagogy. What more could you want? Oh, you want free passes to NAB? Well sure, you can have those too.

 

Anyway, aside from reflections on the conference, here’s an update on some of the research I’ve been working on:

  • Evolved vs Symbolic Persuasion: working within the Dynamic Human-Centered Communication Systems Theory framework to see if mediated messages that don’t rely on symbolic communication (written or spoken language) and only use real/natural sounds and visuals are more persuasive (in the form of an implicit attitude change) than those that do use symbolic communication. Results: Not much. I still feel like we are onto something, but I feel like we need different stimuli.
  • My Missing Bridge: working within the Limited Capacity Model of Motivated Mediated Message Processing framework to see if familiarity with a song impacts the orienting response to edited out sections. Meaning, all of the songs were edited to remove every and any section of a song that was not a chorus after the first chorus. Most of our edited songs had this structure: Intro – Verse 1 – Chorus 1 – Chorus 2 – Chorus 3 – Outro. Typically a pop song would have a structure like this: Intro – Verse 1 – Chorus 1 – Verse 2 – Chorus 2 – Bridge – Chorus 3 – Outro. We thought that people that were more familiar with the song would respond more strongly to the edited version because it defied their expectations. Results: Not much. I feel like we struggled against the language of music in this one. Regardless of familiarity, people have implicit attitudes about western pop music and its song structure. Even if a person is not familiar with the song itself, they’re familiar with how similar songs should be shaped. When this song defies that expectation, people orient to it. That’s my thought anyway.
  • Fletcher-Munson Revisited: put on hold for the time being. Hopefully it will be resurrected this coming semester.
  • Up The Hill Backwards: to be confused with the track off of Scary Monsters. This is the working name for the generative music systems in video games study previously mentioned. This will very likely be my thesis. The literature review is done and the study is designed. Now I just need to polish up the proposal paper. I’m excited about this one because if there are significant findings, it should demonstrate the value of generative music systems in interactive media. Could you imagine playing a video game where each moment feels like the soundtrack to a movie? Perfectly timed with every movement and action on screen? That’s where I’d like to go, personally.
  • Spotify: in limbo. My fault, really. Going to be looking for evidence of inverted payola in Spotify. As my committee chair put it: “Well, of course there IS payola, but the question is where?” Hopefully it’ll be the first place I look. That’d just be convenient.
  • Party Music: a bit hush-hush for now. It’s a collaboration between Cognitive Science, Psychology, and Telecommunication people. What I can say is this: it’s partially about party music.
  • Change Deafness: Ever see one of those comedy sketches where an actor approaches a person to ask for help with a map, then the actor is briefly blocked from the person’s vision only to be replaced by an obviously different (to the audience) actor but the person doesn’t notice? That’s called change blindness. So logically, if change blindness is a thing, then is change deafness? Participants listened to a series of messages where the voice actors change during the message. How big of a change needs to occur before it’s reliably noticeable to the participant? Results: Participants notice some changes but not others. We’re working on quantifying the differences in the voices to better understand why some were noticed and others weren’t.

Other than all that, I’m teaching a course of my own design: Location Audio. It’s all about capturing good audio in the field and how to fix it when it goes wrong. Oh, and my fiancee and I are tying the knot in 2 and half weeks.

I’ll sleep when I’m dead. 

“OR”? Like, Oregon?: Orienting Responses

Standard

A real cornerstone concept in the research I engage in is the Orienting Response, which is commonly abbreviated as OR. It’s your brain and body automatically seeking more information about something that just happened. That “something” may be sufficiently novel or pertinent to you, whatever it is. And once it crosses that threshold to grab your attention, a predictable series of events happens in your body.

But first, let’s back up and examine the types of orienting responses first. As mentioned, there’s generally considered to be two types: the kind that trigged by something novel, and the kind that is trigged by a signal. The novel ORs are the easiest to digest, so let’s start there. The term novel is used in the sense that means new or unusual. So that’s to say something like you’re home alone at night, and you hear a door slam upstairs. (Don’t worry, it was just the wind blowing the door shut… probably. Definitely not a ghost.) That would most definitely illicit an orienting response because the sound of the door slamming was wholly unexpected, especially in a supposedly empty house.

But an orienting response triggered by a novel event doesn’t need to be scary. It could be mundane, like a cut from from one camera angle to the next while watching Doctor Who. You probably didn’t even think about the camera angle change, but the cool thing is your body still responded because it was novel: the whole of what you were seeing just changed position. That’s not possible in real life, so it caught your attention.

A signal-based orienting response would be something like hearing or seeing something you are looking for. The most classic example is when you are minding your own business and suddenly you hear someone say your name. You turn to look and see someone waving in your direction. As you start to smile back and wave, the person they were actually talking to pushes past you to greet them. Ouch. Anyway, that moment where you suddenly hear your name? That caused an orienting response in your body.

Now that you know what an orienting response is, we can talk about its features. As previously promised, ORs cause a series of predictable events in your body. There’s a whole bunch, but the most commonly looked at are heart rate (more specifically, the inter-beat interval) and electrodermal activity. Inter-beat interval in the time between heart beats. Immediately following a stimulus that triggers an orienting response, our hearts slow down for 6-10 beats, then speed back up. Coincidentally, since it is tied to the number of beats and not a strict duration of time, that’s why inter-beat interval is preferable to heart rate expressed in beats per minute.

Electrodermal activity is a measure of the (tiny) increases of sweat caused by an OR in a few spots on the body. Even though the link above references a lie detector, that’s not quite an accurate portrayal, just relatable. And something good needs to come from watching that terrible movie that was actually well received. But when the you experience an orienting response, the sweat glands connected to your parasympathetic nervous system release very small amounts of sweat, causing your skin to conduct electricity more freely. Isn’t that neat?

So there you have it. Our bodies respond in obvious and predictable ways during orienting responses. And they happen all the time without you even knowing it. But any time your attention is redirected, you’re having an orienting response. Neat, huh? Even the smallest surprise causes major changes in your body.

Peter Gabriel knows what’s up

Video

I’m not sure exactly when it happened, but Peter Gabriel’s music is on Spotify US now. I was pleasantly surprised to find this because he’d been making noise for recently about specifically not using Spotify. And I suppose I understand the arguments, but I disagree with him and David Byrne, as much as that hurts to type. Though I guess Spotify is a topic for another day, or even another blog. Just don’t believe that viral infographic from a few years back. Turns out it’s total crap and you shouldn’t believe everything you read.

Anyway.

I was so happy to see Peter Gabriel’s music on Spotify that I promptly pulled up my favorite album of his – Peter Gabriel 3: Melt. Brilliant album in a lot of ways, with lots of quirks. Definitely well received. The most notable one, at least in audio production circles, is that there are absolutely no cymbals, rides, or high-hats of any type on the album. It’s the kind of thing that doesn’t register as wrong if you don’t know it’s missing, but it leaves you feeling odd.

So the above song in particular really struck me in a new way. It’s part of what I love about music: a song I know in and out gives me a new experience because I’m different now. I knew the lyrics before, but now they mean something new. “And Through The Wire” is now a song about media psychology to me. It’s about communicating through media, and how that alters the impact and processing of the messages. Give the lyrics a read while listening and I think you’ll see what I’m going on about. We do, in fact, get so strange across the border of media.

Am I reading into it a bit? Sure.

Wrestling with continuous response measures

Standard

Something I’ve wrestled with on more than one occasion is a phenomenon known as clipping in the audio world. It’s caused when the signal is too loud for the file.  In a slightly more precise sense, there’s more information in the input signal than the medium can store. Check out this graphic: Image

(I’m not crazy about the source, but here it is)

So in audio, once you try to record something louder than the system can handle the extreme ends get chopped off like on the right side of the graphic. That’s gone forever. What gets stored is a very poor representation of the original. And here’s where it comes to continuous response measures: if a subject reaches the top of the scale, that’s clipping and that’s loss of information.

I’ll use myself as an example here. In the past I’ve been a subject for a pre-study test on some stimuli. We were asked to indicate how much we were enjoying the clip we were watching. The first few clips where things like commercials, Maury, and Big Bang Theory. As you may imagine, they were all rated very low. (Bazinga!)

But next, Louis C.K. appears on the screen. Oh man, major jump up in enjoyment right there. This is my favorite comedian. Now I’m probably at like a 7.5/10 on the enjoyment scale. And now that I’m looking more closely, I notice this is from my favorite stand-up special of his. 9/10. And as luck would have it, he starts my favorite bit. I’ve now moved up to 10/10 on the scale of enjoyment because after suffering through Big Bang Theory and commercials, this is heaven.

But here’s the problem: I’m already at 10/10, and we’re only in the set-up of the joke. As the joke progresses and I’m enjoying it more in anticipation of, and then delivery of, the punchline; I can’t express that. I’m at 10/10 already. I’m clipping and the researcher looks at my data and thinks I equally enjoyed the set up and delivery of the punchline. As you may imagine, I was laughing hysterically at the punchline, but only stifling giggles at the set up. Why did I do that, as the subject? Why would I go to the maximum before the punchline? Because I can’t predict the future. All I can do is indicate increase or decrease in enjoyment, and I don’t know what the future holds so I can’t realistically say “gosh, no I should hold out for the punchline.”

So some of you may be noting that once several subjects are run, and the stimulus order is randomized, that’ll come out in the wash. And it sort of does, I’ll give you that. The overall impact on the dataset is minimized. But why not just use a system that can’t clip? Why mix in clipped, distorted data in the first place? Conducting studies is hard enough, so why stack the cards against yourself?

Rothko on research

Standard

Mark Rothko, a favorite painter of mine, once put forth a manifesto with Adolph Gottlieb about the nature of art in 1943:

1. To us art is an adventure into an unknown world, which can be explored only by those willing to take the risks.

2. This world of imagination is fancy-free and violently opposed to common sense.

3. It is our function as artists to make the spectator see the world our way not his way.

4. We favor the simple expression of the complex thought. We are for the large shape because it has the impact of the unequivocal. We wish to reassert the picture plane. We are for flat forms because they destroy illusion and reveal truth.

5. It is a widely accepted notion among painters that it does not matter what one paints as long as it is well painted.

6. There is no such thing as a good painting about nothing.

7. We assert that the subject is crucial and only that subject matter is valid which is tragic and timeless. That is why we profess spiritual kinship with primitive and archaic art.

(source)

What’s so striking to me about these statements is that you can replace the various uses of art and painting with research and theory and the manifesto works just as well. Most points seem to translate directly. Rothko’s second point may be the hardest to defend as the rejection of common-sense may invoke a knee-jerk reaction from the pragmatist in us all, but I’d be willing to interpret it as a romanticized expression for the necessity of imaginative and original research questions. The fourth point is my favorite. It speaks to the excitement of finding a simple yet powerful theory that brings transparency to prior opaqueness. Lastly, the final point about the subject must be timeless and tragic: I’m not sure exactly what Rothko means by timeless, or rather how literally he meant it. Humanity is obviously not timeless, so I’m not sure how a piece of art – which by necessity reflects the human condition – could truly be timeless. Now, assuming he meant timeless to humanity, then I think it’s a fantastic point. The best answers to questions are not temporally locked, but true everywhere and as long as humanity is bumbling about.

(Yes, physics, I can see your smug look from here. You’ve got the “timeless” thing locked down. Congrats.)

“Oh. So, like subliminal messages?”

Standard

That’s the question I often get asked when I begin to explain my research. It’s a fantastic question. I mean, how exciting would it be to actually develop subliminal messages? Ethically dubious (at best), but exciting none-the-less. No, my aims are much less grandiose than world domination.

That being said, audio-in-media research seems to be underrepresented as a whole. There’s probably a litany of reasons, ranging from lack of interest to difficulty of understanding the neurological responses to audio compared to visuals. The prior is something I have in copious amounts and the latter I am equally baffled. But regardless of the difficulty, as every research area has its own challenges, there is presently a dearth of knowledge and I am working to help fill the gaps.

Generally, my current research projects all keep me in the realm of cognitive processing of media and using psychophysiological data to help express that processing. One project is looking at evolved vs symbolic visual and aural communication. More on this one later as it develops. Another, which we’ve taken to calling “My Missing Bridge,” is about cognition of familiar songs that have been edited to remove most verses and – you guessed it – the bridge. “My Missing Bridge” has recently been accepted to the 2014 International Communication Association conference. Another study that’s gearing up is about reinterpreting the venerable Fletcher-Munson study through the lens of LC4MP. I’m really excited about this one, as it could lay some important groundwork for future study. Another that’s quickly growing legs is focused on generative music in video games: can a generative soundtrack increase flow? Aside from audio, I’m involved in a study that involves collecting psychophysiology data on people viewing pornography with Dr. Bryant Paul, who has a great nickname.

So that’s what I’m up to right now. I’ll post updates as they become available.