sound in multimodal texts

The two multimodal texts I examined were Jason Nelson’s “Conversation: Injury Analysis” and Yoon-Hae Chang’s “The Last Day of Betty Nkomo”.

These two texts both utilize sound very differently from each other. Jason Nelson’s “Conversation” is a multi-layered piece in terms of sound, as it lets you layer different narratives on top of each other. It even lets you control the volume and whether you want to control left and right volumes. You have the control to listen to any of these narratives, as you can turn them off. If you wanted to, you can listen to one story at a time, or you can play all of them together. The usage of sound is definitely more complicated in this piece, with the added control over left and right stereo.

Yoon-Hae Chang’s “The Last Day of Betty Nkomo” is fully controlled by the artist, as once you click to play it you have no control on how the story is told. Most interesting thing about this particular piece is the rhythm of the piece: the words flick in and out in tune with the background music, creating an odd rhythm that makes it easier for you to immerse yourself in the words and the story. The story almost follows the music in this piece, and the words almost seem like they are complementary to the music.

The two pieces differ in how much control we get as the audience, and also in the usage of music. “Conversation” doesn’t use music at all, and relies only on voices and technicalities. On the other hand, we don’t get much control in “The Last Day of Betty Nkomo” and the piece relies very much on the music to help carry the story.


Sounds in This American Life

Sound is used in different ways on NPR’s This American Life to add dynamic to the stories being told. I decided to listen to the October 4, 2013 broadcast of This American Life: Secret identities that talked about secret identities in different circumstances. Ir Glass is the main narrator of the show but for each story, the narrator changes. In this episode, different music and sound samples are used depending on the mood of the story.

The opening of the episode tells the story of the talking Paul Bunyan. As Ira Glass is introducing the story, sound samples from the location is played in the background, taking the audience’s imagination to the location as Ira Glass is telling the story. Eventually the background sound gets louder, and an interaction between the talking statue and other people carry on the story, providing examples of what it is actually like in the situation. Music is used throughout the story to add onto the comical mood of the story. The narration of the story jumps around, starting from the in-studio, to Ira Glass interviewing people about the statue, to the voices of the people being interviewed. There is also usage of silence right after the identity of the talking statue is revealed. Although we do not see the actions visually, through the vivid sound effects (of Ira Glass and co. going up the stairs in the statue), the audience is taken into the experience with them. Ending the story, mysterious magic music is used, and classical music is used to transition into the next story. Music is used very well and it really helps set the mood for the piece. The moment of reflection for this piece is how sometimes it’s better if you don’t know the secret behind the show. This idea carries on throughout the episode, applying to each story in a different way.

The second act of the show tells the story of a shy high school girl who changes 180 degrees when she is dressed up as her school’s mascot. The most interesting usage of sound for this story is how the story is told through different people. At first, Elna Baker starts telling the story. Then it smoothly transitions to Navey who tells the story from her perspective, but still staying with the same narrative. After that, it is Navey’s dad who takes the baton to tell the same story from his perspective. This switches in point of view works very well and it keeps the audience more interested in the story. The story utilizes multiple voices throughout, which keeps things more interesting. The moment of reflection in this piece is when Navey fails to do the cart wheel she could do in the mascot costume. It really is striking how we feel more confident when we are hiding behind a mask and disguise ourselves as someone else.

The third act of the show is bit more grim and serious, as it tells the story of a vigilante who shoots bus drivers in a crime-filled city in Mexico near the border. The music that is used in this story invokes a sense of mystery and suspense at some instances and a sense of tragedy in others and it is used very effectively. Different from the previous light-hearted stories, this story does not use any other voices except the main narrator Yuri Herrera’s voice. Even when he is speaking in the perspective of another woman, it is not voiced by a female voice actor or the person herself, but by Herrera. This consistency makes the story sound more like it is part of a novel that is being read by the author, as opposed to the previous lighter stories that sounded more like an oral documentary or a radio play. The moment of reflection comes at the very end, when Herrera, speaking as the bus driver, says that he is afraid of this Diana figure. He speaks extra low and dramatically when he says this, which adds a nice rhetorical effect to end the story.

The podcast is extremely well-paced as I was never bored of the stories being told, and it was really easy to follow the aural stories. I also liked how the podcast had the cohesiveness around the theme of secret identities, and how diverse the stories were.

Sound matters

  • Voice. Vocal deliveries not only include the words being said, but how they are spoken as well. The tone, accent and emphasis of the vocal delivery can affect the meaning of the story. In a sense, the delivery itself holds meaning in the storytelling. There are several qualities that can carry meaning supporting the main story, such as tension, roughness, loudness, and pitch. The voice has a very intimate relationship with the words as they support each other in order to tell the story. Recently, technology advancement has given us with more possibilities to manipulate voice in order to effectively get the message across.
  • Music. Music is largely used to establish the mood and atmosphere, as the music we listen to shape our perception. McKee talks about how we can listen to music on three planes of music: sensuous, expressive and sheerly musical. When we are listening to music on the sensuous plane, we focus on the medium of the music produced, the quality, and the dynamics or intensity of the sound. While listening to music on the expressive plane, we relate to the music and determine the feelings the music evokes in us. While listening to music on sheerly musical plane, we look at the technicality of the music, focusing on details like rhythm and melody. These planes are helpful in choosing what music to use for your storytelling.
  • Sound effects. Sound effects can do many things, such as provide background for the story, serve as a transition, help in inducing a mood, and act as emotional stimulus. Acting as emotional stimulus was something I have never thought about but sound effects used as emotional stimulus can be very effective.

Story-telling experience

Thinking up of stories to tell is difficult. I struggled to come up with something I deemed interesting enough to tell, and when I did find out, I found out half way through the story that it wasn’t very interesting at all. When I was telling this story (of how I made some of my first friends at college), I found out that I need to make sure that I know where the story is going to go and how it is going to end before I begin to tell the story. I get sidetracked often and go on a tangent which in turn ends up confusing me and I lose direction of the storytelling. Something I should definitely work on.

Image Text Assignment

trigger warning: rape, sexual assault

IMG_20131008_105628 IMG_20131008_110317 IMG_20131008_112717

For my image text project I decided to create a stencil piece that also incorporates free-hand writing. Through this piece, I really wanted to voice my frustration with the prominence of sexual assault on college campuses. I also want people to realize that date rape and sexual assault are an extremely serious problem and that everyone should be mindful of when they are making sexual advances.

The main image of the piece is a mouth with the word “no” inside the lips. The text underneath the mouth says “means no”, and together with the text inside the mouth the phrase spells out “no means no”. This phrase is actually an anti-date rape slogan that has been used in anti-date rape campaigns before. In this society where rape culture is still rampant, too often, women are sexually assaulted even if they clearly voice that they do not want to have any sort of sexual interaction. Especially on college campuses, as high as 1 in every 4 women are sexually assaulted during their college career. Initially I only planned on having the lips with the word “no” between them, but I wanted the message to be as clear as possible, hence the whole phrase “no means no”. The decision to make the piece bright red also reflects this sentiment. Red is a colour often associated with stop signs. I want the red of the graffiti to remind people of a stop sign, as this sign could be considered a remediation of stop signs in a way, as it is trying to stop sexual assault from happening. I decided to use simple type to make the words clear and legible.

John Berger has said that the surroundings of an object “confirms and consolidates its existence”, and I really wanted the environment to matter for this graffiti project. Initially, I was planning on putting the graffiti on walls of buildings around campus, but focusing on student residential areas as majority of sexual assault is committed by someone the victim knows. However, I felt as doing so would make it seem like I was targeting just the students living in those areas, which was not something I wanted to do. After contemplating on a better location to put the graffiti, I thought of bars on campus. Approximately 74% of perpetrators are under the influence of alcohol when they sexually assault someone. Through placing these graffiti beside or close to bars, I wanted the patrons to see the graffiti and be conscious of their actions for the night. Also, as many bars on campus are located in busy areas where numerous students pass by every day, in a way, the graffiti serves as a daily reminder to students that no really does mean no. I picked three bars located at three different locations on campus: Canopy Club, Kam’s and Legends. I picked these three bars as they were located at three drastically different locations on campus and even during the day, numerous students walk by them on the way to class.

Stencils can be easily applied on any surface, and are created to be reproduced multiple times. Other media such as painting can have their meanings distorted when it is reproduced, but stencils’ meaning embodies reproduction, which makes stencils perfect as a tool to voice one’s concern on an issue. If stencils are seen and noticed by many people, it becomes an icon. Even though the lips are not real and just a representation of human lips, through the stencil it becomes an icon of voice against rape culture.

Anecdotes and Moments of Reflection

In his piece for The Moth, Al Sharton tells a personal story about growing up. He talks about how he always wanted to be a preacher as he loved to preach. As he grew up he realized that he wanted to spread the word on social justice, and it eventually even read him to be stabbed by someone who was out to kill him. He eventually forgave this man and even exchanged letters with him when the attacker was in prison. He raises questions regarding forgiveness, which I found interesting and was a great moment of reflection for me. I think forgiveness is a very interesting topic, and I certainly believe that there are complicated emotions involved with the act of forgiveness. Sharton paused a lot when he was speaking, which was nice as it gave me time to take in what he was saying easily. Silence gives the listener the opportunity to mull over what is being said. Music and sound can elicit a certain mood and help engage the listeners. I feel like Sharpton used a very ‘reverend’ like voice – he talked like a preacher: very calming and sleep-inducing. Sharpton’s piece could be considered autobiographical and also political, as he drew reference to politics and social influences in his life. With my own project, I think I’ll try to add more pauses as I tend to talk without pauses sometimes.