Monday, July 21, 2008

IMAGES AND METAPHORS - HEART OF EXPRESSION:

Every night, our sleeping dreams are made of images, and in our conscious mind's eye we think in images every day. Our need to express stories in pictures is ancient, instinctive and poetic. Images and metaphors arise spontaneously, intact and whole, from the deepest parts of our psyche.

They allow us to communicate with each other symbolically, so that we can grasp the meaning of a complex idea in a single moment. Electronic media offers many options for capturing and displaying symbolic imagery.

IMAGES: We gather our ability to image as kids, when we imagine wizards and witches, monsters and fairies. More extreme associations come about when we become subject to highly emotional states; like grief, hunger, isolation, anger, etc.

These images arise all the time, our 'inner' child's reaction to our emotional state, a response to our day-to-day experience of the 'outside' world. Memory forces us to cluster images in our minds, and some memories cause exactly the same response in different people. In psychology, Jungian Theory calls this 'Gestalt Consciousness'.

An author can complete a great short story at one sitting, in a frenzied state of urgency, completing it in an afternoon. A single metaphorical image can deliver such emotional passion that the writer can describe in vivid words the exact response any of us would feel under the same circumstance. All the horror novelists do this, conjuring particular images that are so packed with Gestalt emotion that we must respond to them.

Some psychiatrists say they can understand us by analyzing our dreams. Dreams are 'imaging' at its very best, but it is a purely personal experience, using a private subconscious vocabulary. The best multimedia writers understand they must dream with their inner child while sitting at their computer terminals, connecting with software, sounds, 3D objects and video footage.

Without the incredible power of METAPHOR, the product of all of our modern technology will remain mediocre.

METAPHORS: Any writer or artist who can't capture the pictorial essence of the message or icon must think metaphorically. Literary metaphors are powerful because they are compact. A metaphor is a contrasting comparison between things and memories (objects, ideas, processes, feelings, or anything else we use language to describe) from different classes; e.g.

"Heart of Stone", describes a cold distant tough person.

"You are the sunshine of my life”, says a lot about love.

A product name like “Mustang” or "Plymouth Sundance" captures the sense of a fast or expensive car.

A METAPHOR can be done with an ACTION, a VERB or VERB PHRASE:

"He was lucky to slip through their fingers."

"That was a tight squeeze."

A metaphor becomes SIMILE by including the LIKE or AS, e.g.

"She ran like the wind..."

"…as mad as a cut snake…"

“…as dead as a Dodo."

Metaphors 'encapsulate' meaning, making our expression and experiences vivid and memorable. When writing a screenplay, the easiest 'capsules' to describe are charged with emotion; e.g. what you were doing when someone you loved was lost, the circumstance of your first kiss, first broken heart. These examples are easy to symbolize for the audience, because we recall them quickly by clustering the images the same way we always have done before.

This familiarity can be manipulated in storytelling to make an audience believe anything, and then to subvert that meaning with unpredictable juxtapositions. Remember we continually filter our current experience through previously seen images; in the form of our own memories, and the memories of others, by seeing images recorded by them.

Why is it that a really successful Hollywood movie or TV show goes over equally well with an audience in England, Norway, India, China, Brazil or Argentina? Because to differing degrees our experiences are all similar, and it is this dichotomy that the writer must tactfully exploit.

Metaphor, Simile, Analogy, Allegory and Parable; figures of speech, used in conversation, classification and grammar. They are tried and true principles and ready to bend rules, the basic tools on the storytelling menu that you can mould to manipulate and enhance human emotion.

As an interactive writer or designer you must invent the perfect set of metaphorical symbols, not just for your storyline, your service or product, but you must connect it effectively to other enhancing pictures, sounds, animations and sometimes entire databases.

LOOK & LEARN, FEEL & FORMULATE:

To maximize the effectiveness of multimedia, we must learn to COMPOSE AN IDEA WITH THE FIVE SENSES. In interactive composition, we can learn to use our five senses by applying the available technology to each of our sense responses. The user must be able to HEAR AND SEE; the flavour of honey, the smell of perfume, the feeling of wind rushing up a mountain.

Are there more than two multimedia senses? The 'touch' of the navigational interface, different kinds of software 'feeling'? Mouse vibrations? What about web-cam experiences?

Human experience and sensations are what all writers must try to convey, and in multimedia we make connections between the interactive messages and the user’s physical senses as they interact with their computer. All of the senses can come into play and ultimately you're mixing and matching sights and sounds to convey the best experience to your user.

“I smell pictures more than see them. And I hear pictures more than I smell them.” Mick Rock, photographer

IMAGINE NEW WAYS OF COMMUNICATING - Examples of alternative sensory descriptions:

1: The SIGHT - of what an echo looks like, or the flavour of wine.

2: The SOUND - of what a secret is like, of greed, or a sunset.

3: The TASTE - of agony on the battlefield, or a honeymoon night.

4: The SMELL - of hunger, of passion and despair, or wealth and poverty.

5: The TOUCH - of a marriage ceremony, a first day at school, a first date.

REMEMBER TO THINK WITH YOUR SENSES: Writing stories for multimedia requires us to provide possibilities for answering questions, ultimately supplying a range of choices that allow positive and sometimes negative outcomes for the user to experience. The writer’s task is to convert their impressions from all the other senses into a stimulating sight and sound activity, 'transforming' their own sensory input into a visual and auditory experience for the user.

Some interactive applications hold up a mask, some hold up a mirror. Writers must make the user wonder “How did they know that about me?” Memories of images and sounds, anticipation of outcomes, our virtual senses and reality overlap. Remember, if you aim for the general you may get nothing, but if you aim for the specific you may strike upon the universal.
Show an idea rather than an object, and show a dream rather than an idea.

EXPERIENCE WITHOUT SENSATION: Virtual experiences are no replacement for an actual event in real life, which is what makes interactivity such a fragile estate. When you read a book, you know that the page will stay there until you turn to the next. In multimedia, there is no such reassurance; the server may die, the page disappear or the browser crash.

The worldwide web-is not as familiar as a book, and some users are very wary of the new spaces they explore. They know that virtual reality is not real, and it is sometimes perceived as high-tech 'trickery'.

Solving these problems is about being USER-FRIENDLY.

An interactive writer must keep the user comfortable by providing visual equivalents for their immediate context; e.g. the metaphorical symbols like the magnifying glass and trashcan on the Apple desktop. Point and click!

Current software relies on only two of the physical senses, SOUND AND VISION. Who knows, soon it may be others.

Planning is the number one priority in developing interactive applications, so how can we best describe flavours, odours and other sensations with only sight and sound?

NAVIGATION - DIMENSIONS & METAPHORS:

Like any map and compass, clear multimedia navigation should offer the user an easy means of traversing the virtual environment. It is a mainly visual medium and there is a lot of layering of information in multimedia development, but sometimes a picture can’t say as much as one well chosen word. How we arrange our words and deciding what other assets reinforce their meaning is the writer’s task.

Clearly structured applications offer consistent navigation options. Navigation involves the careful arrangement of subject matter (DIMENSIONS) into logical order and provides the means (METAPHORS) to access that content, using visual icons as tools. In this case, DIMENSIONS are all the types of information available, arranged into layers. Making models with index cards, images or objects can be very helpful in representing a clear view of these structures.

In multimedia development the word METAPHOR has a deeper, more abstract meaning than in writing for films or novels. They represent the user’s virtual senses, and also how the virtual world presents itself to the user; e.g. you may use an enamel bowl in your kitchen, it’s a very familiar object to you and delivers the expectation of a certain outcome - holding fruit.

You may see the same bowl in another location, creating for you the kitchen association, but this bowl is used in a laundry, it's full of soap and there are many other bowls in many different rooms. The X-box Eye Toy is an example where the user actually becomes an onscreen metaphor, manipulating other metaphors by actual hand operation in mid-air.

These interactive tools help the user understand both their movements in the document, and also within the overall structure of the INFORMATION SPACE.

This is the most important part of any design. An OVERVIEW DIAGRAM is like a satellite photograph of the information space, developed as an overall structure or map of all the interactive possibilities. This overview can be developed into a functioning 'display' icon to represent the information space.

The use of these 3D maps in gaming navigation has become very complex, and the metaphors are literal representations of objects; e.g. a machine gun, a racing car. Writers can represent any experience, any feelings – violent, poetic or otherwise, in an infinite combination of realities. These activities generate new and unique visual languages, fresh informational symbols, demanding various critical responses from the user.

The on-screen visual metaphor should bear an instinctive relationship to the activity, pushing and clicking on icons; e.g. the 'Grabbing Hand' on the Mac Desktop. Instant access to navigation instructions is always a good idea, so supplying flags and help balloons are easy options. All the interactive elements should play a roll in achieving the purpose of the application.

When conceptualizing complex navigation metaphors, try to think of movements or sensations that are synonymous with the subject. They should appeal to the target audience and match the style and mood of the project. The aim is to encapsulate the most sensuous aspect of an idea, give it a physical shape and feeling, and then represent it as an active graphic.

Consider TIME:

All cultures and philosophies about the universe assume that time passes, perhaps at different rates of speed. An attractive aspect of multimedia, like film and radio, is the opportunity it provides to edit, transpose and transform this temporal sense into tailored dramatic cadences. Time on the screen is much removed from actuality; e.g. time-lapse photography compresses experience, slow-motion extends it.

Television drama is a fabrication, evoking events that never were in a time and place that never was. As in film editing, users experience 'real' time only for the duration of scenes, whereas by cutting and joining shots (or making links) we can radically abridge time.

Timing also refers to the speed that sequential images appear, or the time an interactive element remains on the screen. By editing time we seek to invest human and conceptual relationships with drama and meaning, expressed in terms of cause and effect.

TEMPORALITY: Historically, time has been thought of as an unstoppable linear progression, the flow that moves the universe. Increasingly however, time has taken on qualities of space, becoming a 'fourth-dimension' that is mappable, the implication being that perhaps, like space, time can be entered freely and traveled in more than one direction.

Musical composer John Cage creates temporal structures with sound. He introduced indeterminacy to the process of composition and performance. His Imaginary Landscape, No. 1 (1939) features an ensemble of performers, each of whom turns the dial on a radio for some period of time, waits for a period of time, then continues. The times for each were predetermined by a chance procedure.

In virtual reality, it is a visual presentation that develops over time. While the user may influence this process, it can proceed with or without user interaction. Media elements, which may be images or chunks of text, are added to the screen in a definite rhythm.

Digital media naturally affords this new ability to compose visual work that is temporal. Time itself becomes a form of information, meticulously recorded so that moments can be recreated, understood and relived at will. People, places, and cultures can be archived, and the archive itself becomes a new present, serving to establish connections between the past and the future, navigating within information while looking at a simulation of a 1000-year-old object.

Another feature of time in virtual reality is that information’s linear nature can be broken up, or seamlessly glued back together. For example, these notes you are reading happen to you during a real time experience, one subject after the other, but they were created hypertextually, and you will never know which were written first, therefore making time irrelevant.

LATENCY is an element of temporality that refers to the amount of time it takes for a unit of information to be delivered to the user; e.g. strike a note on a synthesizer keyboard, the sound builds from silence and fades away. A delayed response can create a sense of drama; e.g. the Cheshire Cat in Lewis Carroll’s Alice’s Adventures in Wonderland (1865) fades to invisibility, leaving only his grin, and then even that slowly fades as he's still talking.

Time itself can be an important carrier of information: we use it as meta-information to discern the direction from which data (for example sound) is coming from. In place of time-delays arising from the accidents of place, we could use time-delays as an analogue of other qualities; e.g. the remoteness of one idea (as embodied by an object, building, etc.) from another.

Even what is called 'REAL-TIME' interactivity is usually a fractured, 'unreal' version of time. We usually experience time in multimedia through the duration of pages or screens, which contain information or scenes. Computer software commonly supplies the user with visual indicators that show the status of functions in progress; e.g. download time-bars, hourglass symbols, countdowns.

The means by which time can be manipulated include:

• When the script refers to the duration and sequence of events, in the past, present and future.

• Assembly of images and sounds, fixing duration, rhythm and sequence in accord with scripts.

• The time within pages, shots or scenes can be recorded or distorted by technique and effects.

• When sounds (speech, music, effects) refer to events past, present or future.

• The degree to which time is modified by optical effects; e.g. reverse, slow-motion, looping, etc.

• The mode of presentation of the product to the user.

Consider IMAGE:

It’s a fact that everybody understands the same idea by recalling totally different memories that help interpret our current experience. The smell of a burning fire may trigger a pleasant memory of a BBQ in one person, reminding another of a tragic loss.

"…you will have learned personally the extent to which experience (love, fear or hope) is filtered through ‘already seen’ images."
Umberto Ecco

SPATIALITY: The use of fully dimensional space is one of the defining characteristics of virtual reality, especially in the context of transcultural software development. In real life space can be physical, but it can also be metaphorical. For example:

• HEIGHT is used to indicate 'more' of something (metaphorically, when no real height or quantity is involved, such as a high temperature or a high note). Thrones are higher than commoners' seats to show social power. Large architectural spaces are generally felt to invoke awe, while small ones may be oppressive or cozy, depending on other factors.

• PROXIMITY and ALIGNMENT in 2D are used to show which options belong together. People in social situations use proximity to show how intimate they want to be with others in the space. Orientation is also used, facing someone or turning away from them.

• Using different length lenses (telephoto, wide-angle, etc;) creates different kinds of perspective, so the way scenes are shown alters the viewer's perception. A distant telephoto shot could give the feeling that a character is being observed from afar, while a closer wide-angle on the same subject may suggest the character is aware of it.

“It may be that all human beings have the same perception of space at the biological level of perception. But certainly every society uses space differently, technologically and artistically.”
J David Bolter 1986, Turing's Man - p80.

Bolter's 'may be' is important; the two opposing schools of thought can be characterized as:

• The perception of space is a culturally influenced phenomenon.
• Space is perceived identically, as a universal human phenomenon.

Do we perceive space according to universal optical and perceptual principles on which social and environmental conditioning has no effect; or do we acquire some aspects of our perceptual system from external influences, or by how it's represented? If the very perception of space is culturally determined, a new set of variables enters the equation. What is common between us?

ICONIC REPRESENTATION:

The view-pointless view. We assume that a viewpoint is necessary, and indeed a view without a viewpoint seems nonsense. However, children's drawings and other art forms provide just this; the objects they display carry their canonical views with them as properties of the objects themselves, so that human beings are always displayed full-face, animals are viewed in profile, rainbows are always visible throughout their length.

In a sense, this is what users are familiar with in an ICONIC interface. Flat symbolic representations.

Consider LIGHT: Interactive multimedia is fundamentally a visual medium, being perceived through the agency of light. In spite of this, many multimedia applications are created with little regard for the dramatic potential of light.

Lighting should be incorporated into the fabric of the script, allowing the programmers to fully utilize their resources. If the script doesn’t call for shifts in mood, time of day, weather or locale, the programmer is obliged to simply illustrate the facts rather than bathe them in atmosphere.

Consider the MEANING OF COLOUR: Some colours have specific meanings, due to customary use or for cultural reasons. Sensitive use of colour is very important, due to the wide diversity of interactive users. In western cultures, COLOURS commonly have these different meanings:

• Red – stop, danger, blood, passionate, stirring, cherry flavour
• Green – go, environmental, sickly, restful, apple, mint or lime flavour
• Blue – cold, peaceful, male, challenging, spearmint flavour
• Pink – cuddly, peaceful, female, gentle, strawberry flavour
• Yellow – warm, cheerful, zany, sunshine, banana or lemon flavour
• Orange – tropical, zany, anxious, hot, exciting, orange flavour
• Purple – imperial, high quality, snobby, alluring, wildberry flavour
• White – pure, clean, good, bright, trustworthy, vanilla flavour
• Black – evil, rebellious, death, mysterious, licorice flavour

Our expanded understanding about the biological aspects of communication allows us to invent new metaphors for things other than objects. Different types of form and movement, specific body language, colour psychology and aural stimulation can have a predictable effect on the patrons of a hamburger restaurant; they can also trigger a predictable response within a user.

LANGUAGES – GOING GLOBAL:

Writers often assume that the audience’s experience will be in common with their own, that our understanding (RED MEANS DANGER) will always be the same, but this isn't always the case. Presuming that everybody can read English is an obvious mistake, even in predominantly English speaking countries; e.g. problems of illiteracy, etc.

Well designed graphics and symbols can solve these problems. Interactivity can break through very strong international barriers; e.g. a Japanese web-site could be read in traditional text, with an English translation, or Spanish, Chinese, French. When designing multilingual applications, check text area and font sizes with content from all relevant languages, to make sure there is plenty of space on the screen or button. An Arabian or Chinese font has more character sets, and it may be necessary to use clever translations to shorten multilingual texts.

Why does a French accent remind us of the Eiffel Tower? Because an accent suggests a different culture. Writers should think about how different nationalities and their associated accents can suggest certain 'typical' (possibly negative) racial stereotypes; e.g. German 'Scientists', French 'Waiters', Italian 'Gangsters', American 'Cowboys', Arabic 'Terrorists', etc.

Consider how the incoherence of a character’s speech could suggest; stupidity, recent dental work or a punch in the mouth. Also consider dialects and parochial vernacular; e.g. idiosyncratic Australian rhyming slang like "Have a butcher's hook" (look), or the classic ‘strine "Gidday mate". There is in fact a definite form of 'prison-speak' for inmates of penal institutions, when language is pronounced through tight lips to avoid surveillance cameras.

Dialogue, sounds and music can work on multiple levels:

1: Actual 'real-time' dialogue, (External voices).

2: Character-particular Voice Over dialogue, (Internal voices).

3: Sound Effects and Musical Soundtracks.

4: Or simultaneously; e.g.

Violins play, dialogue in VO: “She looked lonely, so I said...”
Actual dialogue: “Hi... I haven’t seen you before.”
FX: “CLANG!” as she hits him over the head.

In multimedia, the user can switch all these options on and off, and do it in Swahili as well. Alternative dimensions can be included or excluded at the click of a button. Presented together or separately, these possibilities offer many innovative FORESHADOWING alternatives in your interactive scriptwriting. Cross-cultural stories can be designed to play out in a multitude of ways, depending on the culture being targeted, or a selected character’s environmental circumstances.