All that Jazz

Suppose you are interested in music and also have the kind of technology used to make computer-generated creatures in films like Lord of the Rings' Golem. What could you do? The team at Queen Mary have been exploring how musicians have such an intense time when they are jamming.

Jazz musicians when doing an improvised performance for example, seem to be able to lock in to what the others in the band are doing. They are taking part in a performance but all also composing as they go. After a while they seem to instinctively explore new sounds together. What is going on? How are they communicating? It sometimes seems like they must be psychic!

The secret for musicians being creative together, so that they get high quality results and also have a fun time doing it, is to develop a strong feeling of 'mutual engagement'. That just means they are deeply involved both with the music being created and with each other. When that happens they start to push back the boundaries of the understanding they share, both of the experience they are having and of their expectations of what will happen next.

Musicians develop this sense of mutual engagement in a session by combining musical signals with spoken and visual cues. Simple things like stepping backwards or forwards slightly at the right time can make a difference. Unlike when people have a conversation, communication in a jamming session is not so much about passing information, it's about increasing this sense of mutual engagement. Another difference is that these subtle things the musicians do tend to happen together rather than one after the other as in a conversation.

How does film technology help? The way they create computer-generated characters is by using a '3D motion-capture' system. The way this works is that cameras positioned all round an actor modelling the character capture the precise movements of different part of their bodies. That information is then used to control computer-generated characters. If you want to study the subtle ways people behave in groups, like our musicians, the same techniques can be used. All the subtle gestures the people jamming are making can be captured and analysed to see how the intense relationship is happening.

Nick's aim is not just to understand what is happening in such sessions, of course, but eventually to look at how new computer technology might be developed to support those activities.

Some day soon social networking could take a further leap forward, helping people to have intense jamming sessions, even across continents.