Expressing yourself with sound: An Interview with ROLI Sound Designer Rafael Szaban

Since I was a teen I wanted to create electronic music. Somehow the path of wanting to become a musician led me to become an audio engineer working in recording studios and doing live sound. While I'd be swearing at having to align and calibrate a reel-to-reel tape during the day or having to zero a massive mixing desk, I'd be making my own music and sounds at night.

I play keyboards and keep on promising myself to finally master the Native American flute that I bought years ago. My desire to create music has allowed me to do music for jingles, release some albums, do loads of engineering work by recording artists, mixing and mastering albums, but overall, it has allowed me to focus on making my own sounds. This has always been at the centre of all my music-making work. At one point I also taught sound design at the University of Westminster in London.

One day Angus Hewlett (of FXpansion) messaged me asking if I'd be interested in some sound design work for this new start-up. He said that they were working on a new type of MIDI controller. He said it has polyphonic aftertouch, and as soon as I heard that, I enthusiastically said 'yes'. Anything with polyphonic aftertouch piques my interest. Little did I know what was in store awaiting me...poly-aftertouch was only half of the joys this new instrument offered. So I arrived at ROLI in November 2012 and was greeted by an early prototype of the Seaboard GRAND. This early prototype looked like some sort of alien object. An artifact from another world. It was truly impressive and I found myself in MIDI controller heaven.

I was initially invited to make 6 to 12 presets. We sat there in the ROLI studio with Jean-Baptiste Thiebaut thinking of what kind of sounds these should be. Only 12 sounds and no real MPE sound engine to make them on. The synth that I was to work on was a modified version of FXpansion's Cypher. The early days of MPE were like the wild west. A lot of experimentation. Needless to say, it soon transcribed that the path ahead was going to require a long term sound design position. However, my role didn't focus just on sound design. Working in a start-up, I got a chance to do multiple tasks, from product testing, product co-design (Equator1) artist support (making bespoke sounds for artists), assisting in VIP Seaboard demos in our studio, going to Musik Messe and NAMM events, etc.

Our MPE instruments appeal to a very wide range of people. It's not surprising, since the Seaboard is built on the familiar 'keyboard' design. If you know how to play a piano keyboard, you know how to play any of the Seaboards, but of course, the more complex MPE patches require a new approach. On the basic level, a keyboard player gets a Seaboard and they can play immediately.

When it comes to the LUMI range of products it's still easier since these are based on a real, mechanical keyboard as opposed to the continuously expressive silicone surface of the Seaboards, although the LUMI keyboard still provides a flavor of MPE.

Over the years I've observed all types of musicians and non-musicians approach the Seaboard and each time I'd see new ways of approaching that malleable, silicone surface. One famous person even played it with his forehead. I guess the Seaboard might be the only MPE controller where that's possible without injuring oneself.

Then, there were artists into Ambient who would lay their hands on the Seaboard surface and simply massage the whole surface left and right, in this wave-like motion, while also adding pressure. A very sensual experience.

Some of the most highly skilled keyboard players would come up with new Seaboard playing techniques, and I was privileged to see one such technique being born right in front of my eyes. It concerned the act of voicing each new chord in a chord progression, by gliding fingers into the next chord, one finger at a time or several fingers at a time, until the new chord is formed. When done with moderate speed and precision, it is one of the most beautiful sounds I've ever heard.

The methodology of MPE and non-MPE patch creation is definitely different, so we could certainly speak of having a separate process for each. There are specific editing aspects that we need to pay attention to in order to create a balanced MPE patch that plays well. In fact at ROLI/Luminary we need to provide two versions of each preset, one that is an MPE patch, and one which is a normal preset that can be played on a traditional MIDI keyboard and consequently the user should select the appropriate preset browser filter tag ('MPE' or 'Standard') to select the correct preset type for their currently used controller (an MPE or non-MPE one).

When creating an MPE patch we need to be mindful of the 5 Dimensions of Touch and how these translate to the silicone surface of a Seaboard. There is a lot of flexibility in how a patch can be set-up in terms of its response to all those 'dimensions' versus the actual hand gestures. This flexibility comes from the 'expression-curves' which allow to scale modulation in specific ways. This is very important. In a traditional preset made for a normal MIDI keyboard we tend to think of the mod-wheel and aftertouch as the two primary controllers. These are separate and using one does not affect the other. However, if we turn to the MPE side of things, we have 5 polyphonic 'controllers' that are interconnected: STRIKE, GLIDE, SLIDE, PRESS, LIFT.

We start with STRIKE, which is velocity. As soon as we touch the silicone surface of a Seaboard, we may be exerting some pressure and by doing so we begin to interact with PRESS, i.e. polyphonic and continuous-pressure. From here we may start to interact with GLIDE or SLIDE, that is, we can apply pitch-related gestures (GLIDE) or continuous-modulation gestures along the key wave (SLIDE). All of this may happen with the gentlest of touches. But then, we have the last dimension, one which is least talked about, and this is LIFT, i.e. release velocity. In order to use LIFT we will be interacting with pressure (PRESS) and if we remember that pressure here is of the continuous type then we start to realize that it adds another layer of intricate relationship between these 'dimensions'.

The question is, how should these interact and how should the musician approach the patch in order to make this interaction musical? How to make it controllable so that all of those dimensions add to our performance rather than fight with what we want to achieve? We do this with these 'expression-curves'. These are extremely important, and as I say this, I have to mention how little attention is paid to these in most MPE instruments.

Here's an example with the Seaboard RISE. In order to create a good SLIDE modulation response (CC74) on the keywave, we need to think about these 'expression-curves'. We need to match the shape of this 'expression-curve' with the length of the keywave and the ribbon above it. The way we do it depends entirely on your artistic preference. It depends on what you desire the modulation to do on the keywave.

I've developed a specific preference for an expression-curve shape which allows me to slide along the keywave and get the full modulation amount, without having to access the top ribbon. This is important for various playing techniques. For example, if I play a chord, I may want to slide up and hear what is assigned to the SLIDE dimension in its entirety without having to slide off the keywaves, onto the top ribbon. To achieve this, we need a specific 'expression-curve' shape.

These editing tricks come into play at many junctures while editing an MPE patch. 'LIFT' also needs several very specific expression-curve shapes in order to allow us the best expressive gestures. Without these specific shapes, LIFT may in fact get in the way of your normal playing. It may be distracting. So we need to 'balance' it somehow. Expression-curves are the key here. And it goes on like this, even on the GLIDE dimension (pitch-bend) where maybe we want to reduce the LFO-rate first introduced when we SLIDE, but while we GLIDE, we may want to quickly 'kill' the LFO modulation. To do it smoothly, we need a very specific logarithmic expression-curve shape.0.

I'm hoping to illustrate that the way we should approach MPE patch editing vs non-MPE patch editing should carry with it familiarity of your MPE instrument (and the actual synthesizer/sound engine) and have a certain level of playing skill. This should translate into how you approach all those dimensions. It should inform how you edit the sound to make it expressive and reflect any specific playing techniques that you wish to explore.

There are editing tricks that I will gravitate to for a specific sound. Some techniques are now firmly engraved in my brain and when editing certain sounds this 'editing muscle memory' kicks in. This certainly allows me to make a patch quickly, but it's also a trap. It could be described as playing that same chord progression you've learned years ago. You know it by heart, but it becomes tedious. So it's important to find new ways of creating sounds, or rather, it is important to question and keep refining your process.

This topic actually reveals another dilemma. Is a newcomer to MPE going to understand how specific MPE patches should be played? How should they be approached? If a patch has LIFT, how is that used? Or how do we use Absolute SLIDE? With Relative SLIDE how to position the finger in order to slide smoothly?

I've observed how people play MPE patches over the years and very often the understandable excitement of being able to play an MPE sound on your brand new MPE controller takes over, and the required gentle touch is forgotten about. If we simply start playing an MPE patch like we play a normal MIDI keyboard, perhaps by hitting the MPE surface hard, or pressing very hard as we slide or glide, it may create all sorts of issues. All that careful balancing of these 'expression-curves' goes out of the window. This is also a reason why we need some MPE patches that are not as expressive as one would expect them to be. There will be instances where we need to curtail the modulation range in order to make sure that the patch is familiar to newcomers.

How much pressure should an MPE 'club bass' patch have? Normally we play a typical club bass sound by hitting the playing surface hard. We use a lot of velocity. However, by applying high velocity we inadvertently press-in as well and this can create unwanted fluctuations in the sound (pressure on the Seaboard is of the continuous type). So from a firm bass pluck, we could very well end up with this fluctuating 'sausage' of a bass, which is not what you may want. Therefore, there are instances where expression has to be limited in some way, perhaps by limiting the modulation-range, or by applying specific exponential expression-curves, or by applying a 'slew' (lag) process at the same time. All of this is done on a per-patch basis. So, there will be patches that have limited expression on purpose, but will the player realize why that is or will they conclude that it's a bad example of an MPE patch? Conversely, there are very expressive patches which require the player to study the patch a little in order to learn it. This is where a 'measly' act of sliding, pressing, gliding, becomes deeply artistic.

I remember one day we were visited by a musician from a classical ensemble specializing in music from the Baroque period. He came in and demoed the early Seaboard GRAND. Rather than play some complex multi-layered patches, this musician wanted to play a simple synthesized reed-like sound and explore the fine expressive nuance of volume and timbre. I found it liberating to observe, since a lot of people want to play these spectacular, complex sounds which are transformed with extreme modulation, but here I was looking at a classically trained musician just taking it easy with this very plain sound that most people would very likely discard in a second. I took that to heart and consequently created a series of sounds that are for the connoisseur of MPE.

One such sound that I know some people liked a lot is created in Equator1 and it's named 'Mellow Duduk'. The whole sound relies on the 'continuous-pressure' on the Seaboard. If you play this sound on a normal MIDI keyboard you won't hear much. The sound will appear broken. This is a fundamental difference between some specific MPE patches and standard, non-MPE patches, some MPE sounds will just not work on a normal MIDI keyboard.

Lastly, we shouldn't forget that a good sound engine/synth is the key to a good MPE patch. But I guess that this is a whole other topic. Just to mention briefly, COMB filters are very useful when applied to a sample at the GLIDE stage. This is to mask any sample-stretching artifacts that the sample will suffer on extreme pitch-bend. A COMB filter can help.

That's a great question. When sampling, we need to account for the fact that there are certain articulations which musicians will want to apply themselves. For example, take the 'vibrato' articulation. One of the joys of an MPE MIDI controller is that we can add our own vibrato and do it in a very fluid and natural way. So, if we have samples that have vibrato 'baked-in', sampled as part of the main sound, it then becomes impossible to add your own vibrato in turn. This is particularly relevant to solo instruments like bowed strings or reeds. Although, when it comes to ensemble sounds, sampled vibrato is very beneficial. The way multiple players create vibrato in ensembles is very different to how it would sound if we were to wiggle all our fingers in a held chord to create our own vibrato (when using an ensemble sound that was sampled without vibrato). So, we need to think about all of these aspects when planning how to present sampled acoustic instruments.

This goes deeper when we think of sustained acoustic sounds and how we represent the main tone of the instrument. This touches on the topic of continuous-pressure, or continuous-slide and how we obtain dynamic modulation (volume and timbre).

Here, we need to find the best tone, the layer that will constitute the main sound when we trigger the note. This is only one layer and this single layer will be all we have to play with. The reason for this is that we can not cross-fade from a sustained layer to another sustained layer when using pressure or slide without hearing that crossfade, at least not in the Equator2 engine that I'm focusing on. So, when sampling we need to find that tone which will be strong, bright enough, loud enough and then by using synthesis we can alter the character of the sound when applying pressure or slide. Obviously this is not ideal, so I'd say that the field of representing sampled instruments as MPE patches is yet to be refined and there is plenty of scope for companies to come and explore new techniques.

At this juncture I need to mention that the hybrid approach can give some truly excellent results as shown by a true master in the field, Edmund Eagan and his EaganMatrix audio engine. Edmund has created this amazing string sound which uses a tiny 100ms portion of a bowed string sample, analyzed and its spectra functions as a source for additional synthesis in the (extremely) deep EaganMatrix audio engine. The final result is astounding. I guess this is an alternative to the purely sampled approach.

Edmund Eagan Performs

A cheeky question, since I can not reveal any future plans, however, all I can say is, watch this space.

Firstly, it would be great to see MPE instruments being embraced by more musicians. I'm saying this as a musician myself, one who has seen the benefits of MPE in my own music. For example, playing a bass line on a normal MIDI keyboard now feels restrictive to me. I have to play it on the Seaboard RISE. And it's not because I desire my bass line to have some ridiculous amounts of wild expression, but rather, I want to imbue my bass line with tiny, organic expressive gestures, which happen all by themselves.

There is a fluidity and a flow when you develop some skill of playing MPE instruments. Every time I play the same bass line or indeed a lead on the Seaboard, it's different. Something that I can't get from a MIDI keyboard and a mod-wheel: expressive legato transitions from note to note, tiny pitch bends, natural vibrato that is different from note to note, dynamic variation via pressure... This is just so enjoyable and natural that I wish more electronic musicians could experience it. So this is just on that experiential level, but of course in order for people to embrace MPE like they've embraced all of the pad-based MIDI controllers, the price has to go lower, so here's to less expensive but still un-compromised MPE controllers.

I'd also like to see more software instruments implementing MPE, but do so at a deeper level and not by implementing often incomplete or very rudimentary designs. I wish for more developers to understand the importance of 'expression-curves' and design their MPE instruments in such a way that an average user could use these 'expression-curves' easily, without having to dig into a mod-matrix or hunt through some hidden pages.

MPE instruments depend on great audio engines, so I'd love to see more Physical Modeling audio engines being developed, but with MPE in mind from the ground up. Engines that mix Physical Modeling with samples, wavetables, resynthesis can provide amazing results.

Lastly, in many ways the future has arrived already, at least at ROLI. It came but not many people took notice: ROLI 5 Dimensional drums.

We developed the same 5 dimensions of touch that are available on the Seaboard RISE, but this time for drum sounds and a specific MPE drum controller, the Lightpad Block. When using the ROLI Lightpad Block we could play 5 dimensional drum kits. These were drum sounds that could be mangled live, with your fingers in ways that nothing else is able to achieve, not in real-time and by using only your fingers. These sounds were perfect for all manner of electronic genres. You could trigger the main drum sound and by sliding, gliding, pressing you could transform the main sound in very startling ways. This was a huge resource of happy accidents, of arriving on some beautiful textures in a very organic and fluid manner. I remember making some Wu-Tang Clang drum kits from the sounds supplied by RZA and Mathematics, and loving the way the basic samples could be molded into new textures with such ease. A player's dream.