Jordan Rudess on Alternative Controllers

Editor’s note: I asked Dream Theater keyboardist Jordan Rudess to share his thoughts on some of the latest and greatest alternatives to the traditional piano keyboard when it comes to musical and timbral expression and control. AS is Jordan’s wont, he sent me a detailed and well considered missive, which was distilled into the article “The Fretless Keyboard” in our June 2015 issue. As Jordan and I wanted to share it with you in its entirety, here is his original manuscript, only lightly edited. --Stephen Fortner
Image placeholder title

Editor’s note: I asked Dream Theater keyboardist Jordan Rudess to share his thoughts on some of the latest and greatest alternatives to the traditional piano keyboard when it comes to musical and timbral expression and control. AS is Jordan’s wont, he sent me a detailed and well considered missive, which was distilled into the article “The Fretless Keyboard” in our June 2015 issue. As Jordan and I wanted to share it with you in its entirety, here is his original manuscript, only lightly edited. --Stephen Fortner

The devices that we walk around with actually have musical expression capabilities way beyond what is possible on most hardware instruments. When multi-touch devices first became available to the general public, I knew that we were entering a new phase of awareness and possibility for creating music. I remember when the iOS store opened up and incredibly creative apps started emerging that opened up new ways of thinking about controlling sound. I started Wizdom Music and created apps like MorphWiz and SampleWiz, where the focus was on merging the world of audio and visuals together as well as offering independent touch control of every note that was played. I had an overwhelming desire to take advantage of this new affordable multi-touch technology and help to change the way we think about making music.

Today on your mobile device, you can do things like play four simultaneous notes while adding vibrato to two of the notes while increasing the volume on one of them and bending the pitch of the fourth one. If you walk into a music store today, you will still be hard pressed to find any instrument in any price range that can do that.
Luckily for all of us as musicians there are some people who are working on bridging this crazy gap and creating instruments that take advantage of the technology and offer musicians the next level of expression.

I think the goal for musical expression is to be connected mentally and physically with the sound of your instrument at every moment in time. We want the sensitivity of every touch (and every thought) to be recognized and have meaning. The human voice is thought of as the most “connected” and expressive instrument to many, so examining it can clarify this idea. We use the power of our breath and vibrate our vocal cords at different frequency rates and control the tone through the natural resonating chambers of our throat, nose and mouth. Creating and controlling sound this way is the most organic musical experience for a human being as possible! Being in touch with the sound you are making with full continuous control of every moment of its vibratory existence is the goal of expression.

To this day there have been quite a few attempts at creating next-generation hardware instruments. The new instruments that I’ve spent the most time with and that stand out in my mind are the Haken Continuum, Eigenlabs Eigenharp, Roli Seaboard, and Roger Linn’s LinnStrument. All amazing inventions! As of this writing, none of them have caught on with the general musician community yet, and there are many reasons why. The biggest of these is that it takes time to learn a new instrument and there are known, tried-and-true musical interfaces that have a wide range of acceptance in the world.

Musicians in general do not want to learn a completely new interface. They might have spent years of dedicated practice on the piano keyboard or the guitar or other instruments and they are not interested or even have time to change that!

Often an inventor will veer away from something standard and try to imagine a new way to control sound. In concept this is very interesting and for those of us who are sonic explorers we often embrace these ideas. The other real issue of acceptance to these new instruments is in the fact that almost all of them when initially released made no sound by themselves and were designed as controllers which then need to be connected to external sound modules or a computer to make sound. Most musicians don’t want to deal with this added complication or just can’t wrap their heads around all that it takes just to get some sound. Even myself, who loves these cutting edge instruments, will throw my hands up in the air in frustration and just go to my Steinway to play some music!

Luckily things are changing! Roli Labs, a startup company in the U.K. and makers of the Seaboard are aggressively taking steps towards opening up this new world of instruments. They are gathering some of the brightest minds and most successful music software and hardware companies to brainstorm about ways to allow more fluid communication between the instruments and the software. One of the challenges is that all of the instruments I’ve mentioned above, send information on multiple midi channels to accomplish the wonderful independent control of every note’s pitch volume etc. There is a lot of work, however, that needs to be done in the music manufacturing community to bring new standards that make this possible and also take the stress out of the setup for the user.

My explorations into continuous controllers started with the Haken Continuum, which was initially released in 1999 by Lippold Haken. I first used it on Dream Theater’s Octavarium album (released in 2005) and then started touring with it afterwards.

I had always dreamed of being able to play definite pitches like on a traditional keyboard easily but also being able to slide from note to note. The Continuum was a dream come true. Although it is basically a flat surface, it was the first controller that gave me the power of a fretless instrument, as well as individual parameter control of every note. I worked with Lippold back in those days on refining a pitch rounding system that has changed the way I play music and even make my apps. The basic idea is that although the instrument is “fretless” we would allow a note to be in perfect tune on the initial point of contact and then even after a slide to another note the instrument could round (or quantize) when your finger stopped its movement. This is a concept that I actually brought into Geo Synthesizer, an iOS app that Wizdom released in 2011. I love being able to slide from note to note horizontally while there is lots of room on the vertical position of a note to express any parameter of the sound. It’s also very sensitive to touch and in addition to the left and right and vertical movement it recognizes pressure as well, all independently. When Lippold first released the Continuum it was only a controller but he realized later on that it would be so much more accepted if it had built-in sounds. These days the instruments he produces have a very creative custom sound engine built in. When playing the Continuum, it feels very organic and responsive to every touch. The Continuum is a quality instrument that is basically built by hand and quite expensive.

A couple of years ago I spent some time with the Eigenharp Alpha. A Very cool looking instrument that was really exciting to me when it came out!

The Eigenharp is a controller that does not have any internal sound engine. It was shipped with its own software that could house plug-ins, kind of like its own version of Apple’s MainStage. The instrument is very configurable and the greatest thing about it to me were the high-tech key-buttons that could sense velocity as well as side to side motion. It also had two large standard ribbons on it, as well as the possibility of attaching a breath controller. The keys were incredible. I could play faster on that instrument then anything else I’ve ever played. I was thrilled that Eigenlabs thought about the physicality of a person’s finger and what kind of material and action would be ideal for human interaction. A limitation I experienced was that the instrument has a very limited amount of physical space to generate other expressive control from its keys because they are quite small. I wanted to do independent pitch bending and things like volume swells and filter changes and found that pretty frustrating. I generally like my continuous controllers to have room for control! That said the breath controller is a tried-and-true controller (although not independent per voice) and the standard ribbons were nice additions to the instrument’s capabilities. One of the other things I found really difficult about the Eigenharp was that there was virtually no visual information available to tell you what was going on. You had to remember the positioning of various lit up keys to guide you.

The Roli Seaboard: I have a personal interest in this instrument, as I am officially the Head of Music Experience for the company! That said, I will be straight-ahead about it and share with you all my thoughts.

When I first heard about the Seaboard I was super excited about the idea of a keyboard instrument that is part fretless instrument! One of the reasons I have personally put a lot of energy into the Seaboard’s development is because I love the fact that it offers the kind of continuous expressive control I desire, in a form factor that is based on technology that goes back to the 12th century. Keyboards go way back--starting with pipe organs and then from there evolving into instruments like the clavichord, harpsichord and pianoforte.

Instead of keys the Seaboard has what we refer to as Key Waves, hence the name Seaboard. The first release of the Seaboards presented some challenges that are now actually aggressively being pursued by the company. One is that as a controller only, it was not that easy to get it to work with software synths. This is the same challenge that exists on all the new breed of controllers, as to achieve polyphonic aftertouch (and other per-note parameter moves) you have to create a different MIDI channel for every note you play. A few software companies like Spectrasonics, U-he, and FXpansion have made that friendlier, and Roli is laser-focused on working with the music community to get more companies thinking along these lines, which will benefit everyone creating new expressive instruments.

Another growing pain has been that the initial Seaboard that many keyboardists tried were kind of hard to play and demanded a strong touch to get sound. For me, having studied technique at Juilliard, I viewed that as a challenge for my fingers but for most everybody else it’s a bit off-putting. Thankfully as of this writing, new sensors have been designed and are in the current versions of the Seaboard. This is an area of great interest to me personally and I’m involved in continuing to get this right! The feeling of pressing into a Key Wave on the Seaboard is wonderful and very organic. It brings new meaning and control to polyphonic aftertouch. All of us have seen keyboardists trying to do vibrato on a traditional keyboard by wiggling a finger side to side as we held a note, but on a conventional keyboard this does nothing. [Some home console organs in the 1970s and early ’80s (such as the Yamaha Electone line) had a third, monophonic psuedo-synth manual of mini-keys with a motion sensor for the whole keybed, so you could in fact perform vibrato in this fashion, but this was nowhere near as sophisticated as what Jordan is after. --Ed.]

This kind of motion on the Seaboard means a lot though and is also very satisfying! Roli to this date has not implemented the vertical axis (Y-axis) of a Key Wave but if I have anything to say about it (and I do) we will see to this as well in the not too distant future. My feeling is that when Roli figures out how to release an instrument at a price that people can easily afford the Seaboard will stand a great chance of really catching on in a way that has to this date, has eluded other multi-dimensional new instruments.

Roger Linn’s LinnStrument feels like it was made for me! [Read my full review of this controller in the June 2015 issue. --Ed.] The LinnStrument is like a hardware-real life version of Geo! The layout is a grid that is similar to a guitar where the horizontal axis are like frets where you can play chromatically or slide from left to right, and the vertical axis are like the different strings. On the LinnStrument the interval between the “strings” can be tuned to any interval you like. On my Geo Synthesizer app, the interval is fixed to an interval of a perfect fourth--like on a bass guitar. I find the layout of the instrument to be very friendly, although it can be intimidating to a lot of people because it requires thinking in a whole new way! One of the great things about this type of interface is that once you find a scale or chord shape it will be the same throughout the instrument. All the keys can send out different messages in each of three dimensions. The X-axis controls the pitch, the Z-axis is pressure, and the Y-axis can send timbre information such as filter cutoff. It’s also velocity-sensitive. The square key pads are small, so there’s not a lot of physical space like on a Continuum, so depending on what you want to achieve expression-wise, it might feel limiting. The keys have a good feel to them and the lights on the keys can be configured however you like to help guide you. There is no display, so it gets a little tricky to setup parameters but Roger’s website is complete and all the information you need is there. It is strictly a controller and makes no sound by itself.

It’s possible to plug the LinnStrument directly into an iPad right out of the box, which is great, especially since many iOS synth developers are forward-thinking and have designed their synths to accommodate independent voices (MIDI channels) of parameter control. ThumbJam, Arctic Keys, and my own SampleWiz work easily in this regard with almost no setup. There is also a downloadable Logic file available that gets you up and running quickly. The fact that the LinnStrument is lightweight and small in size as well as affordable is groundbreaking. Putting this level of control into this type of package is a breakthrough in the multi-dimensional controller world! Personally I’ve been taking mine everywhere I go! I find I can really burn on the LinnStrument, and certainly for a guitarist or keyboardist who has played some guitar, the note layout will be mastered quickly. Like the other instruments I’ve been talking about, the challenge is that the rest of the synth world, hardware and software, is not totally ready to receive the multi-channel information the LinnStrument outputs, so you shouldn’t expect to play any software synth you may already own easily. Although the LinnStrument is different, I think it’s amazing and would make sense for a young person to learn it as their first instrument, and for someone who already plays an instrument to learn as a second.