One of the reasons to look beyond conventional controllers for music (like the ubiquitous, piano-style black-and-white keyboard) is that these controllers have a certain range of expression. But these constraints also impact people with different physical abilities. The piano assumes a certain kind of physical facility, and even as it makes playing easy for people with that facility, it prevents others from being expressive.

Case in point: reader Niels Schuddeboom has been looking for alternative controllers or software that would allow him to overcome cerebral palsy:

I am looking for a solution to make it easier for me to play complex compositions…Due to my disability I can’t use my fingers all together.

I have cerebral palsy, from a musical perspective, this means that I have very stiff movement of my fingers and wrists. That makes it very difficult to play scales, Chords, loops – in other words, it is very hard to play a complex melodies and completions. When I move one or two fingers, naturally the others go with them.
Of course, there are several alternatives. One of them would be to use the mouse or to use BIAB-like software. Problem with the first solution is that you quickly lose the big picture, the second solution, BIAB like software would in my opinion, kill the nature of musical expression.

About the music I like to play: I actually have a very wide taste, from abient to jazz. Not too technical. But I have to say I haven’t played for a very long time because my mind could think of many things but physically it became a frustration. 2 companies were interested in helping me out, but either their custom solutions were way above my budget or they didn’t get back to me.

Okay, readers: got any advice, even for a place to begin researching? Have any of you had to find a way to adapt musical playing to overcome a physical disability? (I certainly know pianists who have had to redevelop their keyboard technique after an injury — either an accident or repetitive stress.)

It absolutely provides an added impetus to think more broadly about how we play music, in terms of hardware and software. Look forward to hearing what you think.

Photo by talented Flickerer Patrish. Nope, not a digital instrument — but, isn’t that funny, notice the uncanny resemblance to the primary interface on virtually all synths?

  • http://www.koen.smartelectronix.com Koen

    Some months ago, I received a request from a paraplegic drummer to help out with the setup of my KTDrumTrigger plugin. Since he couldn't use his legs, he wanted to control the bass drum by puffng with his mouth, and thus trigger a MIDI note on his electronic drum module through my plugin.

    As for help with melodic/harmonic aspects of music composition, the only thing I could think of is to use some kind of more intelligent harmonizer: not just a "fixed parallel interval" harmonizer, but one that recognizes the key you're playing in and uses fitting harmonizations according to some of your settings. I know that zplane recently released something that can do that (plugin called "vielklang"). It's still automization of some parts of the composing process, but it's a bit more intelligent than most previously existing harmonizers (don't know about BIAB though), and it works on audio, not MIDI, which means you could use it on any type of tonal instrument.

    To just lay down some simple melodic lines, a pitch-to-MIDI convertor may help out as well (Google can probably find some for you). But I guess that's not where you have your problems.

    Other than that, no idea really…

    Hope this helped…

  • Chris

    Since I'm a keyboard player, I'm going to have a narrow scope, but there are plenty of instruments and controllers that don't require a lot of dexterity of the hands: pedalboard (as for organs), Haken Continuum, theremin, maybe the Ondes-Martenot. Of course, the pedalboard and Continuum are just controllers, so you could trigger whole chords, and solo with one of the analogue instruments. And the continuum has an incredible range of expression with just a few fingers.

  • http://morgansutherland.net Morgan Sutherland

    Some interesting ideas to be found in the MIT MediaLab's Human 2.0 conference; particularly Tod Machover's lecture: http://h20.media.mit.edu/

  • http://cameratoss.blogspot.com clickykbd

    Even the adaption of sensing technology to build electronic instruments seems applicable. For example, the Laser Koto:

    Article
    http://www.wired.com/entertainment/music/news/200
    Video
    http://www.youtube.com/watch?v=L9kU3vgfVXA

  • thesimplicity

    I was raised to be a performer first and foremost, so when I severed the nerves in my left index finger a few years back it just about killed all my urges to make music. Nothing can describe the frustration of not being able to physically express an idea that exists so clearly in your brain. But that's not to say a dead finger is anywhere on the level of cerebral palsy. It's more like a set back by comparasion… I'll never be able to do sweet two-hand tapping licks again, but I have enough balance restored to feel comfortable at a piano.

    Having said that, for a while I was really into alternative controllers that didn't rely on the use of fingers. The only real cool one I came up with was a MIDI harmonica type design that was used in conjunction with a pedal board (to set the key, along with other pitch modifications). It seemed fairly intuitive to me… all you really need is control of your tongue and your feet. I never got it fully functional (due to problems with my Teleo block), but I'll see if I can dig up documentation on it and post it to the forums over the weekend.

  • Damon

    How bout a keyboard type instrument with only 2 keys that are more like small levers. Sort of like piano foot petals for your hands

    The keys could also be even 6 or 8 inches long, on semi tight springs for a quick return, allowing you to find expression with your arms and wrists.

    But on 1 or both of the levers, you could set up 1 or 2 simple buttons, that work within the confines of your movement. Not sure if you would press them or squeeze them.

    But 1 lever would be essentially a pitch wheel, and with the other lever plus buttons, you could activate or deactivate the pitch on the other lever. Depending on the level of movement available on your more flexible hand, you could experiment with 1, 2 or 3, buttons. Possible allowing you to trigger chords in some fashion.

    Now, this is just a casual idea, but maybe someone with more expertise could fine tune the idea.

    3 bars maybe, if you could manage more than 1 bar with 1 hand?

    Might each bar have an ability to tilt left and or right, allowing more means of expression.

    Maybe 2 or 3 paddles rather than levers, allowing you to set up several buttons on each paddle, so you could play a multitude of combinations?

    Just an idea.

    Blessings,

    Damon

  • Cubestar

    The korg pad kontrol has velocity sensitive pads with plenty of space and an XY pad that might be useful.

    Next up might be finding someone to make custom PD midi interfaces for touchscreens?

  • http://www.multiwind.com Chris Graham

    I am developing a family of wind instruments that can be also used as hands-free control devices, and as hands-free musical instruments that should be quite richly expressive.

    See my website at http://www.multiwind.com. For an example of how it would be used hands free, see slide 15 in the presentation on my website for a device mounted on a microphone stand for hands free use.

    The retail price is targeted to be well under $400 and they should be available within a year.

    You can contact me: chrisgr at multiwind dot com with comments or for more information.

  • scook

    I too have a severe disability, a type of muscular dystrophy. Making music has always been extremely frustrating for me, and I have tried everything. When I couldn't join band in school, I attempted both the guitar and harmonica. Neither worked due to my lacking strength and dexterity. More recently I've tried a simple Japanese flute called the shakuhachi with similar results even I designed my own custom flutes. Electronic music has been my latest attempt, using sequencers and such. I bought a midi keyboard and almost immediately regretted it- I should have know I wouldn't be able to play at the speed and complexity I want. I would really appreciate your suggestions. Being able to play written music live is my ultimate goal. Thanks for posting this article- some of us could really use some help.

  • velocipede

    I do not know much about it, but it sounds promising.

    http://windcontroller.googlepages.com/

  • http://slonwheels.blogspot.com Niels Schuddeboom

    Thank you all for the comments, great to see that! Keep them comming, software stuff is also very welcome! @ Chris: Continuum is indeed very cool stuff, too bad that my pockets are not deep enough…;)

    To set this all in a broader perspective, I hope, since music is to some many people a great way to express themselves, time will come that software developers like steinberg and propellerheads (and hardware folks as well of course) will look into the possibilities to make their 'ware' more accesible. @ Steinberg and Propellerheads and others I would like to say: I am ready to serve you, I have lots of ideas.

    I'll start researching again! Thanks again,

    Niels – The Netherlands

    PS. And Peter, of course thanks to you for bringing this on the frontpage!

  • http://www.twistedsessions.com/synthetic synthetic

    try the Kaoss pad…

    there are some synth presets in the effector that can illustrate how you can use it..

    it can also be used as a midi controller and you can set up the X as pitch and y as velocity.

    if we can set up the scale already in cubase's input plug ins then set the x controller to go up and down the scale.

    otherwise there are some developments in cyberkinetics. its a pad worn on the forehead which controls a computer.. although the technology is still in development I'm sure the companies that are developing them will be interested in exploring their musical applications.

    hope this helps.

    Cheers and good luck,

  • anon

    Thi midibox community have helped with making custom controllers for people with special physical requirements before. I'm sure if you visit the forums there, the community could help you to make something as per your imagination. Good luck!

  • teej

    How about a Lemur? There won't be any tactile feedback but you are completely free to arrange the interface however you see fit.

  • Damon

    scook

    I too have a severe disability, a type of muscular dystrophy. Making music has always been extremely frustrating for me, and I have tried everything. When I couldn’t join band in school, I attempted both the guitar and harmonica. Neither worked due to my lacking strength and dexterity. More recently I’ve tried a simple Japanese flute called the shakuhachi with similar results even I designed my own custom flutes. Electronic music has been my latest attempt, using sequencers and such. I bought a midi keyboard and almost immediately regretted it- I should have know I wouldn’t be able to play at the speed and complexity I want. I would really appreciate your suggestions. Being able to play written music live is my ultimate goal. Thanks for posting this article- some of us could really use some help.

    ———

    Dear Scook;

    I would like to formally thank you for ending all my shallow complaints regarding my own ability to make music. And I am certain you will find a process that works for you. And chances are you will be better at it than most. Nothing generates inspiration like difficult limitations.

    God Bless,

    Damon

  • http://www.technov.blogspot.com Xavier

    This is so interesting…

  • http://www.guessmen.co.uk Guessmen

    There is an instrument called the Soundbeam http://www.soundbeam.co.uk/ which uses sonar (i think) to track your movements in relation to the sensor and converts this to MIDI information.

    I've seen it used to great effect here in the UK.

  • james

    how about the samchillian:

    http://samchillian.com/

    there's some freeware that uses your pc keyboard. instead of mapping one button or key to one note it uses buttons for the _change_ in note, so you only need use a couple of keys to create complex moving passages.

    incidentally, a blind friend asked me recently about music software that would work with a screen reader – anyone have any suggestions?

  • pete

    One of my favorite controllers is the Roland Handpad. It provides a lot of MIDI functionality, and you can get a lot out of it without having any kind of keyboard chops. The lack of a really good Editor/Librarian is a bit frustrating, but it is not all that hard to program. Maybe that helps as an other off-the-shelf option?

  • Damon

    "Why didn't I think of that?"

    Someone thought

  • E.S.

    What about a custom programmed video>midi interface. It might not be quite practical yet, but within a few years?? It would take lots of experimentation and trial/error, but it seems like some combination of movement and color/pattern/shape recognition could allow a wide variety of midi commands to be sent. and certainly interesting for performance.

    I also think that the harmonica type wind controller+foot pedals is an awesome idea.

  • http://www.n0d3.org Gahlord

    I know that MacOSX has some disability features that focus around shakiness, basically some conditions make a person's mousing abilities difficult because their hand shakes. Go to System Prefs, then look in the "System" section for "Universal Access" Select the Mouse options and twiddle till your machine responds better.

    My guess is that if you have a disability you already know about this, but I wanted to bring it up as it might help bring edge-cases closer to using existing software.

    Also… Peter: Will you please post a round up of options from this thread as a separate post? This is a pretty niche situation and it would be good to conglomerize the results. It's fascinating (as a person who is "normally-abled" but very very interested in haptics). If you don't have the time to do so let me know and I will.

  • http://www.createdigitalmusic.com Peter Kirn

    Absolutely I'll do a round-up of comments from this post! I've just been sitting back and reading. :) Have to decide when to round them up.

  • http://www.pianolessons101.com/pianotechnique.html Daniel

    What a vast amount of helpful suggestions. This compilation of comments about controllers and alternatives for playing is extremely educational and enlightening. Thanks to all.

  • http://slonwheels.blogspot.com Niels Schuddeboom

    I looked at the sensor-based I-cubeX again (http://infusionsystems.com) and apart from its price tag this still is worth a look as well! BTW: Great to read much new comments, thanks!

  • Rozling

    Though I've never used one a Kaoss pad would seem a good idea – I'd say it would allow a certain degree of expression without requiring manual dexterity.

    You could get a wider range of expression by handing over some of the work to the computer. If you know what scale you want to play in at a given moment you could have, say, a Trigger Finger setup for scale selection, another for the notes (chromatic scale plus four modifier pads), and yet another for… arpeggiation/ Instrument/Filter. Or it could go sequentially: bash-bash-bash-bash means you've selected scale, key, rhythm (gated/flowing) and timbre (loop length? Sample start?). Intensity/volume could be the average velocity of all four 'bashes' Another twist is each bash could actually trigger a percussive sample at the same time as selecting these variables (doesn't have to be a different sample each time), and the actual choices you have made affect the NEXT loop or phrase, if you're into loop-based stuff. I'd say something like Bidule would be a good testing bed for this.

    Dummy clips in Ableton Live are a powerful thing, and I find setting them up a quite satisfying way of executing an idea which I've no chance of pulling off in the middle of performance as it just involves too many knob twists/button presses, or precise value intervals which I've no hope of hitting. It means you can put some personality into a sound event even if you're not physically able or don't want to micro-manage the execution of it.

    This is more on the software side of things so I'm not sure how much practical help it is…

    In a past life I was a stenographer. A stenograph machine is like a typewriter/QWERTY keyboard except you press the keys simultaneously in a 'chording' action as opposed to one-by-one. Each key is its own word/letter when pressed (like the 'one sample per bash' I described above), but in combination with other keys it's possible to build up a complex system of commands with relatively little physical movement. The layout of an actual steno machine might make it useless to someone mobility-impaired (the keys are set very close together), but I've often thought the theory could be used in musical applications with a custom-built controller.

    The way I learned the theory you have a bank of 'initial' command keys on the left side of the controler and a bank of 'final' command keys on the right (with four 'in-between' keys). You can 'play' any key on its own (initial keys would often be single-syllable words, final keys often word/phrase endings), or you can combine keys to make a word or an entire phrase. For example, LAI is 'lay', but LAIRJ is 'ladies and gentlemen of the jury'. It just depends on what you define in your dictionary as the action to be executed when a particular combination is pressed. If a stenographer was inclined they could simply define LAIRJ as 'large', but you get more mileage the way it is because you condense five or six hand movements into one. In musical terms LAI could be 'legato A Ionian scale' and LAIRJ could be 'legato A Ionian scale in reverse with filter J'.

    You don't have to include everything in the one command: One of the types of command stenographers use is the '___ Next' command, e.g. 'Cap Next' (KP-N) which capitalises any subsequent word/phrase. So for example you could specify the reverse & filter parts separately.

    Of course as you add more elements you want to control you start running out of space fast. Whoever invented the steno theory got around this by doubling up on the functions of keys, therefore E is E, U is U, but E and U together (they are adjacent in this interface) makes I. As you can imagine takes a long time to memorise, but it is possible and worthwhile when you realise a couple of keys can do a completely different function if you just press them at the same time.

    On the technical side: the computer still just sees an EU in the steno machine's output – it's really just your brain – and by extension your computer's dictionary of commands – which dictates how that EU is to be interpreted.

    One possible limitation of this system is that the command is only sent to the computer when you raise the keys after depressing them… otherwise you would just get a continuous 'stack' of information with no beginning or end, i.e. the system 'waits' for any information until it gets the /end command signal. Maybe your foot could come into play with this… not sure how it would work though…

    I hope this gives you some ideas. Let me know if you'd like me to clarify anything!

    Eoin

  • Rozling

    Ack! I forgot to include this link to an article about the piano keyboard being used to efficiently input data by Italian stenographers – I guess my attitude is, if stenographers are utilising the power of MIDI why don't musicians hijack some of stenography's concepts :)

    http://www.pcworld.com/article/id,111729-page,1/a

  • http://Chordmaps.com Steve Mugglin

    Peter,

    I've been working on, and recently released, a new music tool for all who like writing songs but may not have learned how to play guitar or piano. Besides helping singers and young children and students, it may also bring a certain degree of musical expression to those who are physically challenged. The tool allows the user to play basic chord progressions in any major key with just a mouse (on computers running Windows.) The chords are played using the General Midi soundfont on the computer's sound card.

    The mouse plays 4-note chords (in various inversions), and the arrow keys play the individual notes in the chord, so there is quite a lot of interactivity.

    I was even able to use a Midi Jack and a free VST host (Cantabile 1.2 Lite) to access and play free software synths like purple.dll or crystal.dll.

    (The program does not record performances… in this respect it's more like an acoustic instrument – to hear something again, you have to play it again… but I think Cantabile 1.2 Lite might be able to record the Midi stream flowing through it.)

    On my computer running XP, the response is fast enough to almost play "live." However, some of the other computers I've tested it on have a little more latency, which means it still functions well as a songwriting tool, but is not quite responsive enough to create the feel of playing a live instrument.

    Still, I've been a little surprised at what it can do. I thought your readers interested in alternate controllers might like to know about it. The link is http://www.Chordmaps.com/YoursToPlayIt/

    Thank you.

    Steve Mugglin

    Music Theory For Songwriters

    Chordmaps.com