electro-music.com   Dedicated to experimental electro-acoustic
and electronic music
 
    Front Page  |  Radio
 |  Media  |  Forum  |  Wiki  |  Links
Forum with support of Syndicator RSS
 FAQFAQ   CalendarCalendar   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   LinksLinks
 RegisterRegister   ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in  Chat RoomChat Room 
go to the radio page Live at electro-music.com radio 1 Please visit the chat
poster
 Forum index » Instruments and Equipment » MIDI Controllers and Interfaces
Motion Sensing Interface for Music
Post new topic   Reply to topic Moderators: jksuperstar
Page 1 of 4 [89 Posts]
View unread posts
View new posts in the last week
Mark the topic unread :: View previous topic :: View next topic
Goto page: 1, 2, 3, 4 Next
Author Message
Mr. Mormon



Joined: Dec 10, 2010
Posts: 67
Location: US

PostPosted: Wed Dec 15, 2010 7:53 am    Post subject: Motion Sensing Interface for Music
Subject description: Looking at the upcoming Sixense motion sensing controller
Reply with quote  Mark this post and the followings unread

I originally posted this in Sixense's forum:

Currently, the Continuum Fingerboard, invented by Dr. Lippold Haken, is the most versatile musical instrument (interface) I know of. A pressure-sensitive keyboard-like surface allows three degrees of freedom for each finger: pitch (left-right), volume (pressure), and some other variable (up-down). All of these parameters are continuous (to some resolution) and extremely programmable (sounds, tunings, filters). See my favorite example. It does suffer from some problems, though:
    The output is MIDI messages, which can lead to incompatibility, and seems geared toward other synthesizers, not computers.

    The Continuum and the synthesizer[s] would cost too much for the average hobbyist, and would have an unwieldy setup process.

    As with any keyboard, reach is limited (although it can be reduced by configuring notes closer), and the fingers are not that independent (piano professionals, can you wiggle three fingers on each hand in different directions?). Vibrato and tremolo, possible on the Continuum, are important parts of expression.

    While minor, when two fingers get close enough, the sensors' interpolation sees only one finger. Beat frequencies are impossible unless notes are configured far away from each other, but there goes the range.
While not better in every way, Sixense's upcoming controller looks like the future of music to me. Hold one in each hand, and their positions and orientations control six pitches and volumes respectively (see Wikipedia). The controls and parameters would also be continuous and programmable. The output would be as compatible as the SDK with the computer. The price would be the cost of the controllers, sensor, computer, SDK (for the developer[s]), and possibly whatever Sixense and Razer want in royalties (for a free and open-source program? I hope not!). Control would be with the arms and wrists, precise yet with wider range than fingers, although they would need rest, and the two controllers would need separate areas to avoid colliding or crossing. The beat problem leaves too. And that's to say nothing about portability and durability (or difficulty...).

What say you, reader? Any ideas, corrections, or daydreams? The only video of the controller in musical action so far can be seen here. It's in an early stage, but an admin on Sixense's forum is promising "updated media pertaining to this in the near future."

P.S. About me: I am a humble 17-year-old whose only musical experience is several years of improvisation on the piano.

P.P.S. This future instrument as a whole (that includes the program that will read the controllers and make sound) needs a name. I like Mu6, as in 'I want my Mu6!' No Greek letters; the wordplay needs to be obvious.

Last edited by Mr. Mormon on Sun Jun 12, 2011 2:19 pm; edited 7 times in total
Back to top
View user's profile Send private message
EdisonRex
Site Admin


Joined: Mar 07, 2007
Posts: 4579
Location: London, UK
Audio files: 172

PostPosted: Wed Dec 15, 2010 9:34 am    Post subject: Reply with quote  Mark this post and the followings unread

Heard of AudioCubes?

Oh and welcome

_________________
Garret: It's so retro.
EGM: What does retro mean to you?
Parker: Like, old and outdated.


Home,My Studio,and another view
Back to top
View user's profile Send private message Visit poster's website
Mr. Mormon



Joined: Dec 10, 2010
Posts: 67
Location: US

PostPosted: Wed Dec 15, 2010 1:02 pm    Post subject: Reply with quote  Mark this post and the followings unread

I had not; thank you for letting me know about them. They're pretty cool, but unfortunately have disadvantages compared to Sixense's controller: probably a higher price, no individual orientation information, line-of-sight issues, and, as far as I can tell, lack of a third dimension. Am I wrong on any of these?
Back to top
View user's profile Send private message
EdisonRex
Site Admin


Joined: Mar 07, 2007
Posts: 4579
Location: London, UK
Audio files: 172

PostPosted: Thu Dec 16, 2010 1:46 am    Post subject: Reply with quote  Mark this post and the followings unread

"probably a higher price"

Well, as I don't know the price of the Sixense controller, I couldn't comment.

"no individual orientation information"

depends on what you mean. The point of the cubes is that they interact with one another, and with nearby objects. The Sixense seems more like a Wii controller (or an iPhone) in that it knows if it is level or not. What I was trying to establish was how the Sixense interacts with its surroundings.

"line-of-sight issues"

Well, most controllers have such limitations. There is a defined set of operational parameters.

"lack of a third dimension"

Please define your dimensions. If we're defining them like an airplane's 3 axes, then yes, there are 2 spatial axes. But there are 4 faces to each cube and each face can either transmit, receive or sense (distance).

So I won't tell you if you're "wrong". Applying a single paradigm to determine the "perfect controller" is probably not my idea of a balanced argument. I tend to collect controllers, and tend to take my various controllers at face value as to how expressive they can be using whatever dimensions they have. Quite a few of my controllers work using distance sense; this can be very expressive. Physical orientation of the controller (like a drumstick, or perhaps the Marimba Lumina can work very well; indeed, a Theremin is arguably the ultimate distance sensing controller. The Haken Continuum has been mentioned. I don't own one, but I do have an Eigenharp Alpha which does have multiple control axes per note/event, although the Alpha is at least as expensive as a Continuum. Now, Madrona is working on it's Soundplane which might be promising too; and there are many nifty apps showing up on the iPhone/iPad platforms that are quite capable of both multiple axis touch and orientation of the surface.

I don't know why you say MIDI messages "can lead to incompatibility". That doesn't actually make a lot of sense to me, as MIDI is a venerable standard, which works just fine with computers; a controller that doesn't actually manage to generate MIDI would not, in my opinion, be very useful to either a hobbyist or electronic musician. OSC is considerably more complicated, and allows for much better expression, assuming you can map the OSC back to instruments (usually, though, the instruments are still only midi-capable). Although many of my controllers use internal high bandwidth/high resolution proprietary techniques, they all put out MIDI or OSC (or both). Why a hobbyist wouldn't want that, I don't know.

_________________
Garret: It's so retro.
EGM: What does retro mean to you?
Parker: Like, old and outdated.


Home,My Studio,and another view
Back to top
View user's profile Send private message Visit poster's website
Mr. Mormon



Joined: Dec 10, 2010
Posts: 67
Location: US

PostPosted: Thu Dec 16, 2010 3:29 am    Post subject: Reply with quote  Mark this post and the followings unread

I've heard that pitch bend is a lesser supported MIDI feature, but that isn't a problem here, you're right.

What I'm looking for in an instrument is continuous control. One can only hold two AudioCubes at a time, so with a third fixed, I see control of two orientations along one axis and two positions in two dimensions. That's six variables as opposed to twelve, which means three notes, plus a background loop, instead of six. iPhone apps and the Theremin also have less variables to work with.
Back to top
View user's profile Send private message
EdisonRex
Site Admin


Joined: Mar 07, 2007
Posts: 4579
Location: London, UK
Audio files: 172

PostPosted: Thu Dec 16, 2010 7:06 am    Post subject: Reply with quote  Mark this post and the followings unread

Zipper effect is a noted problem with MIDI; relatively low resolution of continuous controller values does present issues. Also the fairly low bandwidth of MIDI gives rise to flooding of the pipe with dense CC streams. These are known issues. OSC has much higher bandwidth and resolution, although lack of support in legacy hardware means translation back to MIDI anyway. Pick your poison.

Not to defend the AudioCubes, I do own 4 and find they are highly expressive, mainly because your hand can cover more than one face and they are sensitive to each other, and distances from each other. But my point was each controller idea utilises a specific design paradigm, including keyboard controllers (which I've had myriad discussions with folks about the appropriateness of a keyboard vs a ribbon, or touchpad, for example). Like I said, I have a lot of alternative controllers, this being more than a passing interest for me.

Going back to your Sixense controller example, it's not simply generating a lot of CC events. Mapping the CC events to useful musical parameters is fundamental. Perhaps you can describe how you would arrange the axes for which parameters (describe an application of the controller).

_________________
Garret: It's so retro.
EGM: What does retro mean to you?
Parker: Like, old and outdated.


Home,My Studio,and another view
Back to top
View user's profile Send private message Visit poster's website
robsol
Stream Operator


Joined: Apr 24, 2009
Posts: 2492
Location: Bristol UK
Audio files: 495

PostPosted: Thu Dec 16, 2010 8:30 am    Post subject: Reply with quote  Mark this post and the followings unread

People also use the Nintendo Wii for music - on the face of it looks just like the Sixence controllers. I have tried neither but seen some quite nice examples with the Wii and Kyma.
_________________
Muied Lumens Sub Forum
Bandcamp
Back to top
View user's profile Send private message
soundwave106



Joined: Nov 24, 2004
Posts: 331
Location: Elmo's Mud Wrestling Club
Audio files: 2

PostPosted: Thu Dec 16, 2010 8:58 am    Post subject: Reply with quote  Mark this post and the followings unread

EdisonRex wrote:
Zipper effect is a noted problem with MIDI; relatively low resolution of continuous controller values does present issues. Also the fairly low bandwidth of MIDI gives rise to flooding of the pipe with dense CC streams. These are known issues. OSC has much higher bandwidth and resolution, although lack of support in legacy hardware means translation back to MIDI anyway. Pick your poison.


Yeah, pretty much this. MIDI is too low of a bandwidth and, for some controllers, too low of a bit resolution for a huge amount of expression without stair-stepping or data overload. OSC is much better, unfortunately not too many instruments support OSC at this point.

For those with a few of the ol' analog synthesizers, CV remains an option. The Continuum Fingerboard for instance has a high-speed CV output option, which would probably interface quite nicely with the Future Retro XS I have.

(The Continuum Fingerboard is one expensive option though. I'm looking at getting a CV ribbon controller though for extra expressiveness.)
Back to top
View user's profile Send private message Visit poster's website
Mr. Mormon



Joined: Dec 10, 2010
Posts: 67
Location: US

PostPosted: Thu Dec 16, 2010 10:01 am    Post subject: Reply with quote  Mark this post and the followings unread

The Wii remote can't know it's position accurately; that's the difference.

How Mu6 will work: the computer will receive button, trigger, control stick, position, and orientation information (no MIDI etc. to argue over). The default setup might be position along an axis for pitch, orientation around it for volume, control stick angle or trigger pull for a filter, and button state for reconfiguration or another filter. That's simultaneously and continuously six notes and six filters, plus fourteen buttons, that we're comparing other instruments/interfaces to. My price estimate for two controllers and the sensor base is 100 USD.

Last edited by Mr. Mormon on Sun Apr 17, 2011 12:27 pm; edited 3 times in total
Back to top
View user's profile Send private message
EdisonRex
Site Admin


Joined: Mar 07, 2007
Posts: 4579
Location: London, UK
Audio files: 172

PostPosted: Thu Dec 16, 2010 3:42 pm    Post subject: Reply with quote  Mark this post and the followings unread

hundred bux with software? I'd be interested in that since I've spent 60x that on controllers, and not been disappointed with the results.

I continue to be interested in CV controllers too. I have a lot of modular stuff and CV doesn't zipper (in native mode). Soundwave106 has a point.

_________________
Garret: It's so retro.
EGM: What does retro mean to you?
Parker: Like, old and outdated.


Home,My Studio,and another view
Back to top
View user's profile Send private message Visit poster's website
Mr. Mormon



Joined: Dec 10, 2010
Posts: 67
Location: US

PostPosted: Thu Dec 16, 2010 4:37 pm    Post subject: Reply with quote  Mark this post and the followings unread

I base my price estimate on this two-year-old article, which claims that the parts necessary for the controllers and base cost $25. Obviously, there will be other factors in the ultimate price of Mu6.
Back to top
View user's profile Send private message
GovernorSilver



Joined: Apr 26, 2004
Posts: 1349
Location: Washington DC Metro
G2 patch files: 1

PostPosted: Fri Dec 17, 2010 12:38 pm    Post subject: http://www.youtube.com/watch?v=Dxx_YbHyBig Reply with quote  Mark this post and the followings unread

In case Mr. Mormon doesn't know, "CV" means "Control Voltage".

This was the means by which synths were controlled before MIDI was invented - analog synths mostly. Through control voltage, it is possible for changes to settings on a synth module to affect the behavior of one or more other synth modules connected to it via CV jacks.

A simple demonstration of control voltage by Jordan Rudess, featuring Moog Music products, but the idea applies to other CV-compatible devices:
http://www.youtube.com/watch?v=Dxx_YbHyBig

The Moog Etherwave Theremin can produce two different control voltage signals simultaneously - I'm sure there are other gestural controllers that also output CV signals:

http://www.moogmusic.com/theremin/?section=product&product_id=21301

Control Voltage (CV) signals are continuous, not discrete, and therefore not subject to resolution issues, as continuous signals by definition have infinite resolution.

_________________
Current and recent work on Soundcloud

Some old stuff on VIRB
Back to top
View user's profile Send private message Visit poster's website
Mr. Mormon



Joined: Dec 10, 2010
Posts: 67
Location: US

PostPosted: Fri Dec 24, 2010 5:26 am    Post subject: Reply with quote  Mark this post and the followings unread

I think Mu6's resolution will be good enough: 1mm position and 1 degree orientation. Got a steady hand?

EDIT: The full-size Continuum Fingerboard's default range of almost 8 octaves gives it a resolution of over 100 points between semitones. For a 30-inch arm, the same range on a Mu6 (which I'm guessing will scan at somewhere from 60 to 100hz) will give only a little more than eight points between semitones. Wikipedia says at least ten points are necessary to be imperceptible, but I don't know how much time-wise input interpolation will make an improvement. In addition, that calculation is based on my previously explained Cartesian coordinate system - spherical coordinates (arm up-down and front-back angles and arm length) could be the fix if this turns out to be a problem, except for arm length, which would still be a problem...

EDIT 2: I don't know where my calculation or research went wrong, but the pitch control is very smooth.

Last edited by Mr. Mormon on Mon Apr 09, 2012 9:00 pm; edited 3 times in total
Back to top
View user's profile Send private message
GovernorSilver



Joined: Apr 26, 2004
Posts: 1349
Location: Washington DC Metro
G2 patch files: 1

PostPosted: Thu Jan 06, 2011 1:56 pm    Post subject: Reply with quote  Mark this post and the followings unread

Mr. Mormon wrote:
I think Mu6's resolution will be good enough: 1mm position and 1 degree orientation. Got a steady hand?


You sound determined enough to get one. Be sure to keep us updated on your experiences with it and the price.

_________________
Current and recent work on Soundcloud

Some old stuff on VIRB
Back to top
View user's profile Send private message Visit poster's website
Mr. Mormon



Joined: Dec 10, 2010
Posts: 67
Location: US

PostPosted: Fri Jan 07, 2011 10:13 am    Post subject: Reply with quote  Mark this post and the followings unread

Certainly I'll update. Sixense's partner, Razer, recently demonstrated the controller with Portal 2 at CES 2011, but I doubt they'll do a music demonstration there.
Back to top
View user's profile Send private message
Mr. Mormon



Joined: Dec 10, 2010
Posts: 67
Location: US

PostPosted: Mon Jan 10, 2011 7:48 am    Post subject: Reply with quote  Mark this post and the followings unread

CES 2011 is over. Here's an update. The two controllers and the base station are called the Razer Hydra and will be available (initially wired) around April 21 for under 100USD.
Last edited by Mr. Mormon on Mon Feb 07, 2011 9:18 am; edited 1 time in total
Back to top
View user's profile Send private message
Mr. Mormon



Joined: Dec 10, 2010
Posts: 67
Location: US

PostPosted: Sat Jan 15, 2011 5:14 am    Post subject: Reply with quote  Mark this post and the followings unread

I felt like designing a Mu6 webpage (some things here are wishes):

http://mu6.com (taken lol)

[left]

On the upper right is an open PC (plugged into an off-screen outlet) displaying "Mu6". On the upper left is a base station (plugged into the PC) with the words "powered by the Razer Hydra". Four cords plug into that; the left two controllers are replaced by the text "optional".The third is a controller but small/far away. The right controller is large/close with a 'magnifying glass' showing the triggers on the back. Superimposed on the remote are the x/y/z axes with a circle around each. Just to the left of the remote is a musical staff (with the clef replaced by a 6) with a couple of slurred notes and crescendos/decrescendos.

[middle]

vertical line

[right]

(big) Available for PC and Razer Switchblade midnight May 13, 2011 (Get it? May B) in [countdown].
(big) Community (vertical link on upper right)
----
Finally, a multi-note continuous music controller under 100USD:

[left]
    6 notes
    6 variable effects*
    too many binary/discrete-level effects
at once.

[middle]

vertical line

[right]

positions and orientations
control sticks and triggers
arbitrary combinations of 14 buttons
Seriously.
----
    positions and orientations accurate to 1 mm and 1 degree
    alternative coordinate systems to XYZ
    adjustable center of rotation
    filter option for unsteady hands
    configurable on the fly
    programmable
    equalization capabilities for any audio setup
    growing code library
    load and save configurations and performances, including microphone input
    informative visuals like harmonics
    integrated tutorials
    doesn't require batteries
    online community
    The program is free. The PC is not included. Buy the Hydra.
----
*effects include sounds, filters, tunings, ranges, disabling notes, and other code

Last edited by Mr. Mormon on Sun Apr 17, 2011 12:12 pm; edited 7 times in total
Back to top
View user's profile Send private message
Mr. Mormon



Joined: Dec 10, 2010
Posts: 67
Location: US

PostPosted: Fri Jan 28, 2011 8:18 am    Post subject: Reply with quote  Mark this post and the followings unread

Any other facts/wishes anyone would like to add?
Back to top
View user's profile Send private message
Mr. Mormon



Joined: Dec 10, 2010
Posts: 67
Location: US

PostPosted: Sat Jan 29, 2011 1:33 pm    Post subject: Reply with quote  Mark this post and the followings unread

Also, EdisonRex, if you've seriously spent $6000 on stuff, have you tried the Continuum? Has anyone?
Back to top
View user's profile Send private message
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Tue Feb 08, 2011 5:01 pm    Post subject: Reply with quote  Mark this post and the followings unread

This could be exactly what I want. I didn't go with the wiimote because I don't want to use yet another custom driver solution on my Mac with my music software, which is ChucK. I would like to have the device appear as a mouse or joystick so it will easily integrate with ChucK.

Just tonight I set my particular music core, the Karplus Strong algorithm, to respond to a desktop mouse. It's wireless but not 3D. the third axis is the wheel (i need at least three, preferably 6 or more). So I am looking at the gyration air mouse which is a pretty slick device for what it is, but not specifically designed for music control. I guess I ask for a lot, but this is important to me.

They gyration air mouse is something around $80 USD, so if I can get both left and right with 6DOF for a similar price, wowza that rocks!

Now you said it knows position. Having a 3 axis gyro and a 3 axis accelerometer does ENABLE you to CALCULATE position, but it takes some math called Euler equations, which is like math from Hades and beyond. It's trigonometry inside of matrix algebra inside of calculus, and is basically the same math used to guide satellites and spaceships, lol. That's a lot of freakin math and too much for me. Does this product do the math for us? Cause that's the only way that we will know both position and orientation.

I welcome your comments - oh and it's nice to see a younger person at 17 into this, that's cool man. later.

Les

_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
Mr. Mormon



Joined: Dec 10, 2010
Posts: 67
Location: US

PostPosted: Tue Feb 08, 2011 6:58 pm    Post subject: Reply with quote  Mark this post and the followings unread

Thank you. It doesn't use dead reckoning with accelerometers (which would simply be arithmetic and a couple derivatives in this case); it detects a magnetic field weaker than Earth's from a base. Oh, and GO CHUCK! (tutorial broken, though)
Last edited by Mr. Mormon on Fri Apr 08, 2011 2:28 am; edited 4 times in total
Back to top
View user's profile Send private message
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Tue Feb 08, 2011 7:15 pm    Post subject: Reply with quote  Mark this post and the followings unread

Quote:
The device should, it seems, be released around April, the same time as Portal 2, for less than $100. I’ve posted a shakycam demo of Portal 2 being played with the device below.


It looks very good, but will not be available until april, which means september, to paraphrase someone else on the forum. I need something now. I may go with the gyration air mouse. Do you know of anything like this that is available now?

Les

_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
Mr. Mormon



Joined: Dec 10, 2010
Posts: 67
Location: US

PostPosted: Tue Feb 08, 2011 7:46 pm    Post subject: Reply with quote  Mark this post and the followings unread

Sadly, there isn't; otherwise Sixense the company wouldn't have been started, the Hydra wouldn't have gotten the CES hype it did, and I wouldn't care about it. Sorry, but at least the wait's only two months. I doubt Portal 2 will be delayed. Now, there IS some stone age $5,000 medical thing (or something with too many zeroes Shocked ) that can sense position, but I don't remember and can't find it right now.
Back to top
View user's profile Send private message
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Tue Feb 08, 2011 8:06 pm    Post subject: Reply with quote  Mark this post and the followings unread

OK, thanks. Well I can go a long way with just a Gyration Air Mouse for now, then look into the 6th sense product when it does come out. It looks awesome. One question:

You said it detects position with a weak magnetic field, that's great. Now what is the range of that sensing. I want to do large body motions with it, not in a seated position or optionally seated. So, what is the range?

Les

_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
Mr. Mormon



Joined: Dec 10, 2010
Posts: 67
Location: US

PostPosted: Tue Feb 08, 2011 8:54 pm    Post subject: Reply with quote  Mark this post and the followings unread

Six-foot radius. It supposedly can work somewhat beyond that, but no longer with millimeter accuracy. Depending on what "large body motions" means to you, you may want to wait for the eventual wireless version. An admin in the Sixense forum has stated that he has never had problems with the wired version, though.
Last edited by Mr. Mormon on Thu Feb 17, 2011 4:26 pm; edited 1 time in total
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic Moderators: jksuperstar
Page 1 of 4 [89 Posts]
View unread posts
View new posts in the last week
Goto page: 1, 2, 3, 4 Next
Mark the topic unread :: View previous topic :: View next topic
 Forum index » Instruments and Equipment » MIDI Controllers and Interfaces
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Forum with support of Syndicator RSS
Powered by phpBB © 2001, 2005 phpBB Group
Copyright © 2003 through 2009 by electro-music.com - Conditions Of Use