Processing OSC bundles in ChucK

[robhamilton]

Hey all,

As part of my q3osc project [re-implementing the q3apd quake3->osc mod], I'm sending a potentially extremely large number of osc bundles from ioquake3 to ChucK. Is there an established method of receiving not just single-lined OSC messages (e.g. the example OSC_rev.ck) but full multi-tiered OSC bundles?

I'm forming my bundles using oscpack which seems to be a strong implementation of the OSC spec.

Any thoughts or examples?

thanks,

Rob

(more on q3osc can be found at https://cm-wiki.stanford.edu/wiki/Q3osc )

[UPDATE: as Ge clarifies that time-tag support is as of yet not implemented in ChucK (see http://electro-music.com/forum/topic-18909.html ), I think this might get a bit sticky?]

------------------------
Rob Hamilton
Center for Computer Research in Music and Acoustics (CCRMA)
Stanford University
http://ccrma.stanford.edu/~rob

[Kassen]

I seem to remember this is indeed a issue, yes.

I'm quite interested to hear more about what you are doing though, this sounds exciting.

[robhamilton]

Sure, here's a super-quick overview:

So about 2 years ago I did some playing around with Q3APD, which was an ioquake3 mod that streamed user-centric position data vectors, selected weapon ids, and some other variables from the game out as UDP packets to PD. The guys who implemented it however didn't really use OSC as I had originally thought, and instead just send quasi-osc FUDI messages (text strings delimited with "/"s) over UDP. This works well for PD however it isn't a proper OSC implementation and can't be used with standard OSC libraries or apps.

I also wanted to track more data points than the q3apd guys tracked, so a few weeks ago, I (with Ge's help) hacked in the oscpack C++ classes into the open-source ioquake3 codebase, and am currently expanding the kinds of parameters you can export.

Currently, I'm tracking not only user position and state data but also xyz coordinates and bounce instances for each individual projectile fired in the game (i.e. rockets, plasma balls, BFG bolts) which, when mapped properly, lets you do some crazy things in 8 or 16 channel audio environments. Add to that some game-play modification I've added (like homing projectiles and bouncing projectiles) and it gets even stranger. I haven't packaged the mod up for easy installation yet, and am still deciding on how I want to configure my OSC namespaces.

Ideally I want to use ChucK as the basis for my audio engine, not only because I'm working with Ge here at Stanford, but also because the paradigm of sporking individual shreds for individual entities I'm tracking is very appealing, and conceptually makes sense with this user-sporking/firing-sound events/projectiles. I have a nice PD 8-channel audio server for tracking individual motions between the virtual and analog spaces, but its not dynamic and can't deal with dynamically instantiated events like ChucK can/should.

One concern of mine though is that I'm sending massive amounts of data constantly from the game engine, which could cause any audio server to choke. I'm distributing the load by outputting certain events from the game server to multiple ip/port addresses, but its still being tested and as everything with this strange project, is subject to change

The wiki link I posted above has all my latest updates, which I'm working on constantly these days.

Let me know if you have any questions or suggestions.

best,

Rob

[Kassen]

Way cool.

Will you also be using maps and so on speciffically created with the aim of ending up with musically appealing results or is this a investigation into more expressive sounds for games? Perhaps a mix, like Rez or Lumines?

I agree that sporking should work well with spawned objects and projectiles but remember that in practice ChucK doesn't have garbage collection yet. My recomendation would be to spork shreds that "go to sleep" as soon as they are sporked and wait for a (signalled) event to wake them up and set their parameters. They would then synthesise sound for some amount of time and after they are done resume waiting for a event. This avoids garbage and for a project like yours I predect that will make a large difference. There are more strategies for this.

I think you are lucky to be working with Ge, please keep me up to date, I'm no great OSC expert but I do know a thing or two about coding around ChucK lack of garbage collection so I might be able to help.

[robhamilton]

Sleeping the shreds could be a good idea, and could work in well with the data I get, as entity IDs for things like fired projectiles get reused after they go away in the game. There's also a theoretical upperbound to the number of entities ingame at any given time, so there's something to shoot to stay under.

Map-wise, yes, I do build specific maps for compositional purposes; the first paper I published on this was about a piece I did called "maps and legends" which investigates the idea of compositional maps which allow performers to run through pathways with their individual sounds tracking them in an 8-channel environment, giving a great degree of freedom to improvise within the compositional structure.

For this latest version, I'm also experimenting with custom image textures in the game which I'm planning on modelling using edge-detection algorithms, giving me more specific control over the linking between visual environment and auditory output.

[Kassen]

Yeah, and there is a upperbound as there is a limited amount of sounds your computer(s) will be able to pull off anyway. Call me a hippy but I think that if we are combatting garbage then recycling is the obvious answer .

Your project sounds interesting, I'll read more on your page, thanks!