OSC Update

Update on The Hallway Project - 6/13/12 - Name: RainfallPlayer




Jan and I spent a significant portion of our day yesterday playing with Processing, ChucK, and OSC. We were able to resolve one of the two problems which arose - an issue with all of the ChucK files not being closed properly at the termination of the Processing program. However, we could not figure out why one of the programs (ChucKHelloWorld) would not close properly no matter what we did. My only conjecture is that it was the one-way relationship between ChucK (the receiver) and Processing (the sender) that caused the error (as opposed to the two-way relationship in ChucKPlayFileDemo). I noticed that I faced the same problem when Lucy was sending a signal to me via OSC the other day - when she closed her end, my end froze on the last note played instead of ending the program.

This issue definitely warrants further investigation, but, for the time being, we have accepted the successful implementation/termination of ChucKPlayFileDemo and moved forward using that as our framework.

At the end of the day, Jan and I began to dissect the ChucKPlayFileDemo (I'm just going to call it Demo from here on out) code to see what we could keep, and what code would need to be changed/discarded/added. We also began to write out a more extensive outline for what we want our OSC program to look like. I'll talk about our idea/outline first, and then what we have learned so far from looking at the existing code.

The Outline:

A microphone in the hallway picks up noise, and sends that information to a ChucK file. The ChucK file calculates the amplitude of the incoming sound wave (most likely using an FFT analyzer - I found example code which seems to do this calculation) and stores that as an integer. That integer is sent in an OSC message to the Processing program, which alters it to compute how many raindrops to produce at a given interval (raindrops are produced every 1 second - subject to change if this looks bad). There is a low and high cut-off. If the integer passed in is less than a certain number, no raindrops are produced. If the integer passed in is above a certain number, instead of making more raindrops, their speed will increase.

The raindrops are identical, each with a size, color, x-position, and DY (speed at which they fall). The x-position is an x-coordinate randomly chosen between 0 and the width of the screen. When a raindrop's y-position is greater than the height of the screen, the object is destroyed. Perhaps an array list would be useful to keep track of all the raindrops.


What we have learned from existing code:

I currently have example ChucK and Processing files which will be used as a framework for this project. The relevant ChucK file I found computes the gain of an incoming sound. This file is actually targeted towards calculating frequency, which seems to be a more difficult issue than calculating amplitude. I haven't figured out how to test the code yet - it should take an input from the adc and do the analysis, then play out a corresponding sin wave. I will have to see if this works. Also, perhaps a different analysis than FFT will be useful for our purposes - is there one more targeted towards computing amplitude?

The Processing Demo program uses 5 different classes to execute: Chuck, ChuckAttendant, ChuckController, DemoFilePlayerChuck, and ChuckPlayFileDemo, and communicates with 2 ChucK programs: playfiledemo.ck and monitor.ck.

I could go into discussing the guts of each of these programs, but I'd rather not here. Perhaps I could designate a second post to discussing in-depth the functionality of each, but here I'll just cover each file briefly. ChuckPlayFileDemo basically serves as main() - it instantiates a ChuckController and a DemoFilePlayerChuck. ChuckController is in charge of sending commands to ChucK, and ChuckAttendant is in charge of creating the message which lets monitor.ck know that Demo is still alive (and sends it via the ChuckController passed in through its parameter). ChuckAttendant is an extension of Chuck, and is the lowest-level class in the package. It provides the framework for analyzing incoming OSC messages. Finally, DemoFilePlayerChuck is in charge of all the graphical stuff. This class also extends class Chuck. It starts by setting up the graphic objects (balls with rings around them) and then the corresponding sound (.wav files kept in an array, looped through and added one at a time). Then it draws the balls in the frame and updates the affected audio (volume and pan) based on the location of the ball. The nested class Ball is pretty simple: it knows its coordinates, diameter, speed, id, and the other balls. It knows how to display itself, move around the screen, and collide with other balls so that they bounce off of each other. I speculate that this will be the class which we alter most for RainfallPlayer. Finally, monitor.ck is in charge of receiving OSC messages from ChuckAttendant, and terminates ChucK when it is no longer receiving a signal from Processing. playfiledemo.ck is in charge of the sound generation and processing.


Looking Forward

A good place to start might be with converting class DemoFilePlayerChuck into RainfallPlayerChuck. The main changes will be in class Ball (will become class rain), because not much else would need to be altered to maintain functionality.

Eventually we will also need to figure out how to analyze the amplitude of incoming sounds and get those measurements into DemoFilePlayerChuck so it knows how many raindrops to make, at what speed, etc.

I am, at this point, slightly worried that this project may be above our current programming abilities. Future updates will indicate progress and whether this concern is valid.


-Julia