May 27th, 2007 by klilley
Posted in Project (PRID302) | Comments Off
May 3rd, 2007 by klilley
Well it’s taken two days hard work but my final year project is finally complete and in its testing stage.
Currently the project works by using motion tracking with a very primitive form of gesture recognition. The screen is divided into several sections (Percussion, String, Brass, Wood Wind), each section holding a different sound. The partcipant has to draw a circle over a section to activate a sound… The sound can be looped depending on how many circles a partcipant draws i.e. drawing 3 circles will loop the sound 3 times.
The sound can then be manipulated in speed by using the bar at the bottom. When the programme recognises two points over the same area within the bar it will change it to that location. This also works with the volume bar along the right hand side.
I’ve attached a couple of movies that record me playing with it… currently it tracks my blue pen (thanks barclays for your free pens) but when I come to testing it next week in the small media lab I will be using my light wand.
Movie01
Movie02
Movie03
Movie04
Posted in Project (PRID302) | 4 Comments »
March 13th, 2007 by klilley
It has come to the point where my porject is completly stuck. I have spent 3 days sloving a whole load of errors with my neural network and now it just doesn’t want to compile because of one error that I cannot solve. So if anyone knows java/processing - help!
Posted in Project (PRID302) | 1 Comment »
March 12th, 2007 by klilley
Well…. for the past week I have been trying to get my neural network to work so it can recognise different patterns. Have also been talking to a guy on processing.org who has done something very very similar. But it’s all being going a bit wrong mainly because all of a sudden the classes have stop linking..grrrr…but I have the pattern recognition working great and now I am working with my musical housemate to create some notes that will sound good together when played… but I think it will take at least another 3 weeks to get it completed so it is producing sounds of some sort.
It’s a bit hard at the moment to juggle both project and dissertation as both seem to need the exact amount of effort.
Posted in Project (PRID302) | No Comments »
February 27th, 2007 by klilley
I have finally manged to get sound working in my project - woo hoo! so depending on the angle and length of the X and Y location it creates and changes a series of notes. At the moment it sounds like a series of computer beeps, which after 5 minutes you can’t stand to listen to. So I am going to work with a composer (if I can find one lerking in plymouth) and start working on some deccent sounds.
I am taking it one step further by creating a A.I system which you can train to recognise certaincomposing gestures - so the conducting will become really accurate - or at least thats my hope. Luckily I have half the code all ready and waiting but if I manage to get it working is another matter.
Posted in Project (PRID302) | No Comments »
February 22nd, 2007 by klilley
Since my last progress report on the 10th of January my I have focused on the technical side of my project by testing out different methods of manipulating sound. This has been all together a time consuming process – because gesture recognition systems are not common it has been hard to programme one from scratch.
I thought of three ways of creating a system:
1. Plot the path created by the light wand and then when two points intersect create a shape and measure the surface area and manipulate sound according to the size.
2. Plot the points of each shape drawn and store them in a class and if the points are hit again the program attempts to guess the shape and create sound according to that – the program learns from the participant’s actions.
3. Plot the x and y points and measure the distance and angle of each point and manipulate the sound according to the point’s new location to the pervious.
Each method has it’s pro’s and con’s. I created each method and tested it with a basic sound track.
Option 1: was inaccurate and the results were often unpredictable.

Option 2: Worked well in being able to create the shapes of a conductor but slowed down the video capture. Creating less of a real time effect and when the sound was included the system often crashed.

Option 3: If the less technically advanced but worked well. With the measurements it is possible to create a series of if and else statements to look for certain movements and manipulate/create sounds according to this.

My next task to address was to decide which sound library to use. At first I wanted to use a library called Jmetude – this sound library allows processing to create new notes at random and manipulate them. Problems occured when the library begun to build the midi file because it took a large amount of processing power it meant that the video capture would freeze and the program would end. Instead I have chosen to use another library called Krister ESS. This library allows me to import a number of sounds into audio channels and manipulate them in a series of ways.
The interface has changed a number of times over the programs development. Currently it now works by drawing a circle at the current X and Y location and fills it with a random colour.

Future Objectives:
My next step is to find a series of sounds to place within a variety of audio channels. These sounds will consist of a series of instrument notes I plan to download the sounds from http://freesound.iua.upf.edu/. The notes can then be manipulated and hopefully create a orchestra like effect.
Then find a way to begin the installation I was thinking of doing it so the participant has to tap the laptop twice to activate the camera and start the piece.
Mid-way Assessment:
In January I had my mid-way assessment with Dan L in which I presented my project so far. Dans feedback was that I was presenting two projects. One a cool little sound project and the other a confused interactive art piece and that I need to focus on just making this sound installation.
Posted in Project (PRID302) | No Comments »
February 4th, 2007 by klilley
This weekend I have been having a bash at creating my sound class. First problem came with the library and it linking to the class which i solved and now I have one bug which I cannot get rid of and it’s starting to drive me a tad insane. If anyone in the great wide world knows anything about programming with processing - HELP!!
Posted in Project (PRID302) | No Comments »
January 22nd, 2007 by klilley
This week I was on a mission to create a light wand for the computer vision to track easily. I ended up in Maplins and they were actually helpful (for once!!) and I brought a 5mm bright LED, a resistor and a 9V battery. The problem came when I was trying to find a suitable peice of pipe for all the wiring to sit in. In the end I managed to find a toy magic want and adapted it to have the LED at the top with the battery placed at the bottom.
Right now it doesn’t look too great as it’s been held together with selotape until I can find a more suitable way of doing it but for now it works well and processing tracks in the dark.



Posted in Project (PRID302) | No Comments »
January 19th, 2007 by klilley
I have managed to get my project to be able to draw shapes and store them within an array. I created this via the openGL library, for the moment I have had to use bits of other people (who have done a similar thing) code and build mine on top of it but I hope to change this to later on and improve the code that I am building on. So far the only problems I have encountered is the speed of capture for the camera whilst drawing the shapes. The next stage is to transform these shapes into a manipulating different sounds.
Posted in Project (PRID302) | No Comments »
January 17th, 2007 by klilley
Am having some stressing problems with processing. I have managed to build the majority of my project within processing but now I have come to trying to form some form of pattern recognition I am stuggling greatly with developing a way in processing. So am facing the decision of maybe having to switch to MSP and Jitter for my project. The only problem with this will be that I have no experience with the software and it maybe too late in the stage to start learning a whole new language.
Help!
Posted in Project (PRID302) | No Comments »