Last post I started with the basic of dynamic sound in flash. Now we will do something more pratical with what we saw. We will create a piano that play notes by clicking or pressing keys.
I will not spend time creating the piano, so you can download it here.
OK, starting here, we need to create a sound instance, a soundChannel instance, add the listeners and play the sound.
private var sound:Sound; private var channel:SoundChannel; public function Main():void{ init(); sound = new Sound(); sound.addEventListener(SampleDataEvent.SAMPLE_DATA, onSampleData); channel = sound.play(); } private function onSampleData(e:SampleDataEvent):void{ }
Now we will create a var called BUFFER_SIZE to handle the number of samples that will we played.
private var BUFFER_SIZE:uint = 3072;
And now we will create a Vector to handle the sound data.
private var buffer:Vector.<Vector.<Number>>;
Imagine a Vector as an Array, the diference is that you define it’s length an type of data that will store. In this case is a bidimentional Vector that holds numbers. The reason to use a Vector instead of an Array, is that Vectors are faster and we need it to get a lower latency.
OK, now we will fill this Vector with values that we will process before write the sound data, so it needs to be exactly the same size as the number of samples that will be played.
private function createBuffer():void{ buffer = new Vector.<Vector.<Number>>(2, true); buffer[0] = new Vector.<Number>(BUFFER_SIZE, true); buffer[1] = new Vector.<Number>(BUFFER_SIZE, true); var i:uint; for(i = 0 ; i<BUFFER_SIZE; i++){ buffer[0][i] = 0.0; buffer[1][i] = 0.0; } }
The Vector is bidimentional because we have left and right channel.
Now we use this buffer to write the sound data.
private function onSampleData(e:SampleDataEvent):void{ var i:uint; for(i = 0; i<BUFFER_SIZE; i++){ e.data.writeFloat(buffer[0][i]); e.data.writeFloat(buffer[1][i]); } }
So far we can’t hear anything. Now we need to create a class for the musical notes. This class will handle the note parameters and we process de audio data to play this note.
First let’s setup up our note.
package { public class Note { private var _phaseStep:Number; private var _phase:Number; public function Note(semiTone:Number, octave:Number = 0):void{ _phaseStep = 440.0*Math.pow( 2, octave + semiTone / 12 )/44100; _phase = 0; } } }
Now let’s create the method that will process the audio data
public function process(buffer:Vector.<Vector.<Number>>):void{ var l:Vector.<Number> = buffer[0]; var r:Vector.<Number> = buffer[1]; var i:uint; var t:Number = l.length; var amplitude:Number; for(i = 0; i<t; i++){ amplitude = Math.sin(_phase); _phase += _phaseStep; l[i] += amplitude; r[i] += amplitude; } }
What this method does is recieve the buffer Vector, store the left and right channel in two vars and add to each buffer sample the value from the sine position of the note phase.
To test this we will create a note at the main class
private var note:Note = new Note(0, 2); // This is a A2
and call the process method of the note before write the sound data at the onSampleData method
private function onSampleData(e:SampleDataEvent):void{ var i:uint; note.process(buffer); for(i = 0; i<BUFFER_SIZE; i++){ e.data.writeFloat(buffer[0][i]); e.data.writeFloat(buffer[1][i]); } }
The sound is strange because we need clear the buffer before add new data to it
private function clearBuffer():void{ var i:uint; for(i = 0 ; i<BUFFER_SIZE; i++){ buffer[0][i] = 0.0; buffer[1][i] = 0.0; } }
Now we need to call this method before start processing the notes
private function onSampleData(e:SampleDataEvent):void{ clearBuffer(); var i:uint; note.process(buffer); for(i = 0; i<BUFFER_SIZE; i++){ e.data.writeFloat(buffer[0][i]); e.data.writeFloat(buffer[1][i]); } }
OK, now we are hearing something. However this is a musical note, so it need to fade out until it end. To do that we can think about the wave amplitude as a number multiplied by the volume of the note. So if we multiply our note amplitude by a number that decays each sample, we can fade out our note to simulate a note played by a piano.
Back to our Note class we have the follow:
package { public class Note { private var _phaseStep:Number; private var _phase:Number; private var _decay:uint = 30000; public function Note(semiTone:Number, octave:Number = 0):void{ _phaseStep = 440.0*Math.pow( 2, octave + semiTone / 12 )/44100; _phase = 0; } public function process(buffer:Vector.<Vector.<Number>>):void{ var l:Vector.<Number> = buffer[0]; var r:Vector.<Number> = buffer[1]; var i:uint; var t:Number = l.length; var amplitude:Number; for(i = 0; i<t; i++){ amplitude = Math.sin(_phase)*_decay/30000; _phase += _phaseStep; l[i] += amplitude; r[i] += amplitude; if(--_decay<0){ _decay = 0; } } } } }
The initial number of the _decay var, is the number of samples that will be played until the note is mute, in this case 30000. That means that every sample this value will decay a unit until it reachs 0. And the reason to divide this var by it’s initial value is to get the percentage of the volume to this especific sample.
Now let’s create a method in our Main class that will create a note to each piano key. First let’s create an Array to handle all the notes.
private var notes:Array = new Array();
now the createNote method
private function createNote(semiTone:Number):void{ var note:Note = new Note(semiTone); notes.push(note); }
So when we hit a key we will call this method passing the semiTune of the note
private function init():void{ stage.addEventListener(KeyboardEvent.KEY_DOWN, keyDown); C1.note = 0; Csharp1.note = 1; D1.note = 2; Dsharp1.note = 3; E1.note = 4; F1.note = 5; Fsharp1.note = 6; G1.note = 7; Gsharp1.note = 8; A1.note = 9; Asharp1.note = 10; B1.note = 11; C2.note = 12; C1.buttonMode = true; Csharp1.buttonMode = true; D1.buttonMode = true; Dsharp1.buttonMode = true; E1.buttonMode = true; F1.buttonMode = true; Fsharp1.buttonMode = true; G1.buttonMode = true; Gsharp1.buttonMode = true; A1.buttonMode = true; Asharp1.buttonMode = true; B1.buttonMode = true; C2.buttonMode = true; C1.addEventListener(MouseEvent.CLICK, onClick); Csharp1.addEventListener(MouseEvent.CLICK, onClick); D1.addEventListener(MouseEvent.CLICK, onClick); Dsharp1.addEventListener(MouseEvent.CLICK, onClick); E1.addEventListener(MouseEvent.CLICK, onClick); F1.addEventListener(MouseEvent.CLICK, onClick); Fsharp1.addEventListener(MouseEvent.CLICK, onClick); G1.addEventListener(MouseEvent.CLICK, onClick); Gsharp1.addEventListener(MouseEvent.CLICK, onClick); A1.addEventListener(MouseEvent.CLICK, onClick); Asharp1.addEventListener(MouseEvent.CLICK, onClick); B1.addEventListener(MouseEvent.CLICK, onClick); C2.addEventListener(MouseEvent.CLICK, onClick); } private function keyDown(e:KeyboardEvent):void{ if(e.keyCode == 65){ createNote(C1.note); } else if(e.keyCode == 87){ createNote(Csharp1.note); } else if(e.keyCode == 83){ createNote(D1.note); } else if(e.keyCode == 69){ createNote(Dsharp1.note); } else if(e.keyCode == 68){ createNote(E1.note); } else if(e.keyCode == 70){ createNote(F1.note); } else if(e.keyCode == 84){ createNote(Fsharp1.note); } else if(e.keyCode == 71){ createNote(G1.note); } else if(e.keyCode == 89){ createNote(Gsharp1.note); } else if(e.keyCode == 72){ createNote(A1.note); } else if(e.keyCode == 85){ createNote(Asharp1.note); } else if(e.keyCode == 74){ createNote(B1.note); } else if(e.keyCode == 75){ createNote(C2.note); } } private function onClick(e:MouseEvent):void{ createNote(e.currentTarget.note); }
Now we need to process this notes before writing the audio data
private function onSampleData(e:SampleDataEvent):void{ clearBuffer(); var i:uint; processNotes(); for(i = 0; i<BUFFER_SIZE; i++){ e.data.writeFloat(buffer[0][i]); e.data.writeFloat(buffer[1][i]); } } private function processNotes():void{ var i:uint; var t:uint = notes.length; for(i = 0; i<t; i++){ notes[i].process(buffer); } }
It looks like that we’ve got it, but if you play a lot of notes, your application will start to loose performance. This is because every note that you hit will be processed, even if it’s mute. To fix that we can return a Boolean value telling if the note is muted or not when it is being processed.
public function process(buffer:Vector.<Vector.<Number>>):Boolean{ var l:Vector.<Number> = buffer[0]; var r:Vector.<Number> = buffer[1]; var i:uint; var t:Number = l.length; var amplitude:Number; for(i = 0; i<t; i++){ amplitude = Math.sin(_phase)*_decay/30000; _phase += _phaseStep; l[i] += amplitude; r[i] += amplitude; if(--_decay == 0){ return true; } } return false; }
and with that return value we can remove the note from the array
private function processNotes():void{ var i:uint = notes.length; while(--i>-1){ if(notes[i].process(buffer)){ notes.splice(i, 1); } } }
It’s a bit confusing, but if you have any doubts fell free to ask. Next time we will handle sound data loaded from external audio files.
[...] This post was Twitted by scarpelini – Real-url.org [...]
great stuff…
Is the delay between click and a note playing just standard as3 latency? No way around it?
The delay depends on the number of samples you provide. The minimum number of samples is 2048, this results in a delay of 46ms. This is because the SampleDataEvent requests the audio data before play it, so if you hit a note, it will be played after the previous audio data that was writen.
However, you have to be careful with the number of samples that you provide, because less samples means more data being processed in a small period of time.
Amazing Anaya… very good sample.
[]’s
Great Tutorial and nice explanations!
one completely different question:
do you think with SampleDataEvent one can make a CLEAN fade in/out or crossfade for mp3 files? I tried several ways with timers and OnEnterFrame and even Tween Engines- but it always makes little click noises for every step.. or do you know a better way?
I’m trying to create a class that i could use for multi-sample(?) web-sounddesign for my diploma…
anyway great work!
Marian
Obrigado, very nice job!!