I have not posted on the DSP forums yet, that is a good idea. I don't suspect this project will need a ton of computing power since I just need to analyze a few short words. I don't need any prediction modeling or large vocabulary libraries stored. And besides, the msp430 is all I have to work with from my University.

Thank you Lorin. I think your message is very helpful. It seems the most difficult part will be the pattern matching.

>How is this function storing the data into the flash? I can't make sense of it. 

This function instruct DMA controller to move data w/o CPU intervention. Source and destination address, total bytes of transfer are set-up elsewere. This function just enables flash write , starts DMA copy process, waits for DMA completion interrupt to occur and after that disables DMA and flash write mode.

Usually an automatic speech recognizer needs to process audio data in real-time. This is often done by collecting "frames" of audio data in a Ping-Pong buffer. While audio data is being collected in one buffer, the ASR is processing the data in the other. Real-time must be maintained, that is, the recognizer must complete processing the data in one buffer prior to the next one becoming available. On the MSP430 this can be done using the ADC and DMA. I have extracted audio capture code I used for such a project and it is attached. The target board was the MSP-EXP430F5438. The code and comments can help show how to set up and collect audio data.Hope this helps. 

audio_capture_01_00_bsd.zip

Thanks Lorin for the code. I am not quite at this point yet. I am trying to locate the data that is being transferred from the DMA so I can have my voice data template. Once I can have that data in vector format then I plan on comparing the incoming data with the template. Does your code work without configuring some settings? It doesn't appear to be updating data. 

> I believe the data is being stored inside the TB0CC4 register.
Code you posted is not about storing but playback using timer PWM. Together with external lowpass filter it forms audio DAC.

In response to Ilmars,

I realize this is a part of the playback code, but really that's what I need. I just need to be able to read the data from the flash, store it into some dummy array, then compare with the incoming data using the method Lorin described above. Shouldn't this effectively read from this flash?

                    
                    test[blah] = (*((unsigned char*)PlaybackPtr));
                    blah++;

where PlaybackPtr is originally pointing to the first flash memory address ?                   

In response to Lorin,

Right, I understand this code is not processing any data. But the data collection also seems odd in that the data in Abuffer will change but it ranges from about -30000 to -28000 and doesn't seem to change in a way that correlates with my voice. WIth the user experience audio capture I was receiving data in range from about 50-250 that I could definitely see was changing with my voice. 

Included in this post is the code to capture audio data in a Ping-Pong buffer and stream the data out of the MSP-EXP430F5438 USB UART port. The data is collected at 8kHz sample rate. Depending on how the streamed data is read and stored by the host PC,  it may have to be byte swapped.

audio_capture_01_01_00_00_bsd.zip

Thank you very much. I'll fire this up tonight. If I have trouble understanding it I'll let you know. 

Lorin I have two questions:

What do negative numbers in the context of the captured audio mean? I was expecting everything to be absolute value. Secondly, the uart output is garbage data on my screen. Could this be a symptom of having to byte swap the values? I am sure I set the baud rate to 230400 and I can tell that the data matches of with whatever sound there is. 

Micah,

It actually looks like you have got it working. I also use PuTTY, and the output is supposed to look like nonsense.  That is because the program streams raw binary audio data samples. The code is written so the ADC will output audio samples as 16-bit two's compliment binary audio samples to the buffer. These two's compliment binary samples are streamed to the PC a byte at a time. Set PuTTY to capture all data in the log file. Its also a good idea to set PuTTY to not output any bell sound, since the binary audio data bytes will randomly match the ascii bell value. After collecting data for a while, stop collection. Then open the log file in some program that can read and plot binary data, such as Matlab. You should get a signal showing the speech data as shown below. If it does not look like a speech signal, then bytes may have to be swapped, since PuTTY sometimes seems to have some extra bytes at the beginning of the log file.

Hi Lorin,

My data plotted by matlab definitely doesn't look like a audio signal, as all the values are positive. Byte swapping this data would still give a positive value, so that makes me think the data needs to be swapped before its sent to putty. I am confused how to do this. From what I understand, the code sends the data through UART with this:

// Transfer data to USCI_A1 trasmit register UCA1TXBUF
   __data16_write_addr( (unsigned long)&DMA2DA & 0xffff,
                       (unsigned long)(&UCA1TXBUF) );

inside the setupCapture function. My question is how could I edit this data before it is sent if its being transferred from the dma behind the scenes. As well, the setupCapture function is only called once in the beginning so I am also confused how each frame is sent via uart if those lines are only called once. Thanks for your help. 

Ok, thanks. Its make more sense. Ive attached some recorded audio. Out of curiosity, did you try or were you able to extract the FFT code from the users example? I am having a heck of a time integrating that code into this one. audioCapture1.brd

I looked at the data you sent me. I plotted both the original data, and the data byte-reversed. They are shown below.  It appears that at some points in the transfer of the data bytes are being dropped.  I played the section of code in the original data in the range of samples between about .5e4 and 1.5e4, and in the byte-swapped data between about 1.5e4 and 4.2e4. These are valid audio signals. The byte dropping does not happen in my tests. You could try to record again with the PC running as few apps as possible to see if that helps. 

I have not tried to extract the FFT code from the UserExperience demo code. Since the FFT is written in assembly it may be a little more difficult to adapt to your application.  You might also want to look at FFT example code in the MSPWare  IQMathLib example. Under MSPWare look at iqmathlib -> examples ->CCS->msp430

Huh, interesting. I'll look further into why my PC may be doing this. In any case, my application does not critically depend on the uart streaming, but its a nice feature.

I was able to integrate the FFT code. It took some fishing but it is now working. For your code, it is wise to extent the frame size to 1024? The reason I ask is the because the FFT code performs a 512 point FFT and the audio is stored into a buffer of size 1024.  I suppose I could also collect enough samples then truncate to the necessary length if the frame size is best at 160.

Hi Lorin,

I followed your advice to pad the current frame with the prior frame to the appropriate length. I am having some problems when I apply the window however. It seems that when I include the window code: 

for (i = 0; i < 256; i++)
      {
    	  double multiplier = 0.54 - .46*cos(2*PI*i/255);
    	  voice_data[0][i] = multiplier * frameZeroShift[0][i];
    	  voice_data[1][i] = multiplier * frameOneShift[1][i];
      }

the interrupt is called several times in a row and processAbuf repeats its value, thus not allowing me to apply the window to voice_data[0]. Do you know why this could be happening? It seems strange. Could it be from the cos function that is called?

The main problem in the code is in the line making the cos function call. These types of issues are encountered often in writing efficient DSP code. The buffer interrupt is occurring every 20ms. If the processor is running at 16MHz, this means that it must complete all processing for a buffer of data in the 20ms, or about 320,000 instructions at best (many instructions will take more than one cycle). So each iteration of the loop must take less than 1250 instructions at best, or it will not complete the loop before the next interrupt occurs. Actually, the processing of the loop must take much less than 1250 instructions in order to have cycles remaining for the recognition processing. Since this MSP device does not have a floating point hardware unit, processing of floating point data uses function calls to implement the operations. Efficient use of cycles is paramount.

In this loop the multiplier values remain constant for each frame of data. So the values of the window multiplier array should be calculated offline, and stored as a constant vector in flash memory. That way it will not have to be calculated at all in the loop.

Another issue is that specifying multiplier as a double requires all multiplication to be double precision real values. Using a regular float would require less computation.

To make the loop run in very few cycles, it can be written in fixed-point, keeping track of precision and decimal points. That way in the loop the sample data values and multiplier values can be written explicitly and immediately to the MPY32 peripheral multiplier memory-mapped register variables and the multiplication result from the MPY32 can be transferred to the output array. This is the method I often use. It requires understanding of the MPY32 peripheral, but it is very fast.

I see. So I probably won't even be able to run the dtw algorithm without the MPY32. Do you have an example on how to use it in C? I can only find examples in assembly. 

So I've included the QMathLib library from TI to compute the cosine function. It appears that this function combined with the data shifting operations is still too time consuming. I haven't included the time warping operation yet either. I am not sure how this will be possible. This is my code for shifting the data:

if(bufCount > 1)
      {
    	  for (i = 0; i < 256; i++)
    	  {
    		  if(i==200)
    			  flag=1;
    		  if(processAbuf == 1)
    		  {
    			  frameOneShift[1][i] = frameZeroData[0][63+i];
				  if(i>=96)
					  frameOneShift[1][i] = frameOneData[1][i-96];
				  if(i == 255)
					  flag = 1;

    		  } //if processAbuf

    		  if(processAbuf == 0)
    		  {
    			  frameZeroShift[0][i] = frameOneData[1][63+i];
    		  	  if(i>=96)
    		  		frameZeroShift[0][i] = frameZeroData[0][i-96];

    		  }
    	  }
      }

What options do you think I have? I know you said that the 160 frame size was optimal, could I possibly use a frame size of 256? This would reduce accuracy I suppose but it would take out the shifting operations. 

Please confirm that you are not calculating the Hamming window coefficients as part of the Hamming window processing loop. The Hamming window coefficients must be calculated offline and stored as a constant array of data, which will locate them in flash. That way the Hamming window weighting loop can simply be a multiply.

If you have RAM memory available, what I would recommend is setting up two input audio buffers of length 256. Change the code that does the audio capture so that the DMA writes to the last 160 samples of each buffer, rather than the beginning part of the buffer. Then when the DMA has written a frame of data to the last 160 samples of the current buffer, immediately upon the DMA interrupt write the last 96 samples of the prior buffer to the first 96 samples of the current buffer that contains the new frame data. Here is example loop code I used in a project. Maybe memcpy might even be more efficient. 

     // Copy overlap data to currentAbuf for the frame, WIN_SIZE=256 FRAME_SIZE=160

      for( i = 0; i < WIN_SIZE - FRAME_SIZE; i++ )

      {

         Abuffer[currentAbuf][i] = Abuffer[priorAbuf][FRAME_SIZE + i];

      }

Presently I am not calculating any window coefficients, just to test if the code was fast enough. What do you mean by 'offline'? 

Oh. I thought you were still running the Hamming processing loop because you mentioned that you were calculating the cosine.

By offline I mean calculating the Hamming window coefficients on a PC using Matlab, a program, or whatever. Then in the MSP code include

the window coefficients as a constant array.

int_least16_t const hamming[256] = {....};

The code to set up one of the 256 point frames of data should only take about 100 or so cycles of the processor's time. There should be plenty of time left for processing.. 

Hi Lorin,

It seems that my data is still being corrupted when I transfer it to my PC. This is making it difficult to create a template. I am running with only CCS open, do you have any idea why this could be happening? When I play the recording in Matlab its just static with bits and pieces of valid audio.