This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Data converting for raw data to csv file in HSDC Pro

Hello,

When I capture data with Get_Capture_Data_32bits function from automation dll, I get the raw data around 36000. However, when I save it with Save_Raw_Data_As_CSV function, I get good converted raw data with small numbers, and I see it is much more better for plotting, and analyzing the results. I want to convert the data from Get_Capture_Data_32bits to data like that. Can I know which formula is used for this converted data in csv? 

  • Farid,

    Can you attach the file containing results of the Get_Capture_Data_32bits function? I'm not sure I understand what you mean when you mention getting the data around 36000. I'm not familiar with that function myself, however I will try to replicate on my side. If you can attach the file it will help. 

    Thanks, Chase

  • Thanks for quick response.

    Here is the screenshot of the csv that I have saved with numpy after I collect the data with Get_Capture_Data_32bits.

    And here is the csv that I have saved directly with the Save_Raw_Data_As_CSV  function from HSDC Automation DLL.

    As you can see the second csv is much better than first one. That is because it converts the raw data inside the dll that I don't know. My question is to find out that function for converting the data.

  • Can you please upload both files as attachments? My initial thoughts are this is something to do with return type mismatch from the dll call.

    Thanks

  • Hi Farid,

    The only difference between these functions is a conversion of the data from an unsigned int16 to a signed int16. Subtracting (2**15) from these unsigned int16 values will give you the values returned by the Save_Raw_Data_As_CSV function.

    Regards, Chase

  • Thanks for the response. I subtracted the value as you said, however still I didn't get the result (plotting) like I did with data from saved csv. That is because I send the data only for channel 1 and channel 2, but I get the almost same result for every channel in Get_Capture_Data_32bits. However I can see it in data of Save_Raw_Data_As_CSV. 

    Now I want to tell you what I did  by steps: 

    -- First I read the data with Get_Capture_Data_32bits function with sample window 4096.

    -- Then I convert it as numpy array and reshape it, and take only first 8 column. Because this function send me in interleaved mode, and I don't use interleaved mode, that is why the column between 8-16 are all zero. The whole function is like that.

    np.frombuffer(CaptureData_32bits, dtype = np.int32).reshape(16,Number_Of_Samples_Per_Channel)

    --Finally I subctract from the data 2**15 as you suggested and save it to csv.

    When I plot the b scan of data for every channel, I get similar result for every channel despite of different values. However I get different result from data when I saved it with directly Save_Raw_Data_As_CSV function. 

  • Hi Farid,

    I have used the same approach and am not seeing this issue - the data I am seeing is the same. I am thinking that you may be passing a capture event by accident which is changing the data. Let me create an experiment for you to try. Stay tuned.

    Thanks

  • Hi Farid,

    Import the attached file into HSDC Pro.  ADCdataToImport.csv

    To import this in HSDC Pro, click File -> Import Data File. Choose 2 channels, set the resolution to 16 bits, and make sure 2s complement is checked. (see image below)

    Next, run the python script. 

    # -*- coding: utf-8 -*-
    """
    Created on Mon Apr 11 10:30:34 2022
    
    @author: CW
    """
    
    def loadDLL():
        """This function loads the HSDC Pro 64-bit DLL.
        \nArgs: {None}
        \nReturn: None
        """
        
        if not 'HSDCProDLL' in locals():
            loadedDLL = cdll.LoadLibrary("C:/Program Files (x86)/Texas Instruments/High Speed Data Converter Pro/HSDCPro Automation DLL/64Bit DLL/HSDCProAutomation_64Bit.dll")
            return loadedDLL
        else: 
            pass
    
    
    if __name__ == '__main__':
        
        import ctypes
        from ctypes import cdll
        import numpy as np
        
        default_timeout = 10000
        NumChannels = 2
        Number_Of_Samples_Per_Channel = 4096
        
        
        HSDCProDLL = loadDLL()
        HSDCProDLL.ADC_Analysis_Window_Length(ctypes.c_ulong(Number_Of_Samples_Per_Channel),default_timeout)
        
        # Save data using CaptureData_32Bits approach:    
        Capture_Data_Array_Len = NumChannels * Number_Of_Samples_Per_Channel
        CaptureData_32bits = (ctypes.c_ulonglong*Capture_Data_Array_Len)()
        HSDCProDLL.Get_Capture_Data_32bits(Number_Of_Samples_Per_Channel,0,CaptureData_32bits,Capture_Data_Array_Len,default_timeout)
        
        data = np.frombuffer(CaptureData_32bits, dtype = np.int32)
        data_from_CaptureData_32bits = data.reshape(Number_Of_Samples_Per_Channel*2,2)[:][:Number_Of_Samples_Per_Channel]
        data_from_CaptureData_32bits = data_from_CaptureData_32bits -(2**(16-1))   # convert from unsigned to signed
        
        
        # Save data using the CSV save approach:
        SaveCSVFilePathWithName = b"C:/Users/Public/Documents/Texas Instruments/High Speed Data Converter Pro/Example Files/Python Files/ADCdataNEW.csv"
        HSDCProDLL.Save_Raw_Data_As_CSV(SaveCSVFilePathWithName,default_timeout)
        
        #load CSV data to np array
        import pandas as pd
        csv_data = pd.read_csv(SaveCSVFilePathWithName.decode("utf-8"),header=None).to_numpy()
        
        #compare results:
        print(np.array_equal(data_from_CaptureData_32bits, csv_data))

    This python script will save the data using both approaches. After subtracting the 2^15 offset, the data from the CaptureData_32bits reads the same as from the CSV approach. To download the python script, click on the upper right corner of the embedded code viewer.

    Thanks, Chase

  • Thanks for the code and explanation. It seems I have reshaped the array wrongly. Everything is perfect now.