This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

CodeCoverage results cannot be retrieved using DSS Scripting

Other Parts Discussed in Thread: CCSTUDIO

Hi,


this post got quite lenghty, sorry about that..

I have a DSS script which initialises/configures the analysis properties (i.e. startAddress, transport, etc.), runs the analysis, executes a test set and tries to retrieve the analyzer results. The problem is that the analyser does not produce any results; the exported CSV files have only column headers but no data. I need some help to get those results.

I have CCSv6.0 installed in Ubuntu Linux and the target I'm using is C66xx.

First I tried measuring code coverage via CSS IDE and it worked: CCS started to receive the PC trace as expected on given startAddress and Code Coverage information could be extracted. No problem.

Then I tried to repeat the same thing using DSS scripts. I reused the same ccxml file and used the same analysis properties which were proven to work. And yes, the script "seems" to work to an certain extent:

- the connection is established since I'm able to get information about the target, all necessary object instances (i.e. DebugSession, ScriptAnalysisSession, etc.) are created, symbol used as startAddress can be found from target, etc.

- it seems that at least SOMETHING is received from the analysis, since if I increase the size of the buffer, the time for extracting the analysis information increases also.

- when the testcase used in code coverage measurement has been executed and trace data should be available for retrieval, invocation for analyzer.waitForEndOfData -returns after few seconds and does not wait indefinitely.

- call to analyzer.exportDataToCSV creates csv-file with log headers which are showing column names you would expect for code coverage data.

..BUT:
- what I do NOT really know is that is PC trace (or code coverage, or whatever analysis is used) really started when the given startAddress is executed and is some trace really retrieved.
- even IF something is retrieved, theres nothing is available in csv -files.

I also tried to use a predefined analysis settings, i.e. I saved the settings from CCS GUI with a name and used loadAnalysis -method to load those settings. Unfortunately it didn't work either, but caused an obscure exception in CoreTrace.js.

I'm starting to be out of ideas here. One culprit might be that I'm just using those Analysis, Analyzer, AnalysisSession, etc. objects incorrectly. I just can't get my head around on which is what..  I've been following the script examples coming with CCS and I've tried to use guidelines available in TI pages, but I haven't been able to find any clear omissions or incorrect configurations in my code.

Also, there are some obscure errors/exceptions/warnings coming up, from time to time, when trying to execute the script. (See below)

***
warning: dispatch exception: Ti::Sds::Trace::ChannelError
identity: 6978C715-BAEE-4ED6-88FD-7BDAC8038C81
facet:
operation: GetChannelAttribute
warning: dispatch exception: Ti::Sds::Trace::ChannelError
identity: 6978C715-BAEE-4ED6-88FD-7BDAC8038C81
facet:  
operation: GetChannelAttribute
***

..this error/warning doesn't (?) seem to affect the execution of the script. If it does, I don't know how.


The script I've created is included below. The script defines a class named Rowboat (don't ask me why) which handles the initialisation of connections, execution of testcases, extraction of results, etc. The executed script is implemented below the class definition.
=============================================
/**
 * script file starts
 */

importPackage(Packages.com.ti.debug.engine.scripting);
importPackage(Packages.com.ti.ccstudio.scripting.environment);
importPackage(Packages.com.ti.dvt.engine.scripting);
importPackage(Packages.com.ti.dvt.analysis.traceviewer.activity);

importPackage(Packages.java.lang);
importPackage(Packages.java.io);
importPackage(Packages.java.util);



var Rowboat = function(){
    
    this.scriptingEnv = ScriptingEnvironment.instance();
    this.debugServer = this.scriptingEnv.getServer("DebugServer.1");
    this.debugServer.setConfig("/home/tk/ti/CCSTargetConfigurations/codecoverage.ccxml");
    
    this.debugSession=null;
    this.dvtServer = null;
    this.analysisSession=null;
    
    this.outFilePath  = "/home/tk/my.out";
};

Rowboat.prototype.initiateConnections = function(){
    
    
    this.debugSession = this.debugServer.openSession("Spectrum Digital XDSPRO STM LAN Emulator_0/C66xx_0");
    
    
    this.debugSession.target.connect(true);
    this.scriptingEnv.traceWrite("Loading symbols from "+this.outFilePath);
    
    try{
        this.debugSession.symbol.load(this.outFilePath);
        
    }catch(err){
        
        print("Exception thrown during loading of symbols; "+err.message);
    };
    
    
    this.dvtServer = this.scriptingEnv.getServer("DVTServer.1");
    this.scriptingEnv.traceWrite("DVT server created");
        
    this.analysisSession = this.dvtServer.openAnalysisSession();

};


Rowboat.prototype.terminateConnections = function(){
    
    this.scriptingEnv.traceWrite("**** Rowboat.terminateConnections invoked ***");
    try{
        if(this.analysisSession!=null){
            this.analysisSession.terminate();            
        }
        this.dvtServer.stop();
        this.debugSession.terminate();
        this.debugServer.stop();
        //this.scriptingEnv.traceEnd();
        
        if(!this.debugSession.target.isConnected()){
            this.debugSession.target.disconnect();
            this.scriptingEnv.traceWrite("Disconnected from target:"+this.scriptingEnv.target.isConnected());
        }
    }    
    catch(err){
            print("Exception caught in Rowboat.terminateConnections(): "+err.message);
        
    }
        
};

Rowboat.prototype.initiateAnalysisSettings = function(){
    
    this.scriptingEnv.traceWrite("*** Rowboat.initiateAnalysisSettings invoked ***");
    
    try{

        this.scriptingEnv.traceWrite("invoking loadAnalysis.. ");
        this.analysis = this.analysisSession.loadAnalysis("Hardware Trace Analyzer/Code Coverage");
        
        this.analysis.setProperty("core",this.debugSession.getName());
        this.analysis.setProperty("startAddress","testcase_1");
        this.analysis.setProperty("bufferSize","512 kB");
        this.analysis.setProperty("receiver","Pro_Trace");
        this.analysis.setProperty("transportType", "Pro Trace");    
        
    }catch(err){
        print("Exception caught in Rowboat.initiateAnalysisSettings(): "+err.message);
    }
    
};

Rowboat.prototype.activateAnalysis = function(duration){
    
    this.scriptingEnv.traceWrite("*** activateAnalysis invoked ***");
    this.debugSession.target.runAsynch();
    this.scriptingEnv.traceWrite("print out analysis object to see if it's valid: "+this.analysis.getName());
    this.analysis.run();
    
        
    this.scriptingEnv.traceWrite("Analysis started");    
    
};

Rowboat.prototype.runTests = function(duration){
    
    this.scriptingEnv.traceWrite("*** Rowboat.runTests invoked ***");
    
    

    var runTestsResult = runCommand("python","runTestCases.py");
    
    this.scriptingEnv.traceWrite("Sleeping "+duration+ " ms.. ");
    Thread.sleep(duration);
    
    
};

Rowboat.prototype.getAnalysisResults = function(){
    
    this.scriptingEnv.traceWrite("*** getAnalysisResults invoked ***");
    
    try{
        var table=0;
        this.debugSession.target.halt();
        Thread.sleep(5000);
        
        var analyzer=this.analysis.findAnalyzerByName("Code Coverage");
        
        this.scriptingEnv.traceWrite("Waiting end of analysis data from analyzer: "+ analyzer.getName());
        
        analyzer.waitForEndOfData(60000, table);
        
        this.scriptingEnv.traceWrite("All data received from analyzer.. ");
        
        this.scriptingEnv.traceWrite("Start exporting measurement results to csv file");
        
        var cwd = this.scriptingEnv.getCurrentDirectory();
        this.scriptingEnv.traceWrite("Exporting results to " + cwd);
        analyzer.exportDataToCSV("Line Coverage", cwd+"/"+"CodeCoverageLogs.csv", null);
        
        this.scriptingEnv.traceWrite("Trying to get dataset from analysis session.. ");
        var dataset = this.analysisSession.getDataSet();
        if(dataset.length>0){
            
            for(var i=0;i>dataset.length;i++){
                
                this.scriptingEnv.traceWrite(dataset[i]);
                
            }
        }else{
            
            this.scriptingEnv.traceWrite("No dataset received.");
            
        }
        
        
        analyzer.end();
        this.scriptingEnv.traceWrite("results exported");
        this.analysisSession.endAnalysis(this.analysis);  
        

    }
    catch(err){
        this.scriptingEnv.traceWrite("Exception caught on line "+err.lineNumber+" with message: "+err.message);
    }
        
};


var rowboat=new Rowboat();
rowboat.initiateConnections();
rowboat.initiateAnalysisSettings();
rowboat.activateAnalysis(5000);
rowboat.runTests(25000);
rowboat.getAnalysisResults();
rowboat.terminateConnections();

print("Script exiting..");

/**
 * script file ends
 */

    

  • Hello Timo,

    Thank you for posting such a detailed test case, especially your whole script. It was very helpful to have all that information. I was able to go through it and I think i know what the issue is:

    Timo Kauppinen said:
    Rowboat.prototype.activateAnalysis = function(duration){
        
        this.scriptingEnv.traceWrite("*** activateAnalysis invoked ***");
        this.debugSession.target.runAsynch();
        this.scriptingEnv.traceWrite("print out analysis object to see if it's valid: "+this.analysis.getName());
        this.analysis.run();
        
            
        this.scriptingEnv.traceWrite("Analysis started");    
        
    };

    I see you run the target asynchronously and then you run the analysis. I would recommend running the analysis first. Otherwise by the time the analysis starts, you may have missed capturing the data you were trying to capture. Most people start the analysis right after configuring it. Give that a try and see if that helps.

    Thanks

    ki

  • Hi,


    thank you for your response!

    I moved the runAsynch() method invocation just before starting the test cases (i.e. calling Rhino's runCommand -function). Unfortunately it didn't produce any better results. Still, it helped me forward, just a little bit:

    instead of fetching an analyzer instance, I tried getting a "Trace Viewer" data provider instance by using analysis.findDataProviderByName("Trace Viewer")  and used its exportDataToCSV -method to see whether it would produce any output. To my surprise, it produced a humongous 100MB csv-file, which seemed to contain just trace and not Code Coverage data. But at least something :)

    As I looked through examples of using Code Coverage using DSS, it seems that getting data out of "data providers" and "analyzers" should work in similar manner. But in this case, they didn't.

    Any suggestions what I could try next?

    Timo

  • Timo Kauppinen said:
    I moved the runAsynch() method invocation just before starting the test cases (i.e. calling Rhino's runCommand -function). Unfortunately it didn't produce any better results. Still, it helped me forward, just a little bit:

    Can you show me a code snippet of where exactly you moved it to?

  • ..something I could have done immediately..

    Here is the change I made:

    <snip>

    Rowboat.prototype.activateAnalysis = function(duration){
        
        this.scriptingEnv.traceWrite("*** activateAnalysis invoked ***");
        
        // ************ ..runAsynch was moved from here.. **********

        this.scriptingEnv.traceWrite("print out analysis object to see if it's valid: "+this.analysis.getName());
        this.analysis.run();
        
            
        this.scriptingEnv.traceWrite("Analysis started");    
        
    };

    Rowboat.prototype.runTests = function(duration){
        
        this.scriptingEnv.traceWrite("*** Rowboat.runTests invoked ***");
        
        //************.. to here *************
        this.debugSession.target.runAsynch();    
        
        var runTestsResult = runCommand("python","runTestCases.py");
        
        this.scriptingEnv.traceWrite("Sleeping "+duration+ " ms.. ");
        Thread.sleep(duration);
        
        
    };

    </snip>

    Another peculiarity (and an error on my behalf) I noticed was that I had invoked the waitForEndOfData method so that the intended parameters were reversed: the timeout value was on the place of the datatable and vice versa. For some strange reason, the script still "behaves" somewhat better then and no exceptions are thrown. If the parameters are as they should, either the timeout throws an exception or the script hangs indefinitely.  The documentation didn't define what actually the "datatable" variable is or what values it should contain, and I just defined it as 0 (zero). I guess the function ignores the values if they are not applicable.


    Timo

  • Hi Timo,

    Sorry for the delay in response. I have been playing with this for a bit and I don't see any issue with your script after the new modifications. I actually took your script and tweaked it a bit to run my executable and it generates coverage data everytime. The only differences I have is that I did a standard run instead of an asynchronous run since my program terminates and I am capturing coverage data for everything while you are capturing just a section of coverage data and you run the target for 25 seconds before halting the target. I also disabled the runCommand call that runs your python tests. What kind of tests are you running with those python scripts?

    thanks

    ki


  • Hi,

    nice to hear that the script has demonstrably worked. Perhaps one could conclude now that the problem is in the tests: either the given start address is not executed or it is executed but not observed correctly by the analysis. Could there be any other alternatives? I'm quite positive that the problem is not in the execution of the start address, since I know that the test cases are executed.

    The python script connects to the target and when connected, it starts the execution of test cases which, I have assumed, the analysis should be able to observe. A function name has been used as a startAddress. The test case execution is finished before the 25 second time limit has passed.

    As I mentioned earlier, I was able to get code coverage data via CCS GUI by using the same ccxml-file and the same start address. I start the measurement via GUI and then started the test execution script from the command line. So the general idea of how I've planned to measure the code coverage data seems valid.

    Is there any way  to observe in DSS script if given startAddress is executed?

    Timo

  • I was playing around with your script with some more test cases and I found another place where the cause of your issue may be. See the line in BOLD below:

    Timo Kauppinen said:

    Rowboat.prototype.getAnalysisResults = function(){
        
        this.scriptingEnv.traceWrite("*** getAnalysisResults invoked ***");
        
        try{
            var table=0;
            this.debugSession.target.halt();
            Thread.sleep(5000);
            
            var analyzer=this.analysis.findAnalyzerByName("Code Coverage");
            
            this.scriptingEnv.traceWrite("Waiting end of analysis data from analyzer: "+ analyzer.getName());
            
            analyzer.waitForEndOfData(60000, table);
            
            this.scriptingEnv.traceWrite("All data received from analyzer.. ");
            

    The doc for that API is:

    --
    waitForEndOfData

    public void waitForEndOfData(java.lang.String dataTable,
                                 int timeout)
                          throws ScriptingException

        Waits until all data is available in the specified table

        Parameters:
            dataTable -
            timeout - in milliseconds, 0: no timeout
        Throws:
            ScriptingException
    --

    I think the parameters are reversed in your call to the API. Most likely you meant:

    analyzer.waitForEndOfData(table, 60000);


    Can you try this in your script and see if this helps?

    Thanks

    ki

  • Hi,


    Yep, I also noticed that one some time ago. I mentioned this accidental reversal of parameters already a while ago (see my post on 12th Sep.)

    I've tried all overloaded versions of the method, with following results:

    - If I use the method without any parameters, it (seemingly) always stays there for indefinite time. I assume that with 512 kB buffer size it should not take over 12 hours to wait for the data. I've even tried this with 4 kB buffer with same results. Seems that there's just not anything there. I wonder why this method does not return if there is no data to wait?

    - if I use the method with timeout (for example that 60000 ms ) it has always thrown the timeout exception (or similar), and the end result is the same => no result. Nothing is exported to CSV.

    - Just recently I tried the "datatable" parameter version and it gave a little better result. waitForEndOfdata() -call actually returned and the script proceeded to exportDataToCSV invocation but it didn't proceed from there anymore. I cancelled the script after 16 hours. One issue in this case was that  I couldn't find any documentation about what is actually meant with this "datatable" parameter. What are the correct parameter values there? I assumed that the values are the buffer/analysis names so I used string "Line Coverage" as parameter. It might be worthwhile to try this with "correct" values since it has been the most promising of available options.

    Funny thing here is that the method waitForEndOfData() -method actually "works" the best when it is given those incorrect "reversed parameters" you mentioned. E.g. if there's no data to wait or export, the method invocations just return and do not hang indefinitely etc.

    It's frustrating that it's not possible access the "raw data" that might be produced during analysis. At least I haven't found any way to do it. Sometimes the exporting of data seems to take a while and I get the impression that it is actually doing some data processing, but nothing excluding column headers, is not produced into the csv file.


    Timo

  • This issue is being worked off-line. There a bug with the code coverage feature with collection and exporting of Data.  This bug will be addressed in the next release.