different result compiler with debug and without debug environment

evs said:

If I do a sizeof of my main data struct the size is 0.

Make sure a correct prototype of VT100_printf is visible.  If not, the compiler does not pass the arguments to VT100_printf correctly.

evs said:

 When I run the program with debug environment the program works fine. When I run the program standalone the program doesn't behave as expected. I checked all the compiler parameters and they are the same.

So, when you run under control of CCS, everything works as expected.  Is that correct?  Please describe exactly how you run the program standalone.

Thanks and regards,

-George

VT100_printf is used multi time at that point with different arguments and there are working fine.

The problem:
I compile my program in Release with debugger environment and if I give the command to spin the servo motor everything works fine.
When I compile for Flash(no code change), I checked that all parameters are the same as release. When the jtag is connected and I start the program with debug environment the program works correctly. When I remove the jtag programmer and power cycle. The core program works, only if I give the command to spin the servo motor. the motor It doesn't spin. The only change is the is the removal of the Jtag programmer and debug environment.

I have encounter the same problem, that the servo doesn't spin. And I traced the problem t back. In a change in the variable struct of the Intraspin. because of that I think I think that somehow the compiler optimized some variables in the struct so the alignment in the library is not correct.
The problem is it really hard to debug because the program works fine when I run in the debug environment.

Because of timing and size I can't disable the compiler optimization(-O2) to see if that fix the problem.

evs said:

VT100_printf (" sizeof: %d \n\r",sizeof (servo_obj));

The expression "sizeof(servo_obj)" is of type size_t, but the %d argument expects an "int" argument.  Cast the expression to int, as follows:

VT100_printf (" sizeof: %d \n\r",(int)sizeof(servo_obj));

evs said:

VT100_printf is used multi time at that point with different arguments and there are working fine.

Function prototypes, particularly when they have variadic arguments (like printf) can be very tricky. If a prototype is not in scope, you can get subtle problems that don't show up at every call site. Please double-check that you have a valid prototype of VT100_printf visible at every point where you call it.

evs said:

The only change is the is the removal of the Jtag programmer and debug environment.

The compiler RTS function printf requires the debugger to be attached to function, as it is actually CCS which does the I/O. Perhaps VT100_printf similarly requires a debugger to be attached?

The problem is not the VT100_printf! It is the interface to the Intraspin library that doesn't work in flash. I only use VT100_printf! to debug the problem. 

Can you please tell me how I can verify that after compilation the data structs are not modified.  and give me detailed information what  happens in the debug environment at the controller level.

At this point I have a working prototype but it is useless because I can run it standalone.

and yes I already checked the obvious.

The following happened: suddenly the motor starts spinning when I run the controller standalone.
I cleared the build and rebuild the code and program the device. motor won't run.
there are no changes in the code or in the compiler config. I always give the same commands when I test my program. I'm trying to reproduce the result. without luck so far.
But this starts to look like a compiler issue.

Is there a max size of a struct?
If I compile with -O4 I got this error message:
error: symbol "Readcontr1_handler" redeclared with incompatible type:
"SERVO_Handle()[struct _SERVO_Obj_

Pointing multiple time at the same line of code?

evs said:

Is there a max size of a struct?

No.

evs said:

error: symbol "Readcontr1_handler" redeclared with incompatible type:
"SERVO_Handle()[struct _SERVO_Obj_

Please see this forum thread for a discussion of this situation.

Thanks and regards,

-George

I'm not at all familiar with VT100_printf or Intraspin, and I don't know what you are referring to when you talk about the debug environment at the the controller level, particularly if you mean a standalone environment. I am a compiler expert, and this forum is dedicated to understanding the compiler.

I seriously doubt the compiler has somehow changed its idea of how big the structure is if you haven't modified the structure. That's just far too basic of a thing for the compiler to change mid-compilation, even in the name of optimization. In my expert opinion, it's far more likely that something is incorrectly reporting the size of the structure, or something is going wrong during the reporting.

The symptoms you have described are far too removed from the potential source of the problem for us to guess at what bug you are facing. Perhaps it is a compiler bug, but if so, we would need to pin down precisely what is going wrong, and backtrack from there. It would help, for instance, to know whether you are recompiling when you switch from hosted (attached to a debugger) and standalone operation.

Archaeologist,

 Thanks for the reply. I'm just a user of the compiler and I see different result with the same source code and compiler options. And user error is absolute an option. But in my opinion TI didn't a great job in building a robust development environment. So can you please set your superiority aside and give me some answer so I can rule out some potential problems.  Because at this point I have 2 years of work down the drain because of potential compiler issues.

The problem:

 My program run correct with the Jtag interface connected and the CCS debug environment.

 When I run remove the JTag connector and power cycle the controller. A part of the TI closed source motor library(Intraspin) doesn't work  properly.

why they are related, is a mystery. I think it is has something to do with the data order or initialization of the structs for the close sourced library. If I change the data order in the library structs I get the same similar results of the problem.

My question: How do I verify that the data order of my structs. isn't modified after optimization.

I don't have access to low level results of the compiler or the source code of the Intraspin or any documentation what happens at the low level at the debug environment. So at this point my options are very,  very limited to debug this issue.

George,

Thanks for the reply. So the workaround is to use -O3. I was wondering are those problems not related. So somehow the linker can't workout the data structure.

I have one big global struct with all my data , define in my main and a function to distribute the data across the program.

defined in main.c
// Global.
SERVO_Handle servo_handle;
SERVO_Obj servo_obj;
// in the main program: servo_handle -> servo_obj


SERVO_Handle Readcontr1_handler(void) {
return servo_handle;
}

defined in main.h
SERVO_Handle Readcontr1_handler(void);

This is called throughout of the program so access the data. this is really top down. So why does the linker have a problem?

evs said:

So why does the linker have a problem?

Carefully inspect the definition of the type SERVO_Handle seen in each file.  This inspection must be recursively carried out for every field in SERVO_Handle that is not a built-in type like int or float.  You will eventually discover a difference.

Thanks and regards,

-George

I'm sorry you are having such trouble with the toolset.  I am not trying to condescend, I am trying to the best of my ability to help you get to a solution as quickly as possible.  I'm more than happy to try to get you answers, but I don't understand your problem well enough yet to be very helpful.  

The data order of members in a struct is immutable; the compiler is not allowed to rearrange them.  The size and layout of the members is implementation-defined, but also immutable; the compiler is not allowed to change them based on optimization.  You note "I see different result with the same source code and compiler options."  In this case, the structs should appear exactly the same every time.  You say you have observed them not to be the same, which would definitely be a bug, but it is very unlikely the bug is in the compiler's handling of structs.  It could very well be a bug in the compiler, perhaps manifesting somewhere in how the struct size or layout gets reported.

I can tell you in detail how the CIO mechanism works.  This is how the RTS low-level C I/O routines talk to CCS to get CCS to perform system calls on its behalf, such as write().  However, by the symptoms you are describing, the CIO mechanism seems to be working just fine, so I doubt the problem is related to CIO.  Note that CIO will only work when a debugger like CCS is attached, through JTAG or whatever other connection.  If you still want details on CIO, and the wiki article is not sufficient, please let me know.

I am not familiar with any other low-level debug mechanisms other than CIO.  I am not familiar with the very low-level JTAG mechanics.  You would probably need to post in a device-specific forum to get information about standalone debug mechanisms.  I don't know if there is a forum dedicated specifically to InstaSPIN.

How are you observing the size of the struct in standalone when you observe it to be size 0?  I don't know what mechanism you're using, so I can't suggest how you might inspect details about the struct and report them to the observer.

Regarding this issue ...

evs said:

 When I run the program with debug environment the program works fine. When I run the program standalone the program doesn't behave as expected. I checked all the compiler parameters and they are the same.

While anything is possible, it is highly unlikely this is somehow due to the compiler.  It is far more likely that something is wrong in how you start the system.  This error is somehow avoided when you use the debugger to start.  Such system startup issues are beyond the expertise of us compiler experts.  I recommend you start a new thread in the InstaSPIN forum.  

Thanks and regards,

-George

George Mock said:

Regarding this issue ...

evs

 When I run the program with debug environment the program works fine. When I run the program standalone the program doesn't behave as expected. I checked all the compiler parameters and they are the same.

While anything is possible, it is highly unlikely this is somehow due to the compiler.  It is far more likely that something is wrong in how you start the system.  This error is somehow avoided when you use the debugger to start.  Such system startup issues are beyond the expertise of us compiler experts.  I recommend you start a new thread in the InstaSPIN forum.  

Thanks and regards,

-George

I already ask in C2000 / Intraspin forum. Only the tech know how is not sufficient to get any decent answers. That is why I tried again here.

It is perhaps a initialization problem only  I don't see why it is in the source code.   When I power cycle device. program it with CCS. the program run always correct. When I power-cycle device and run it standalone the intraspin part of the program fails.  So it seems that the reason it fails is a external factor. 

Archaeologist,

 Thanks for the detailed information!  I spend all day looking at memory dumps of my structs in different state of the program. It is bit  finding a needle in a haystack.  But  didn't find any reason that the compiler/linker is the problem.

Do I understand correct that the CIA talks to Jtag state machine in the IC? And the jtag  state machine is direct connected to ram/flash.  So there is no code execution in the controller. Is there  command to clear the memory/ram. So I run the code without old values?  But I can also do that manually by clearing it at the start of my program.

CIO does not know anything about JTAG. It talks to the host (the debugger) by writing to target memory. The debugger uses its memory probing ability (perhaps using JTAG, perhaps some other mechanism) to read target memory and decode the request.

All of your remaining questions in that post are beyond the expertise of the compiler team.

I'd like to get a clearer understanding of precisely what you're seeing.  Please forgive me if these questions seem obvious or pointless, but I believe the answers will help me analyze the problem.

1. You said a few times that you "give the command to spin the servo motor."  How exactly do you give the command?  Is there an external interrupt or peripheral to carry this message when the device is in standalone mode?
2. When you use VT100_printf in standalone mode, where does the message show up?
3. What source package or library provides VT100_printf?  Is it provided by TI?
4. What happens if you compile for standalone mode, but leave the JTAG/debugger attached.  Does the program work reliably and repeatedly?
5. How do you get the program into FLASH? Do you use CCS to write it, and then unplug the JTAG?  Do you use some sort of FLASH loader?  If so, which?

Archaeologist said:

I'd like to get a clearer understanding of precisely what you're seeing.  Please forgive me if these questions seem obvious or pointless, but I believe the answers will help me analyze the problem.

1. You said a few times that you "give the command to spin the servo motor."  How exactly do you give the command?  Is there an external interrupt or peripheral to carry this message when the device is in standalone mode?
2. When you use VT100_printf in standalone mode, where does the message show up?
3. What source package or library provides VT100_printf?  Is it provided by TI?
4. What happens if you compile for standalone mode, but leave the JTAG/debugger attached.  Does the program work reliably and repeatedly?
5. How do you get the program into FLASH? Do you use CCS to write it, and then unplug the JTAG?  Do you use some sort of FLASH loader?  If so, which?

I have a jtag + serial port dongle copied from Ti schematic. I have 2, one for programming and 1 where the jtag cable part is removed and only the serial ports is used.

1. I connect via serial port to my controller and have a command interpreter in my 28069M.

   I give the following commands:

  - mstart

  - speed set=1

mstart initialized  the servo controller and motor. And seems to work correctly.

 speed set=1. That gives the command to spin the servo at 1000 rpm. This is the parts that works correctly if the debugger is attached and doesn't when I run stand-alone.

2/3.

void VT100_printf (const char *format, ...)
{ char buf [100] = {0};
  int i,c;
   
    va_list arg;
    va_start (arg, format);
    i =  vsnprintf (buf, sizeof (buf), format, arg);
       va_end (arg);
    for (c=0;c<i;c++) Cmdline_putc (buf[c]);
}

void Cmdline_putc(unsigned char c)
{ fputc (c,0);
}

int fputc(int character, FILE *stream) {   
    SCI_putDataBlocking (Serialport,character);
    return 0;
}

4. If I compile for flash and download and start the program with CCS the program works correctly.

5. I use CCS to download the flash the program. switch between JTag dongle and Serial-port dongle and power cycle.

 I verified that all the compiler parameters are the same for  flash and release.

 I compared both map files and they are the same.

 I checked that all the interrupts are running and at the correct speed. My program is time critical so if there a not correct the program will fail. And the servo motor not running can be the result.

 I have CPU usages meter if that change at a given state that give a good indication something is wrong. It doesn't.

 I have a quite advanced error handling system that often gives me a warning before i noted something is wrong. It doesn't

 I look at memory dumps of the interface between my program and the intraspin library. It is impossible to look at variable level but the structure looks correct.

 I used the CLA close to 100% utilization. To verify that I don't go over the 100%. I have a speed indicator that the CLA processes don't go below a threshold.

The last thing I can think of. The intraspin library uses fix math  and I uses floating point math. That is a problem!

Because the compiler handles the library calls return value as floating point and they are different then fixed point.That doesn't work correctly. I solved that to make the conversion between fix and floating point in mine part of the code. So I do not see any reason why should be a problem again.

Edit: To verify this. I let the controller calculate the motor wire resistance. they are both standalone and with the debugger attached correct. (that happens in the mstart command)

Thanks for looking into my problem.

 

Given your answer to number 4, I believe it is extremely unlikely to be a compiler problem. The very same executable, without recompilation, works with JTAG attached, and does not work when standalone.

If I had to point a finger, I would suggest that the command interpreter is depending on something that uses JTAG instead of SCI.

It is conceivable that something is mistakenly using CIO. You could try to detect any uses of CIO in your program by replacing the seven low-level I/O functions with dummy functions that call abort, as follows. Just drop these into the source code in the standalone version.
int open(char *p, int f, int m) { abort(); }
int read(int fd, char *b, unsigned c) { abort(); }
int write(int fd, char *b, unsigned c) { abort(); }
int lseek(int fd, int o, int g) { abort(); }
int close(int fd) { abort(); }
int unlink(char *p) { abort(); }
int rename(char *o, char *n) { abort(); }

If the standalone program is not mistakenly using CIO, then this program is beyond the expertise of the compiler team. You'll need someone with more expertise debugging standalone embedded programs to help debug this, and you're much more likely to find that on the C2000 forum, or perhaps the InstaSPIN forum.

evs said:

4. If I compile for flash and download and start the program with CCS the program works correctly.

When you download a program using CCS it will write the sections to the load address regardless of if the load addresses are in flash or RAM.

If a section has a load address in RAM for some reason, when you download the program using CCS the program will be stored at the load address in RAM correctly. However, after a power-cycle any sections which were loaded into RAM will be lost thus allowing a run-time failure.

Can you run the sectti utility from the Code Generation Tools XML Processing Scripts on the .out file from your flash build and post the output from sectti.

Hi Chester,

Thanks for looking into my problem:

This you mean this:

************************************************************
REPORT FOR FILE: Servo_controllerV3.out
************************************************************
                Name : Size (dec)  Size (hex)  Type   Load Addr   Run Addr
-------------------- : ----------  ----------  ----   ----------  ----------
              .cinit :       1682  0x00000692  DATA   0x003e9049  0x003e9049
               .text :      73874  0x00012092  CODE   0x003e0000  0x003e0000
           codestart :          4  0x00000004  CODE   0x003f7ff6  0x003f7ff6
            ramfuncs :      11214  0x00002bce  CODE   0x003d8000  0x00012000
              .stack :      16384  0x00004000  UDATA  0x0000a000  0x0000a000
               .ebss :       5956  0x00001744  UDATA  0x0000c000  0x0000c000
            .esysmem :      16384  0x00004000  UDATA  0x00010000  0x00010000
            .bss_cla :          4  0x00000004  UDATA  0x00008de4  0x00008de4
          .const_cla :        176  0x000000b0  DATA   0x00008d8c  0x00008d8c
             .econst :       6068  0x000017b4  DATA   0x003da306  0x003da306
             .switch :        136  0x00000088  DATA   0x003e94c8  0x003e94c8
              IQmath :        620  0x0000026c  CODE   0x003e9392  0x003e9392
            Cla1Prog :       6716  0x00001a3c  CODE   0x003d95e8  0x00009000
     Cla1ToCpuMsgRAM :        136  0x00000088  UDATA  0x00001480  0x00001480
     CpuToCla1MsgRAM :         68  0x00000044  UDATA  0x00001500  0x00001500
        Cla1DataRam0 :        604  0x0000025c  UDATA  0x00008a24  0x00008a24
          CLAscratch :       1044  0x00000414  UDATA  0x0000881a  0x0000881a
      CLA1mathTables :        792  0x00000318  DATA   0x003f5f80  0x00008c00
             LOG_RAM :      16384  0x00004000  UDATA  0x0000e000  0x0000e000
         .scratchpad :         52  0x00000034  UDATA  0x00008800  0x00008800

------------------------------------------------------------
Totals by section type
------------------------------------------------------------
  Uninitialized Data :      57016  0x0000deb8
    Initialized Data :       8854  0x00002296
                Code :      92428  0x0001690c

evs said:

         .const_cla :        176  0x000000b0  DATA   0x00008d8c  0x00008d8c

The .const_cla section is an initialized section which has both the load address and run address as in RAM.

When the CCS debugger downloads the program the .const_cla section will be initialized, but after a power-cycle the contents of the .const_cla section will be undefined which would explain a run time failure.

To solve the problem I think you need to:

a) Change the linker command file to give the .const_cla section a load address in flash and a run address in ram, by using a linker copy table.

b) During program initialization add code to copy the .const_cla section from flash to ram.

i.e. the handling of the .const_cla section needs to be similar to the Cla1Prog and CLA1mathTables sections which already have a load address in flash and a run address in ram.

Which exact device does the project use, and did you base the linker command file upon a controlSUITE example?

I'm using the 28069M I hugely modified the CMD file from controlsuite. Because they are really made for examples and not for real applications.
I copied the .const_cla from one of the examples. If I look back at the cmd files provided by controlsuite there is no cmd file one for CLA in flash. and why isn't it shown in the memory alocation views. this is problem is impossible for me to find. frustration, frustration, frustration.
but I think it is really likely this is the problem. which is nice.

My old declaration:
.const_cla : > CLARAM1, PAGE = 1

I modified in now to: copied from clamath (RUN_START vs LOAD_SIZE???)
.const_cla : LOAD = FLASHD,
RUN = CLARAM1,
LOAD_START(_Cla1constLoadStart),
LOAD_END(_Cla1constLoadEnd),
RUN_START(_Cla1constRunStart),
PAGE = 1

Then I got the following error messages:
Multiple markers at this line
- #10265 no valid memory range(NULL)
- #10097 memory range not found:
- <a href="file:/c:/ti/ccsv6/tools/compiler/dmed/HTML/10099.html">#10099-D</a> program will not fit into
Can you give me a hint what I do wrong?

To make it complete the part of the code in my CLA init code:
memCopy((uint16_t *)&Cla1constLoadStart, (uint16_t *)&Cla1constLoadEnd, (uint16_t *)&Cla1constRunStart);


Thanks.

Chester,

I fixed this problem. Only I still have issues with the clamath. it doesn't work any more.
Can you please take a look :

.const_cla : LOAD = FLASHCLA,
RUN = CLARAM1,
LOAD_START(_Cla1constLoadStart),
LOAD_END(_Cla1constLoadEnd),
RUN_START(_Cla1constRunStart),
LOAD_SIZE(_Cla1constLoadSize),
PAGE = 1

CLA1mathTables : LOAD = CLAFLASHTABLES,
RUN = CLARAM1,
LOAD_START(_Cla1mathTablesLoadStart),
LOAD_END(_Cla1mathTablesLoadEnd),
LOAD_SIZE(_Cla1mathTablesLoadSize),
RUN_START(_Cla1mathTablesRunStart),
PAGE = 1

Cla1Prog : LOAD = FLASHD,
RUN = CLAPROG,
LOAD_START(_Cla1funcsLoadStart),
LOAD_END(_Cla1funcsLoadEnd),
LOAD_SIZE(_Cla1funcsLoadSize),
RUN_START(_Cla1funcsRunStart),
PAGE = 0

and in the cla init everything is copied:
memCopy((uint16_t *) &Cla1funcsLoadStart, (uint16_t *) &Cla1funcsLoadEnd,
(uint16_t *) &Cla1funcsRunStart);
// Copy CLA math tabes from flash to RAM
memCopy((uint16_t *)&Cla1mathTablesLoadStart, (uint16_t *)&Cla1mathTablesLoadEnd, (uint16_t *)&Cla1mathTablesRunStart);
// Copy CLA_const
memCopy((uint16_t *)&Cla1constLoadStart, (uint16_t *)&Cla1constLoadEnd, (uint16_t *)&Cla1funcsRunStart);

Somehow the math result are now always zero. Any suggestions?


curiously enough if you google .const_cla there is no example how to copy from flash to ram. En even in the flash cla_math cmd file the const_cla are only defined in ram.

Chester Gillon you're my hero of the day!!!
This solved my problem!!
I really doubt if I ever found it my self.
Thanks!!!