This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Double Precision Floating Point Accuracy Problem

Other Parts Discussed in Thread: TMS320F28335

The Problem is when i declare variable "a" of type long double and store some value in it a = 32224.61593030503 in it.

When i read the variable "a" in the watch window of Code Composer Studio. The Precision of the value in that variable is lost it shows me the value a = 32224.615234375.

 I am using Code Composer Studio V3.3 and Code Generation Tools v 6.2.4 on Wondows 7. The board i am using is TMS320F28335 ezDSP. The debugging is performed using Spectrum Digital XDS510USB Plus JTAG Emulator

  • Hi,

    I see the same behavior. I then checked the actual memory value on one of the many IEEE-754 data converters around on the internet and they agree with CCS: the 64-bit value in memory (0x40DF782760000000) translates to 32224.615234375.

    In this case I don't see a way to increase precision given the C2000 compiler's long double is specified as a IEEE 64-bit double precision value.

    Hope this helps,

    Rafael

  • desouza said:
    I then checked the actual memory value on one of the many IEEE-754 data converters around on the internet and they agree with CCS: the 64-bit value in memory (0x40DF782760000000) translates to 32224.615234375.

    Agree with the result of the IEEE-754 data converter. However, the hex value had the last 7 hex digits as all zeros, which suggests there might be some truncation occurring somewhere.

    If I feed the original posters value of 32224.61593030503 into a IEEE-754 data converter the 64-bit double hex value reported in 40DF78276B66F12E which has the first 9 hex digits matching your value, but non-zero values in the last 7 hex digits.

    Therefore, is the error that the  a = 32224.61593030503  assignment is loosing some precision?

  • Hamza Ahmed24 said:
    The Problem is when i declare variable "a" of type long double and store some value in it a = 32224.61593030503 in it.

    The problem is that the unsuffixed literal constant 32224.61593030503 is implicitly double precision.

    A "L" suffix needs to be added to the constant to make the constant explicitly long double precision.

    e.g. with the following program on a TMS320F28335:

    #include <stdio.h>
    
    long double  a = 32224.61593030503L;
    long double  b = 32224.61593030503;
    
    int main(void) {
        printf ("sizeof(a)=%lu  sizeof(b)=%lu\n", sizeof(a), sizeof(b));
        printf ("a=%.16Lg\n", a);
        printf ("b=%.16Lg\n", b);
    	
    	return 0;
    }
    

    The CCS 6.1 debugger watch window reports the value of a as "32224.61593030503", and the value of b as "32224.615234375".