Compiler: fprintf in append mode very slow

user6369719

Tool/software: TI C/C++ Compiler

Hello,

I'm developing embedded software using code composer studio V10 with the TI optimized ARM compiler. I'm trying to use the CI/O utilities (which are really great) provided by TI on a Hercules TMS570LS09 target using the debug probe XDS200 from Digital Spectrum.

Every thing works very well, put when I try to write in a file that has been opened in append mode using fprintf or fputc, it's very very slow making it almost unusable. Indeed writing one single characters in a file takes almost 500ms. If the file is open in write mode it's more than 100 times faster.

I've already tried to change the buffering modes, with the following lines of code:
char buffer[256];
setvbuf(fp, buffer, _IOFBF, sizeof(buffer) / sizeof(buffer[0]));
But it doesn't improve the performance significantly.

Does anyone one knows why there is such a huge difference of speed between the two way of opening file? Is there a way to improve the write speed of file opened in append mode?

Thanls for your help

over 5 years ago

0 George Mock over 5 years ago

TI__Guru**** 249380 points

Please search the TI ARM compiler manual for the sub-chapter titled The C I/O Functions. Note how the C I/O functions like fopen require the code to run under control of Code Composer Studio. That is because the your host operating system handles the lowest level of the input/output operation.

Try a similar program on your host system. If you see similar difference in performance when using append mode, then the problem lies somewhere in the host system. Please let me know what you see.

Thanks and regards,

-George

0 user6369719 over 5 years ago in reply to George Mock

Prodigy 70 points

Hi George,

I tried to measure the performance of the foloowing function, once compile with the TI ARM compiler running on TMS570LS09 with code composer studio, and once compiled with gcc and running directly on the windows host machine.

static void file_write_method1(void)
{
    FILE *fp;
    int charcount = 0;

    if ((fp = fopen("method1.txt", "w")) != NULL)
    {

        for (charcount = 0; charcount < 20000; charcount++)
        {
            fputc((charcount % 92) + 0x30, fp);
        }

        fputc('\n', fp); /* at the end still... */

        fclose(fp);
    }
}

static void file_write_method2(void)
{
    FILE *fp;
    int charcount = 0;

    if ((fp = fopen("method2.txt", "a")) != NULL)
    {

        for (charcount = 0; charcount < 200; charcount++)
        {
            fputc((charcount % 92) + 0x30, fp);
        }

        fputc('\n', fp); /* at the end still... */

        fclose(fp);
    }
}

Target	# of fputc	method 1 execution	method 2 execution time
host	20	16 ms	84 ms
host	20'000	38 ms	116 ms
host	2'000'00	1.46 s	2.22s
target	2	550 ms	1.5 s
target	20	530 ms	7.9s
target	200	580 ms	65 s
target	20'000	19 s	?

Note that the timing may differ a bit between different trials.

before execution the two function the method1.txt and method.txt were deleted so that fopen create both files.

The timing of the method 2 on the target is so slow that it makes it unusable, ans I think it's not a host problem as when running on the host both method are much closer in term of execution time. Any idea what can the problem be?

Thanks for your help

Louis

0 user6369719 over 5 years ago in reply to user6369719

Prodigy 70 points

I think I understood why fprintf was so slow.

When a file is open in append mode, every-time we want to write a character using fputc, the file pointer is moved to the end of the file using fseek. For whatever reason this operation is very slow, even if the file is almost empty.

The fprintf in my use case was writing every characters one by one using fputc making it very slow. A workaround is using fputs instead of fputc, indeed this function will write a complete string and the file pointer shall only be moved once to the end of the file. Thus using this function we will have almost the same performance as writing in a file opened in write mode.

Here is the same code as above this do what I want and isnt too slow:

static void file_write_method3(void)
{
    char buffer[2048] ={'\0'};
    FILE *fp;
    int charcount = 0;

    for (charcount = 0; charcount < 2000; charcount++)
    {
        buffer[charcount] = (char)((charcount % 92) + 0x30);
    }

    buffer[charcount] = '\n'; /* at the end still... */

    if ((fp = fopen("method3.txt", "a")) != NULL)
    {
        fputs(buffer, fp);
        fclose(fp);
    }
}

Code Composer Studio™︎

Code Composer Studio forum

Compiler: fprintf in append mode very slow