• State Resolved
  • Locked Locked
  • Replies 9 replies
  • Answers 1 answer
  • Subscribers 53 subscribers
  • Views 705 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Compiler/TM4C1294NCPDT: Lock-free synchronization and volatile

Franco Souza
Prodigy 150 points

Part Number: TM4C1294NCPDT

Tool/software: TI C/C++ Compiler

I have a large global data structure that is shared between two threads of execution (I'm using TI-RTOS, but the threads might as well be ISRs), in producer-consumer fashion, and want to use a flag to indicate that the data is ready to be read. I always assumed all I'd have to do is declare the flag as volatile and everything would work. However, my understanding of this answer is that this is incorrect: while the writes and reads to the flag are guaranteed to be consistent, I can't guarantee their ordering with respect to reads and writes to the global data structure. The only way to guarantee that this ordering will be preserved is to declare the large data structure as volatile.

This, however, is very awkward if the large data structure is a class, since C++ does not provide default constructors and assignmentfor volatiles (so I have to define these manually for all types used by my class, effectively punishing me for using OO). So, it seems the correct solution is to implement some sort of memory fencing. The linked topic mentions that disabling interrupts implements a full memory barrier, so I'm very tempted to just use a __disable_irq() something followed by __enable_irq() to avoid reordering. Would that work? Is there a better solution?

  • 0
    TI__Guru* 96960 points
    Hi Franco,

    TI-RTOS has several solutions to this atomic updates. The best solution depends on several factors. For example:
    1. What type of threads (Hwi, Swi and/or Task) will be accessing the global memory.
    2. # readers/#writers of the memory
    3. How many accesses need to be atomic. Note an access could be reads, writes, or a combination.

    A. If number of accesses are small, a simple Hwi_disable/Hwi_restore is the easiest, fastest and works for any thread. Just make sure to limit the time spent with interrupts disabled. Also, never call a function that could block or result in a context switch (e.g. Semaphore_pend, Task_sleep, etc.). Note: Hwi_disable/restore can be nested also as long as the order is preserved. This handy if you call a function that might disable interrupts also.
    key1 = Hwi_disable()
    key 2 = Hwi_disable()
    //blah
    Hwi_restore(key2)
    Hwi_restore(key1)

    B. If the number of accesses are getting too large for Hwi_disable/Hwi_restore and only Tasks will be accessing the data, a GateMutex (or GateMutexPri) is probably your best bet. It allows nesting (with the same constraints as nesting Hwi_disable/Hwi_restore).

    C. If the number of accesses is getting too large for Hwi_disable/Hwi_restore and any type of thread might be accessing the data, the solution is a little harder. Here you can use a binary Semaphore. For the ISR's use a timeout of zero and check the return. If you don't have semaphore, that means you cannot access the data in a safe manner. What you do in this case depends on your application.

    Note: if only tasks are accessing the data, you can do a cute trick. Make all these threads the same priority. Then you don't need any synchronization since TI-RTOS is preemptive. So you are guaranteed that task of the same priority will not preempt you. Of course, don't call in blocking calls during the accessing of the data.

    Todd
  • 0 in reply to ToddMullanix
    Prodigy 150 points

    Hi Todd, thanks for the answer.

    Ok, so let me see if I understand you: Hwi_disable and Hwi_restore impose "hard sequence points" on code, so the following code is correct lock-free programming:

    // Shared data:

    LargeData global;

    bool dataAvailable = false;

    // Producer Thread:

    void producerThread()

    {

      while(1)

      {

        // Do some work

        Hwi_disable();

        bool canWrite = !dataAvailable;

        Hwi_restore();

        if (canWrite)

        {

          global.modify(); // Writes to global

          Hwi_disable();

          dataAvailable = true;

          Hwi_restore();

        }

      }

    }

    // Consumer thread:

    void consumerThread()

    {

      while (1)

      {

        Task_sleep(10000);

        Hwi_disable()

        bool canRead = dataAvailable;

        Hwi_restore();

        if (canRead)

        {

          doSomeWork(global); // access to global. Might even write to it.

          Hwi_disable();

          dataAvailable = false;

          Hwi_restore();

        }

      }

    }

    Is the code above correct? Also, could I avoid disabling interrupts in some places by, say, declaring dataAvailable as volatile?

  • 0 in reply to Franco Souza
    TI__Guru* 96960 points
    Hi Franco,

    I need to look at it a bit , but is there just one consumer and one producer thread? Also, what type of threads are they...Hwi, Swi, Task, POSIX pthread, or a mixture?

    Todd
  • 0 in reply to ToddMullanix
    Prodigy 150 points
    Hi Todd,

    In my particular case, I only have one producer and one consumer. The consumer is a Swi that must run periodically (so it doesn't actually contain the eternal loop shown, just the inner part) and the producer is a Task.
  • 0 in reply to Franco Souza
    TI__Guru* 96960 points
    Hi Franco,

    Can you check that logic? Calling Task_sleep in a Swi is not a good thing:) Also if the producer is a Swi, it has an infinite loop (as does the consumer thread).

    Todd
  • 0 in reply to ToddMullanix
    Prodigy 150 points
    oh sorry, I explained incorrectly, Task_sleep() is also not there (I wrote the example as if they were both Tasks, but in my actual code the consumer Swi is posted by the timer interrupt. The Swi does its thing without sleeping or looping, and at the end checks if there's some work to be done coming from the producer). The producer Task really is supposed to be infinite. (I guess that could only be a problem if it never sleeps or waits and I have Tasks with less priority with work that needs to be done, but that's really what I want).
  • 0 in reply to Franco Souza
    TI__Guru* 96960 points

    Thanks. I have some options for you.

    1. If the consumer Swi really only needs to run when there is new data present (e.g. does not need to be driven by a timer), you can do this. Note: this approach requires there is only one producer task. This works because the Swi_post will context switch to the Swi and it will complete before the Task can run again.

    // Shared data:

LargeData global;
Swi_Handle consumerHandle;

void producerThread()
{
   while(1) {

    // Do some work

    global.modify(); // Writes to global
    
    Swi_post(consumerHandle);
  }
}

// Consumer thread:

void consumerThread()
{      
      doSomeWork(global); // access to global. Might even write to it.
}

    2. If the reading/writing of the global data is not too intensive (i.e. not too many cycles...and you need to determine what's too many) and you want the timer to drive the Swi. Also there are no blocking calls while you are doing this, then there is this. Again this works since there is only 1 consumer and it is a Swi

    LargeData global;
    Bool dataUpdated = FALSE;

void producerThread()
{
   while(1) {

    // Do some work
    key = Hwi_disable();
    global.modify(); // Writes to global
        dataUpdated = TRUE;
    Hwi_restore(key);    
    
  }
}

// Consumer thread:

void consumerThread()
{      
      key = Hwi_disable();

          if (dataUpdated == TRUE) {
          doSomeWork(global); // access to global. Might even write to it.
              dataUpdated = FALSE;
          }
      Hwi_restore(key);     
}

    Note: the Hwi_disable/Hwi_restore in the consumerThread is really not needed if you are sure there is only one consumer thread (and it is a Swi) and the producer thread(s) are tasks. This is because the task will never preempt a Swi.

    3. Finally if you want something general purpose (e.g. multiple producer/consumers and different types of threads) and so you can easily change it in the future. Again the consumer is driven by the timer.

    LargeData global;

Semaphore_Handle producerHandle;  //binary initialized to 1
Semaphore_Handle consumerHandle;  //binary initialized to 0

void producerThread()
{
   while(1) {

    // Do some work

    if (Semaphore_pend(producerHandle, 0) == TRUE) {
        
        global.modify(); // Writes to global
        
        Semaphore_post(consumerHandle);        
  }
}

// Consumer thread:

void consumerThread()
{      
    if (Semaphore_pend(consumerHandle, 0) == TRUE) {
        doSomeWork(global); // access to global. Might even write to it.
        Semaphore_post(producerHandle);
    }
}

    Todd

  • 0 in reply to ToddMullanix
    Prodigy 150 points

    Todd, thanks for the thorough answer. One last thing regarding your second suggestion though: as you mentioned, if handling global takes too long, I could run into trouble (because Hwis will be disabled by a long time). That's exactly why I wanted lock-free code. In my case, global is kinda big (a few hundred bytes) and the Swi runs really fast (every 50us, but I would make it faster if I could). Could I just disable Hardware interrupts when writing to the flag variable, using it as an atomic flag? Here's a modified version of your second suggestion with what I have in mind: (no other threads touch the global data)

    LargeData global;
Bool dataUpdated = FALSE;

void producerThread()
{
  while(1) {
    /* Do some work. May interact with other threads though other interfaces. */
    key = Hwi_disable();
    bool canWrite = !dataUpdated; /* will not write to global unless the consumer already processed it. */
    Hwi_restore(key);
    if (canWrite) {
      global.modify(); /* Write happens before disabling interrupts. Consumer is not reading it now because only the producer can set dataUpdated, and we just checked it */
      key = Hwi_disable();
      dataUpdated = TRUE;
      Hwi_restore(key);
    } else {  /* global data was not processed yet */
      /* store important stuff to be made to global somewhere else, for later use */
    }
  }
}

// Consumer thread. Is a Swi that runs periodically:
void consumerThread()
{
  key = Hwi_disable(); /* I think I don't need these, but just in case. */
  bool canRead = dataUpdated;
  Hwi_restore(key);
  if (canRead == TRUE) {
      doSomeWork(global); /* access to global. Might even write to it. */
      key = Hwi_disable(); /* I think I don't need those, but just in case. */
      dataUpdated = FALSE;
      Hwi_restore(key);
  } else {
      /* Usual work that does not involve global. Very fast. */
  }
}

    PS: sorry for the edits, I'm having some trouble with the insert code tool and newlines.

  • 0 in reply to Franco Souza
    TI__Guru* 96960 points
    Hi Franco,

    I don't see any holes in your approach. I agree that the Swi does not need the Hwi_disable/restores but these are fast APIs, so keep them or add a big comment on why they are not needed (for future maintenance).

    Todd