This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

RM48 FreeRTOS, taskDelay() SCI DMA problem

Other Parts Discussed in Thread: HALCOGEN

Hello,


I have the Hercules USB development board running Halcogen generated code based on the 950ZWT FreeRTOS example. I am using version 3.06 of Halcogen and 5.3.0 of Code Composer Studio.

I have two tasks, a Blinky Task which toggles an LED and an SCI Transmission task which outputs messages which I view on the Code Composer Studio Terminal. The Blinky Task sends messages to the SCI Transmission task to be printed via an sci_printf call (which does simple printf style formatting and sends the message via a FreeRTOS message queue to the SCI Transmission task). Everything works fine, until the taskDelay() function executes in the Blinky Task, after it executes, although a message is sent to the SCI Transmission task, and the task attempts to send the message out, the message does not go out. It's as if the DMA transfer completely fails.

If I set the taskDelay to 1, for 1 clock tick, instead of 3000, for 3 seconds. Everything works and all the messages are correctly sent via DMA transfer to the SCI. I am confused.

Here is the Blinky Task:

static portTASK_FUNCTION( BlinkyTask, pvParameters )
{
    const portTickType delay = 3000/portTICK_RATE_MS;    // 3 second delay?
    portTickType lastTickCount;

    sci_printf("Blinky task up and running\r\n");

    SET_LED(LED_MIDDLE_TOP);
    while(1)
    {
        lastTickCount = xTaskGetTickCount();

        sci_printf("Blinky Task ... before task delay, tick count is %d\r\n", lastTickCount);

        vTaskDelay(1);

        sci_printf("Blinky Task ... after task delay\r\n");
        TOGGLE_LED(LED_MIDDLE_TOP);
        sci_printf("Blinky Task ... toggled led\r\n");
    }
}

Here is code to set up the DMA Control Packet.

void dmaConfigTxCtrlPacket( uint32 sourceAddress,
                            uint32 destinationAddress,
                            uint32    length) {
    dmaTxCtrlPacket.SADD = sourceAddress;            // RAM address
    dmaTxCtrlPacket.DADD = destinationAddress;        // peripheral address
    dmaTxCtrlPacket.CHCTRL = 0;                        // channel control
    dmaTxCtrlPacket.FRCNT = length;                    // source address length
    dmaTxCtrlPacket.ELCNT = 1;                        // 1 byte
    dmaTxCtrlPacket.ELDOFFSET = 0;                    // element destination offset
    dmaTxCtrlPacket.ELSOFFSET = 0;                    // element source offset
    dmaTxCtrlPacket.FRDOFFSET = 0;                    // frame destination offset
    dmaTxCtrlPacket.FRSOFFSET = 0;                    // frame source offset
    dmaTxCtrlPacket.PORTASGN = 4;                    // port b
    dmaTxCtrlPacket.RDSIZE = ACCESS_8_BIT;            // read element size
    dmaTxCtrlPacket.WRSIZE = ACCESS_8_BIT;            // write element size
    dmaTxCtrlPacket.TTYPE = FRAME_TRANSFER;            // transfer type: frame
    dmaTxCtrlPacket.ADDMODERD = ADDR_INC1;            // read address mode: post increment
    dmaTxCtrlPacket.ADDMODEWR = ADDR_FIXED;            // write address mode: fixed
    dmaTxCtrlPacket.COMBO = AUTOINIT_OFF;            // autoinit off

}

And here is the code to send the message via DMA. Note that I added the FTC interrupt enable to see if I got an FTC interrupt on the first message that fails, I didn't. And note that those first lines of code were executed before the task was started and the behavior was the same. I put the code here to see if maybe the DMA set up was corrupted, but the behavior is the same:


static void sciDMASend(int length, uint8 *data)
{
      dmaEnable();
     dmaEnableInterrupt(DMA_CH0, BTC);    // enable interrupt after receipt of data
     dmaEnableInterrupt(DMA_CH0, FTC);
     dmaReqAssign(DMA_CH0, 29);

    dmaConfigTxCtrlPacket(                    // First transmission control packet
                (uint32)(data),                // source address is message
                (uint32)(&scilinREG->TD),    // destination address transmit buffer of SCI LIN
                length);                    // number of byte to transmit
    dmaSetCtrlPacket(DMA_CH0, dmaTxCtrlPacket);
    dmaSetChEnable(DMA_CH0, DMA_HW);        // set dma chan 0 to trigger on h/w request

    scilinREG->SETINT = (1 << 16);            // enable TX DMA
}

Any ideas or suggestions would be great. Thank you.

Ruth

  • Ruth,

    Hard to say from the information above.   

    Are you using the same DMA channel each time you call the sciPrintf() function?   Is there any check between that function and the call to sciDMASend() to make sure the first DMA channel was finished before the 2nd one is enabled?
    Seems like this might be what you're trying to do by adding the vTaskDelay() call but that would just add a time delay; it would be more robust to check the DMA to see if the channel is complete before submitting another request. 

    I'd think you would have more problems with the delay of (1) rather than 3 seconds,  so this is a little odd that it's working out the other way.

    Do you have the DMA and the SCI configured to ignore suspend (register GCTRL, bits DEBUGMODE[1:0]  for DMA and Register SCIGCR1 bit CONT for SCI)?   If so you could break on the function that submits the DMA request and in the time between breaking and running again the DMA should definitely finish the first message.  Then you could further investigate why the FTC interrupt isn't occurring.   It would be good to know the state of the DMA control/status registers when this happens and the DMA channel is active still even after the message is transmitted, that could be a big clue.

    If it is, then you could program another 'dummy' DMA channel to be software triggered, and then trigger it manually by writing to the SWCHENAS register.   This would force the DMA to dump the contents of the first stalled channel into it's RAM (the current control packet set) and then you could see where it got stuck.

    There's one more tool at your disposal I should mention - the DMA has a watchpoint so you can make it break the CPU whenever it accesses a particular address.   This is something you could use if you have any doubt as to whether it actually accessed a particular address, like the end of your 1st buffer or the beginning of the 2nd buffer...  in case there's some issue in the SCI and it's not a DMA issue.

    Last, I can't tell from the code above where the dmaTxCtrlPacket is stored (is it in the processor RAM or is it mapped directly onto the DMA's RAM).   Also is it a struct using bit fields ?    Reason I ask is that the fields RDSIZE, WRSIZE, etc are all in the same 32-bit address and some even share part of the same byte, so the compiler needs to at least merge the separate writes into byte writes or higher.  Otherwise, you might be overwriting one write with a subsequent one.  It might be helpful to look at the disassembly of this section to see what code is generated.

     

  • Hi,

    Thanks for the reply, let me clarify a few things.

    To ensure that only one entity is ever writing to the SCI at a time, the SCI transmission task receives freeRTOS messages containing a message to be transmitted to the SCI. The task then blocks on a semaphore take; the semaphore is given when the BTC interrupt is received. Always on the same channel, channel 29.  So I am quite certain that no overwrites are occurring.

    Here is the sciTxTransmisisonTask main loop (note I am toggling a different LED just for debug purposes):

    for( ;; )
        {
            // Block waiting for next message
            if (sciTxMsgQ && (haveMsg = xQueueReceive(sciTxMsgQ, &msgBuf, portMAX_DELAY))) {

                SET_LED(LED_LEFT_TOP);    // Turn the LED on while we put out the message

                if (haveMsg != pdPASS || !msgBuf)
                    s = (signed char *) "sci_printf: no msg or no msgBuf\n\r";        // paranoid msgBuf set check
                else if ( (msgBuf < FIRST_MSG_BUF) || (msgBuf > LAST_MSG_BUF) )        // paranoid msgBuf range check
                    s = (signed char *) "sciprintf: msgBuf is out of range\n\r";
                else
                    s = (signed char *) BUFFER(msgBuf);

                /*
                 * get string length and verify length.
                 */
                i = strlen((char *)s);
                if (i > MSG_BUFFER_LEN)
                {
                    i = MSG_BUFFER_LEN;
                    sciErrCount++;
                }

                // send message via DMA transfer, BTC notification will give semaphore
                sciDMASend(i, (uint8 *)s);

                if (xSemaphoreTake(sciTxCompFlag, portMAX_DELAY) != pdTRUE)
                {
                    sciErrCount++;    // should never happen
                }
                returnMsgBuffer(msgBuf);

                CLEAR_LED(LED_LEFT_TOP);    // Turn the LED off while we are not doing anything.
            }

    Therefore, after a message is sent to the SCI, because the BTC interrupt did not occur (because nothing went out via the DMA, despite the fact it was written to), this task is effectively hung. Which means any further messages sent to the task sit in the queue, and the queue eventually becomes full. Then the sci_printf() function formats the message but because it cannot send the message to the sciTXTransmission task, it simply drops it.


    I have the taskDelay() in the BlinkyTask simply as function of the Blinky Task, I want the LED to blink slowly enough I can see it.

    The DMA transfer ONLY fails, i.e. nothing put out, no BTC or FTC interrupt, after the Blinky Task's taskDelay() of about 12 clock ticks or more. In all other cases, all messages sent via the DMA succeed, the FTC and BTC interrupts all occur.

    The DMA control packet is stored in CPU RAM, in a static variable of type g_dmaCTRL, which is a type generated by Halcogen.


    Hoping for more ideas, thanks for your time!

    Ruth

  • Hi Ruth,

    Is your UART bit rate about 1200?

     

  • um, no, I have it set very high, 115,200.


    Also, so I can proceed, I changed the taskDelay to taskYIELD(), and have added an additional task to read the temp sensor and ambient light data ( in  a loop, also with taskYIELD() rather than a task Delay()). I'm getting all my messages from both tasks.

    It is very odd. I tried "manually" creating a delay by simply spinning and checking the tick count returned by getTickCount() until the tick count reached 3000. That failed spectacularly by blocking all but the first message.

    I keep thinking it has to be a timing related problem, but unclear where the problem might be since I would expect that being really busy, i.e. taskYIELD() rather than taskDelay() would be worse. But that code is singing along as I type this.


    Hmm, I could try the taskYIELD approach in my "manual" delay method. But even if that works, it doesn't explain why the taskDelay is causing a problem. (One other thing I tried was playing with the priority of the tasks, they are currently all one above the Idle Task; but making one task higher or lower than the other had no effect.).


    Thanks so much for your time.

    Ruth

  • Ruth,

    Here's a potential theory that I think might go along w. a bit rate near 1200  (assuming your tick is ~1ms)..  It might actually be a better fit for 2400 baud though.  We can come back to this..

    The UART generates DMA requests when TXRDY is set, and you enable the DMA request.

    So let's say the 1st message has completed (I hope you get the 1st BTC for this one ... )  and then the last byte(s) is sitting in the SCI to be transmitted.

    If you very quickly submit the next DMA request, then you'll be writing to the DMA again before the SCI generates the next DMA request.  This is OK.

    But if the SCI finishes the last byte and generates the DMA request first, and *then* after that you submit the next DMA packet, I think what will happen is that submitting the DMA packet probably clears that request from the SCI.  You would be sitting with the SCI having TXRDY active and the DMA waiting for a DMA request (because it cleared the one SCI sent).

    I'm basing this theory on:


    To test this theory for sure, I'd probably suggest something opposite of what I suggested yesterday,

    That would be to program both the DMA and the SCI to HALT when the CPU enters debug mode.
    Then, if you break at the beginning of the sciDMASend() function when it's set up to fail, and you inspect the DMA control registers, I'd be expecting to see the bit in PEND set for the channel that youare using.    And I'd expect that after dmaSetCtrlPacket(DMA_CH0, dmaTxCtrlPacket) you would see that this bit is cleared,  and again after

    scilinREG->SETINT = (1 << 16);            // enable TX DMA

    that it's stayed cleared.

    You could also try this with a setup for 1ms and verify that the 2nd time you hit the sciDMASend() the PEND isn't set..

     If so then it might be enough to add

    static void sciDMASend(int length, uint8 *data)
    {
        dmaSetCtrlPacket(DMA_CH0, dmaTxCtrlPacket);
        dmaSetChEnable(DMA_CH0, DMA_HW);        // set dma chan 0 to trigger on h/w request
       ...
        scilinREG->CLRINT = (1 << 16);            // disable TX DMA
        scilinREG->SETINT = (1 << 16);            // enable TX DMA
    }
     

    I'm not sure if this is enough to force a new DMA request or not... but I'd probably try it first.

    Now I know the above might seem like it violates that handshake that you've got going through the OS but I don't think so.   Because,  the handshake is based on the DMA transferring the last character of the last frame (BTC).   When this occurs, there will still be just about 2 characters in the SCI that still need to be transferred;  one character (or most of one) in the shifter and another in the TX data buffer.  

    It should take about 10 bit times (assuming n81) to have the shifter finish, load from the tx data buffer, and have the TX data buffer be empty & generate the DMA request.   If you've gotten in & out of servicing the DMA *before* this occurs then I don't think the request would be lost.  But if it takes longer, that last request might be lost by writing to the control packet.   This is where I guessed 1200 baud,  although it might be 2400 baud.   12ms should come out as ~14 characters at 1200 baud and ~7 characters at 2400 baud.  Since there's software overhead it might fit the 2400 baud case better.  If your tick is 1ms and you're in this range on bit rate then I think this might be a good angle to work on.

    On the other hand if you're running 19200 and your tick is 1ms then I don't think this theory is a good fit and we might need to think through it again.

    Just as SWAG but maybe in the worst case it'll stimulate some ideas.

    EDIT(cleaned up formatting and added an (s) to the first 'byte')

  • Hi Ruth,

    115200 - maybe you can double check my math in case I'm doing something brain-dead.

    1s/115200bit = 8.6us/bit.  Then 10 bits for a character would be 86us which is much much less than even one OS tick.
    Is that about right (tick is 1ms)? 

  • Sounds right.

    It is very confusing because I am sending a LOT of messages right now and they are just flying by on the terminal (2 tasks are sending messages to be put out and yielding the processor in their main loops), but if I add the taskDelay() to block the Binky Task for 3 seconds, something goes awry.

    I will try setting up the failure to see if the scenario you suggested might be happening. But again, I am sending the messages far faster now and there are no issues.

  • Hi Ruth,

    Thanks.  The timing seems to not calculate out but the direction of having a longer delay be a problem seems right for this sort of scenario where the DMA is waiting on the SCI and the SCI is waiting on the DMA and therefore everything's 'hung'... so I appreciate you checking for this.   

    Meanwhile,  Going back to your original post:

      1) Which message is the last one that generates the BTC interrupt?  Is it

            sci_printf("Blinky Task ... before task delay, tick count is %d\r\n", lastTickCount);

     2) Which message is the one that's submitted to the DMA but doesn't begin to transfer?

    Also, you had mentioned 12 counts and I took this to mean that you'd experimented with delay and had found 11 counts would work, 12 counts in delay would fail.  Just want to confirm that this is 12ms being the threshold for failure.
    [You mentioned 3s again in your last post - so I want to make sure I didn't misunderstand the threshold at 12ms].

     

  • Hi,

    Yes, the last message to come out is the before task delay message, if taskDelay(3000) is used, no further prints come out.

    And I just successfully manually delayed by checking the tick count, and after 3000 ticks go by I toggle the LED. All the messages keep coming. Soo, I put the taskDelay(3000) back, and it worked!!

    ACK! Took out the sensor task, and it didn't work again!! Put it back and it all works again! There must be something going on in the FreeRTOS kernel.

    Tonight, I will try to check one of the failing cases to see if the scenario you suggested might be happening. If I don't get anywhere, do you want me to send you the code?

    Thanks so much for your time! This is a weird one.

  • Hi Ruth,

    Sure - if you're getting stuck then I think it's probably something fairly complex and having the code to look at would be a big help.  

    If you can't post your code here on the public forum, then I think you can send me an email directly or send me a friend request and that'll let you send it to me privately.   If it's really sensitive we can setup something even more secure.

  • Hi Anthony,

    You were absolutely RIGHT!

    The PEND bit was set, cleared, and remained cleared when the message that fails was being sent. I added the scilinREG->CLEARINT (1 << 16) to disable the interrupt before I enabled it; and that FIXED it!


    When this problem occurs, the Idle Task is running continuously because there are no tasks ready to run and there is no other task at the Idle priority leve to cause the Idle task to yield. When there is another task to run, then the Idle Task doesn't run and the problem doesn't exist. Must be something funky, ...

    In any event, problem solved! Thank you so much!


    Ruth