This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

RTOS/AM5726: A lower priority task "interferes" with the job of a higher priority task

Part Number: AM5726
Other Parts Discussed in Thread: SYSBIOS

Tool/software: TI-RTOS

I have a few tasks running with different priorities on IPU, whose priorities are set between 1 to 15 during the task creation.

Among them, a low priority task (with priority 3) "interferes" with the job of a high priority task (priority 15).

The lower priority task is responsible to communicate with Input/Output modules through UART driver: Read/Write requests are sent every 125 ms by the program and are answered by the IO modules.

The higher priority task is responsible to monitor system diagnostics through ADC channels: The ADC measurements are triggered by a GPIO interrupt which occurs every 5 ms.

If we disable the low priority task, the high priority task measures the system diagnostics without any problem.

As soon as we enable the low priority task, the high priority task fails to measure the system diagnostics in a timely fashion, which results in wrong measurements. In fact, the time we spent in one loop of the high priority task is doubled from time to time when the low priority task is active! It seems that the high priority task is sometimes interrupted by the low priority one.

 Could you please give us a little bit of information regarding our issue and tell us how we should proceed to obtain more information about this problem?

Thank you!

  • Hi,

    Can you elaborate the SYSBIOS version used for this test? For the tasks, are they all HWI, SWI or just tasks with different priorities? In SYSBIOS, tasks are readied for execution in strict priority order; tasks of the same priority are scheduled on a first-come, first-served basis. The priority of the currently running task is never lower than the priority of any ready task. Conversely, the running task is preempted and re-scheduled for execution whenever there exists some ready task of higher priority.

    In RTOS ROV view, there are HWI, SWI and task information. Can you see they are pure tasks and ADC is priority 15 and UART is 3? This is just driven by 2 different timers?

    Regards, Eric
  • Hi,

    Any update?

    Regards, Eric
  • Hello,

      Thank you for your reply and sorry for my late answer.

      I was in vacation and then I had to spend some time to setup the debugging session using CCS.

      Versions:

        Sysbios Version:  6.46.05.55

        CCS Version: 7.4.0.00015

        Compiler Version: TI v17.9.0.STS (ti-cgt-arm)

        XDC Tools Version: 3.32.01.22

       In our IPU program, we have a combination of HWIs, SWIs, and tasks. The 2 tasks I was talking about in my previous post (the ones that interact with ADC and UART), are "normal" tasks created using Task_create API.

       As you suggested, I tried to use Runtime Object View tool from CCS to see the tasks information. However,  I get the following error when trying to access the Task information:

       "Target memory read failed at address: 0x8007b86c, length: 80 This read is at an INVALID address according to the application's section map. The application is likely either uninitialized or corrupt."

       Do you have any suggestion how I can fix this issue to be able to see the task information using ROV? Do you think it may help if I use a newer version of XDC Tools (3.50.4.43 for example)?

     Thanks,

    Reza

  • Reza,

    Yes, you can try a different setup XDCtools to see if it works. Also, I am not sure if you use any TI processor SDK RTOS driver or not. If you used TI UART driver, the driver supports polling mode (by CPU)? Is ADC using interrupt? You may try that if helps: software-dl.ti.com/.../index_device_drv.html

    Regards, Eric
  • Hi Eric,

       Hope you are fine.

       I installed the version 4.03.00.05 of RTOS Processor SDK and used the version 3.50.4.43 of XDCtools.

       Nevertheless, I still get the same following error when I try to see the Task information from Runtime Object View Tool:

        "Error: java.lang.Exception: Target memory read failed at address: 0x8007c4ec, length: 80 This read is at an INVALID address according to the application's section map. The application is likely either uninitialized or corrupt."

       Any suggestion how I can fix this issue? Using ROV could be the key to know more about and fix my issue with the tasks.

       

      To answer your questions:

       - Yes, we use TI UART driver (UART_socGet/SetInitCfg, UART_init, UART_open, UART_write and UART_read APIs).

          Read works in Blocking mode and Write in non-Blocking mode. Please see below the function which is responsible to initialize UART:

    int32_t DUART485_slInit()
    {
       UART_Params params;
       UART_HwAttrs uart_hwAttrs;

       UART_Params_init(&params);
       params.baudRate = DUART_BAUD_RATE; // -> 115200
       params.readEcho = UART_ECHO_OFF;
       params.readDataMode = UART_DATA_BINARY;
       params.writeDataMode = UART_DATA_BINARY;
       params.writeMode = UART_MODE_CALLBACK;
       params.writeCallback = DUART485_voCbTx;
       params.readMode = UART_MODE_BLOCKING;
       params.readTimeout = DUART_READ_TIMEOUT; // -> 30
       params.readReturnMode = UART_RETURN_FULL;

       /* Modify the UART default configuration */
       UART_socGetInitCfg(UART_INSTANCE, &uart_hwAttrs);
       uart_hwAttrs.baseAddr = SOC_UART5_BASE + IPC_REGISTER_OFFSET;
       uart_hwAttrs.intNum = 48; /* use interrupt #48 */
       uart_hwAttrs.txTrigLvl=UART_TXTRIGLVL_8; /* TX FIFO trigger at lowest level */
       UART_socSetInitCfg(UART_INSTANCE, &uart_hwAttrs);

       /* configure the irq corssbar to route the irq from uart5 to ipu1-core1-irq48 */
       DIRQ_voCustomCSL_xbarIpuIrqConfigure(1, CSL_XBAR_INST_IPU1_IRQ_48, CSL_XBAR_UART5_IRQ);
       UART_init();

       uart_handle = UART_open(UART_INSTANCE, &params);
       if(uart_handle == NULL)
       return ACDF_FATAL_ERROR;
       return ACDF_OK;
    }     

       -  As I partly explained in my first post, our high priority task is triggered by a GPIO interrupt (every 5 ms) and then configures and reads the ADC channels through the I2C driver.

          The I2C driver does not use interrupts and works based on polling. Please see the code that reads from and writes to the i2c module:

    static uint32_t DI2C_ulMasterWriteChar(uint8_t i2cSlaveAddr, uint8_t Value, bool bStop)
    {
       uint32_t errStat = 0;

       I2CFIFOClear(CUSTOM_SOC_I2C_BASE, I2C_TX_MODE);
       I2CFIFOClear(CUSTOM_SOC_I2C_BASE, I2C_RX_MODE);

       /*Clear all interrupt status*/
       I2CMasterIntClearEx(CUSTOM_SOC_I2C_BASE, I2C_INT_ALL);

       /* Set i2c slave address */
       I2CMasterSlaveAddrSet(CUSTOM_SOC_I2C_BASE, i2cSlaveAddr);

       /* Set data count */
       I2CSetDataCount(CUSTOM_SOC_I2C_BASE, 1);

       /*
       ** Configure i2c as master-transmitter and to generate stop condition
       */
       I2CMasterControl(CUSTOM_SOC_I2C_BASE, I2C_CFG_MST_TX);

       /* generate start */
       I2CMasterStart(CUSTOM_SOC_I2C_BASE);

       /* wait for bus busy */
       while(I2CMasterBusBusy(CUSTOM_SOC_I2C_BASE) == 0);

       /* wait for transmit ready or error */
       while((I2CMasterIntRawStatusEx(CUSTOM_SOC_I2C_BASE, I2C_INT_TRANSMIT_READY) == 0U) &&
       (I2CMasterIntRawStatusEx(CUSTOM_SOC_I2C_BASE, I2C_INT_ARBITRATION_LOST | I2C_INT_NO_ACK |
       I2C_INT_ACCESS_ERROR | I2C_INT_STOP_CONDITION ) == 0U));

       errStat = I2CMasterIntRawStatusEx(CUSTOM_SOC_I2C_BASE, I2C_INT_ARBITRATION_LOST | I2C_INT_NO_ACK | I2C_INT_ACCESS_ERROR);

       /* Send the data */
       I2CMasterDataPut(CUSTOM_SOC_I2C_BASE, Value);

       /* clear transmit ready interrupt */
       I2CMasterIntClearEx(CUSTOM_SOC_I2C_BASE, I2C_INT_TRANSMIT_READY);

       if(0 == errStat)
       {
          while(I2CMasterIntRawStatusEx(CUSTOM_SOC_I2C_BASE, I2C_INT_ADRR_READY_ACESS) == 0U);
       }

       if(bStop)
       {
          /* generate stop when requested */
          I2CMasterStop(CUSTOM_SOC_I2C_BASE);
          /* wait for stop to happen */
          while(I2CMasterIntRawStatusEx(CUSTOM_SOC_I2C_BASE, I2C_INT_STOP_CONDITION) == 0U);
       }
       /* wait for register access ready */
       while(I2CMasterIntRawStatusEx(CUSTOM_SOC_I2C_BASE, I2C_INT_ADRR_READY_ACESS) == 0U);

       return errStat;
    }

    ******************************************************************************/

    int32_t DI2C_slOnlyRead( uint8_t ucSlaveId, uint8_t* pucData, uint8_t ucSizeData)
    {
       uint32_t i;
       uint32_t errStat = 0, fatalError = 0;

       I2CFIFOClear(CUSTOM_SOC_I2C_BASE, I2C_TX_MODE);
       I2CFIFOClear(CUSTOM_SOC_I2C_BASE, I2C_RX_MODE);

       /*Clear all interrupt status*/
       I2CMasterIntClearEx(CUSTOM_SOC_I2C_BASE, I2C_INT_ALL);

       /* Set i2c slave address */
       I2CMasterSlaveAddrSet(CUSTOM_SOC_I2C_BASE, ucSlaveId);

       /* Set data count */
       I2CSetDataCount(CUSTOM_SOC_I2C_BASE, ucSizeData);

       /*
       ** Configure i2c as master-receiver and to generate stop condition
       */
       I2CMasterControl(CUSTOM_SOC_I2C_BASE, I2C_CFG_MST_RX);

       /* generate start */
       I2CMasterStart(CUSTOM_SOC_I2C_BASE);

       /* wait for bus busy */
       while(I2CMasterBusBusy(CUSTOM_SOC_I2C_BASE) == 0);

       for (i = 0; i < ucSizeData; i++)
       {
          /* wait for receive ready or error */
          while( (I2CMasterIntRawStatusEx(CUSTOM_SOC_I2C_BASE, I2C_INT_RECV_READY) == 0U)
          &&
          (I2CMasterIntRawStatusEx(CUSTOM_SOC_I2C_BASE, I2C_INT_ARBITRATION_LOST | I2C_INT_NO_ACK | I2C_INT_ACCESS_ERROR ) == 0U));

          errStat = I2CMasterIntRawStatusEx(CUSTOM_SOC_I2C_BASE, I2C_INT_ARBITRATION_LOST | I2C_INT_NO_ACK | I2C_INT_ACCESS_ERROR);

          /* if we get an error, do a stop and return failure */
          if(errStat) {
             fatalError=1;
             break;
          }

          /* Read the data */
          pucData[i] = I2CMasterDataGet(CUSTOM_SOC_I2C_BASE);

          /* clear receive ready interrupt */
          I2CMasterIntClearEx(CUSTOM_SOC_I2C_BASE, I2C_INT_RECV_READY);
       }

       if(fatalError == 0U)
       {
          /* wait for register access ready */
          while(I2CMasterIntRawStatusEx(CUSTOM_SOC_I2C_BASE, I2C_INT_ADRR_READY_ACESS) == 0U);
       }

       /* generate stop when requested */
       I2CMasterStop(CUSTOM_SOC_I2C_BASE);
       /* wait for stop to happen */
       while(I2CMasterIntRawStatusEx(CUSTOM_SOC_I2C_BASE, I2C_INT_STOP_CONDITION) == 0U);
       /* wait for register access ready */
       while(I2CMasterIntRawStatusEx(CUSTOM_SOC_I2C_BASE, I2C_INT_ADRR_READY_ACESS) == 0U);

       if(errStat)
          return ACDF_FATAL_ERROR;

       return ACDF_OK;
    }

         

    Best,

    Reza

  • Hi,

    In the UART code, will change params.readMode = UART_MODE_BLOCKING to CALLBACK mode help? If there are low priority task and high priority task and there will be task switching, which may take several hundreds of cycles. Another thing is the GPIO latency when running from IPU core, see e2e.ti.com/.../756351.

    If you can use a scope to measure the GPIO toggle, do you see a strict 5ms ON/OFF when there is no UART task? And the 5ms toggle showed big jitter when you use UART? If no scope set up, you may use a timestamp tracking each time the GPIO ISR is entered?

    Regards, Eric
  • Are you using mutex's in your code, e.g. for critical sections, etc.?  If you have a 3rd "medium" priority task involved, you can end up with priority inversion:

    1. Low priority task grabs mutex.
    2. High priority task pre-empts low priority task.
    3. Medium priority task becomes ready.
    4. High priority task attempts to grab the mutex already owned by low priority task.  It correspondingly is blocked.
    5. Medium priority task will now run for a long time (till it blocks due to some other reason).
    6. Low priority task completes critical section and returns mutex.
    7. High priority task runs again.

    In the above example, the medium priority thread is effectively holding off the high priority thread.

    There's a different mutex called MutexPri that can solve this problem.  Here's how it looks:

    1. Low priority task grabs mutex.
    2. High priority task pre-empts low priority task.
    3. Medium priority task becomes ready.
    4. High priority task attempts to grab the mutex already owned by low priority task.  It correspondingly is blocked, which temporarily elevates low priority task to high priority.
    5. Low priority task (which is temporarily high priority) completes critical section, and returns to normal low priority.
    6. High priority task runs again.
    7. Medium priority task can run once high priority task blocks.

    I suspect you have something along these lines occurring in your system.  It is a classic issue for multi-threaded systems.

    Best regards,
    Brad

  • Hi Eric,

      Thank you for your reply and support.

       Actually, after a few tests and the discussions I had with my colleagues, the Task Interference problem was fixed by 

      -> Changing the params.writeMode from CALLBACK  to BLOCKING mode.

      Again thank you very much for your support. 

    Cheers,

    Reza

      

  • For drivers that are configurable, usually BLOCKING is intended for tasks while CALLBACK is intended for Swi.