TDA2P-ACD: Missed IPC messages

Part Number: TDA2P-ACD

Query as below:
Our implementation flow- Good case

  1. System Power up

  2. Core A15 starts and doing some CRC Validation

  3. Core A15 waits for Core M40 to request the CRC Validation status

  4. Core M40 turned ON lately (Core A15 turned ON early) and send Validation status request to A15

  5. M40 waits to receive the response from A15 with a timeout of 500ms

  6. A15 received the request and respond back with the validation status to M40

  7. M40 received and validation status success

Bad Case

  1. System Power up

  2. Core A15 starts and doing some CRC Validation

  3. Core A15 waits for Core M40 to request the CRC Validation status

  4. Core M40 turned ON lately (Core A15 turned ON early) and send Validation status request to A15

  5. M40 waits to receive the response from A15 with a timeout of 500ms

  6. A15 NEVER received the request and hence not responding back with the validation status to M40

  7. M40 timed out and reports failure (defined error code)

In Core A15 multiple task initialization is happening during the system boot. One of the task is Log receiver service. This task is 50 ms periodic . It sends the IPC message to all instance of Remote Log Sender to  request to initialize the remote log, providing with address of the remote log buffer in the shared memory. There is no Acknowledge and IPC RX. It's regardless of the status of each instance. This task sending IPC message to the all the cores (EVE1, EVE2, M40, DSP1, DSP2).

Now, We are suspecting if the Bad case (missed IPC message) arises due to IPC resource not present. May be some timing issue or the mailbox ISR implementation missed interrupt if two interrupt comes withing same time.

we can discuss more as this conversation goes on.

  • Partha,

    I just returned from a business trip and I will be delayed  1-2 days in catching up with your thread.

    Thank you for your patience.

    -Josue

  • Hi Josue, 
    Hope you are doing good. Thank you for your response.
    Please see my query and looking forward to hearing a response from you.

    Core A15 not receiving IPC message coming from Core M40. This issue occur 2~5 in every 100 reset cycles.
    See below:

    Mailbox ISR implementation Approach No. 1
    START Mailbox ISR
    1. Read interrupt status (which queues have messages)
    2. Repeat while interrupt status is non-zero:
    FOR each queue:
    IF this queue has interrupt (message available):
    → Read one message from that queue (FIFO pop)
    → Process the message (callback)
    Clear interrupt for this queue immediately
    END FOR
    → Read interrupt status again
    END WHILE
    END Mailbox ISR

    Here, ISR reads interrupt status and starts processing queues
    For each queue: Reads message
    and immediately clears interrupt bit for that queue (see the for loop)

    Mailbox ISR Implementation Approach No. 2
    START Mailbox ISR
    1. Read interrupt status (which queues have messages)
    2. Repeat while interrupt status is non-zero:
    Initialize handledIrq = 0 (to track processed queues)
    FOR each queue:
    IF this queue has interrupt (message available):
    → Read ONE message from the queue (FIFO pop)
    → Process the message (callback)
    → Mark this queue as handled
    (add its bit to handledIrq)
    END FOR
    IF any queue was handled:
    Clear interrupts for all handled queues (single write)
    → Read interrupt status again
    END WHILE
    END Mailbox ISR

    Here, in this implementation, I have tested for ~2500+ reset cycles, Issue NOT seen
    Instead of clearing immediately in for loop, Clearing the interrupt after processing all queues. Done using accumulated mask handledIrq


    Question:

    1. How does the issue get resolved by preventing premature interrupt de-assertion? Issue not seen in Approach No.2.
    2. Specifically, what problem occurs when we read a message from a queue and immediately clear the interrupt for that queue before moving on to process other queues?
    3. Could this early clearing of the interrupt lead to missed messages or incorrect behavior, and if so, under what scenario does this happen?


    Thank you for your time.

    BR,
    Partha

  • Hello  ,

    I will have to delay my response further due to bandwidth. Help me understand what SW you are using, what SDK version are you on?
    Are you looking at any mailbox examples?

    -Josue

  • We do not use SDK for application. Our Secondary Boot Loader (SBL) is from TI SDK 03_05.
    Our goal is to communicate Core A15 with Core M40 in our application
    Below is the steps how we are communicating-

    1. System Power up

    2. Core A15 starts and doing some CRC Validation

    3. Core A15 waits for Core M40 to request the CRC Validation status

    4. Core M40 turned ON lately (Core A15 turned ON early) and send Validation status request to A15

    5. M40 waits to receive the response from A15 with a timeout of 500ms

    6. A15 received the request and respond back with the validation status to M40

    7. M40 received and validation status success

    But We are facing an issue, which is sometimes A15 failed to receive message from M40.
    Messages are taken care by the Mailbox ISR which we have implemented.
    From previous comment I have tried to show you two approaches how we are implementing mailbox ISR: Approach 1 and Approach 2. 
    We noticed that A15 failure to receive the message occurring in Approach 1 but no failure observed in Approach 2.

    We want to understand why Approach 2 is working. Could you please help us to check. We want your suggestion or any TI Mailbox ISR implementation example.

    BR,
    Partha

  • Partha,

    Unfortunately, only the SDK examples are supported. Do you mean that you are writing your own Mailbox drivers? 
    -or-
    Are you using the drivers from this SDK?

    TI SDK 03_05

    This is the RTOS SDK version number right?

    -Josue