This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Multiple interrupts from Ethernet packet

I have a working driver that sends and receives Ethernet packets on an EVM6678L. Tracing the behaviour of this driver has revealed that it is getting FIVE interrupts for every Ethernet transmission or reception.

Just looking at transmission, the return queue is 654 and I am catching the interrupt associated with the queue manager event 93 (QM_INT_PASS_TXQ_PEND_14).

I have searched the TI documentation but it is typically obscure on these interrupts; I have been unable to find a definition of exactly what "queue pend" actually means or exactly what causes such events to be signalled. Can anyone point me at a TI document that gives this information or are users supposed to guess?

Each transfer is made up from two linked packets. The return policy is set to 0, so I would expect to see a single interrupt when the transfer has completed and the linked structure is put on the return queue. Instead, after the first interrupt, I find the single entry on the return queue as expected; after each of the subsequent four unexpected interrupts, the return queue is empty.

I could understand getting two interrupts perhaps (one when something is pushed onto the queue and another, unwanted one, when it is taken off), but five? I have tried various experiments, and there are always exactly five interrupts (no more, no less) for every complete transfer.

Does anyone know what might be going on or how I can reduce these interrupts to one?

  • Peter Robertson,
    The CSL_intcxxx is for the C66x CorePac interrupt controller.
    The CSL_cpintcxxx is for the chip-level interrupt controller.
    You can find the example, how to enable queue pending signal from Queue Manager Subsystem (QMSS) in the Multicore Navigator and map them to core interrupt.

    The Resource Manager have a set of APIs for Interrupts, Hardware semaphores, etc.
    It provides example code for initializing and using the PA, QMSS and CPPI subsystems.

    The interrupt can be triggered by multiple conditions. After the CorePac services the interrupt, the CorePac
    needs to write a value into the EOI_VECTOR field in the End of Interrupt register to acknowledge that it has cleared the interrupt condition before a new interrupt can be generated.

    Find the more detailed information at: Chip Interrupt Controller (CIC) User Guide.
    http://processors.wiki.ti.com/index.php/Configuring_Interrupts_on_Keystone_Devices

  • Thank you for your response.

    I started my post by stating "I have a working driver...".

    "The CSL_intcxxx is for the C66x CorePac interrupt controller.
    The CSL_cpintcxxx is for the chip-level interrupt controller.
    "

    I do not use CSL.

    "You can find the example, how to enable queue pending signal..."

    How could I be getting the interrupts my post was about if I didn't know how to enable the signal?

    "The interrupt can be triggered by multiple conditions."

    This is clear. What is not clear is the list of those conditions. Where is this documented?

  • Hi,

    Apologize for this. Please mention driver code, if it is from MCSDK.
    If possible please provide a code snippet so that we can examine this issue.

  • The driver is my own.

    As I mentioned, I am catching the interrupts associated with queue manager event 93 (QM_INT_PASS_TXQ_PEND_14).  See Table 5-12 in SPRUG9E.

    My question is really simple. Where is the TI documentation that says explicitly what causes this event 93 to be signalled? I have been unable to find anything that discusses it. "QM_INT_PASS_TXQ_PEND_14" gives no useful information. I suspect that, once again, this is something that has not been documented because it is considered to be obvious by people who have been working with it for years.

    As I am seeing it signalled five times for every one Ethernet packet sent from the DSP, there must be five things happening that signal event 93. All I am interested in is that one event that shows that the transaction has completed by pushing the descriptor onto queue 654.

    As I am getting exactly 5 interrupts for each transaction, this cannot be a problem with acknowledging interrupts.

    To repeat, what causes event 93 to be signalled?

  • Hi,

    The documentation is less for queue pend and qm_int_pass_txq_pend_14.

    Queue Pend: This signals are the secondary events. First,this will be routed to the chip-level interrupt controller (CIC0) and then be mapped to the CorePac interrupt controller. Then the event will be mapped to the core interrupt from the CorePac interrupt controller.

    QM is used for all PKTDMAs that get their queue pend signals from physical Queue Manager.

    The mapping of channels to events is fixed. The mapping of queue number to channel number is fixed only for queues that drive queue_pend signals.

    You can see your old E2E post, it may help you.

    http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/245243/857758.aspx#857758

    http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/178446/644472.aspx#644472

  • I believe my question was very simple and quite clear.

    I did not ask about what happens once a queue pend singal has been generated, but this is the question you seem to have answered. The other posts to which you refer are also about what happens after the event has been generated and have nothing to do with my actual question.

    I asked "To repeat, what causes event 93 to be signalled?".

    Please do not answer, "a Queue Pend event".

    What is a queue pend event?

    Where is this documented?

    "The documentation is less for queue pend and qm_int_pass_txq_pend_14."

    Perhaps you meant to write "non-existent"?

  • I could not find detailed document for queue pend event. I can find this below information from Multicore Navigator User Guide.

  • I have already mentioned the association between queue 654 and event 93 a few posts ago.

    How would you suggest I go about getting the information I need as you seem to have verified that it's not something that TI considers worth documenting?

    Before getting too far from the actual problem, let me repeat that I am trying to find out why I get event 93 being signalled FIVE times for every output transaction.

  • Peter,

    A queue pend signal is an internal level sensitive signal connecting the queue manager output to the associated packet DMA. A TX queue pend (*_TXQ_PEND) signal is asserted if there are any packets currently pending in the corresponding TX queue i.e. the queue is not empty. As you said, this is probably one of those things that is considered obvious by those familiar with the peripheral and we will get this added to the documentation.

    As to your real question on why event 93 is triggered 5 times when you expect it to be triggered only once, I cannot comment unless I get more details on your queue manager setup. The best way to proceed forward is if you attach a pared down test that reproduces this issue so that we can reproduce it ourselves and take a deeper look.

  • Hi Peter,

    I apologize for the lack of documentation on queue pend. The queue pend signal is actually a bundle of signals that started out connecting the Queue Manager to each Tx channel of each Packet DMA (one queue pend per Tx channel). We then added several more queue pends to trigger interrupts directly once a specific queue has been pushed to.  The way it works is when the queue transitions from empty to non-empty, the signal goes high and drives the interrupt controller.  It will not trigger again until the next empty-to-non-empty transition.  As to why you're getting 5 interrupts per message is rather strange.  Perhaps if you tried an isolated, simple test where you push a single descriptor, field the interrupt and clear it, the problem will be exposed.  You can also try triggering interrupts by using the QM's interrupt distributor registers, and fielding/clearing the interrupts exactly as if they were handled by the Accumulator firmware.

      -dave

  • Thank you for answering my actual question.

    It is going to be difficult to generate a small example as I am not using any of theTI development tools (apart from the compiler/assembler/linker).

  • fyi, Dave (who just posted) is a lot more familiar with QM than I am. I will defer to his suggestions.

  • Peter, maybe this will help.  Here's some old code (for tci6616, Nyquist) that sets up an interrupt for a queue pend, and the ISR code that handles it.  For the ISR, it's normally best (as always) to minimize the time in the ISR, so it's better to set a flag (or semaphore, etc.) and do pushing/popping outside of the ISR.


     

    /* This function sets up a QMSS Queue Pend interrupt, given a que number.
    * It is written for Nyquist (TCI6616).
    *
    * For low-level debugging, the following is useful info:
    * 0x02600000 = base address of CPINTC registers (on Nyquist)
    * 0x01800000 = base address of GEM's IntC registers
    *
    */

    void setup_qpend_interrupt(Uint16 qnum)
    {
      Int16 chan;
      CSL_IntcParam             vectId1;
      Uint8 events[
    10] =
     
    {134, 135, 136, 137, 138, 139, 140, 141, 142, 175};
    //test only:
    //Uint32 *reg = (Uint32 *)0x02600200;
    //Uint32 mask;

     
    /* Step 1: Translate the queue number into the CPINTC input event. */

     
    /* qnum is expected to be 662..671 */
      chan
    = (qnum - FIRST_QPEND_QUEUE);
     
    if ((chan < 0) || (chan > 10))
     
    {
       
    printf("Invalid Queue Pend queue %d\n", qnum);
       
    return;
     
    }

      cp_event
    = events[chan];

     
    /* Step 2: Map the CPINTC input event to the GEM input event
       *         (which are 56 to 63). */

     
    CSL_CPINTC_disableAllHostInterrupt(cphnd);
     
    CSL_CPINTC_setNestingMode(cphnd, CPINTC_NO_NESTING);
     
    /* Map the input system event to a channel.  Note, the
       * mapping from channel to Host event is fixed. In our case,
       * channel 0 maps to host event 56, chan 1 to 57, etc. */

     
    CSL_CPINTC_mapSystemIntrToChannel(cphnd, cp_event, gem_event-56);

     
    /* Enable the system interrupt */
     
    CSL_CPINTC_enableSysInterrupt(cphnd, cp_event);
     
    /* Enable the channel (output). */
     
    CSL_CPINTC_enableHostInterrupt(cphnd, gem_event-56);
     
    /* Enable all host interrupts. */
     
    CSL_CPINTC_enableAllHostInterrupt(cphnd);

     
    /* Step 3: Hook an ISR to the GEM input event. */
      vectId1
    = CSL_INTC_VECTID_12; //4 through 15
      hIntcQmss[QPEND_IDX] = CSL_intcOpen(&intcQmss,
                                           gem_event,
    // selected event ID
                                          &vectId1,
                                           NULL);
     
    /* Hook the ISR */
      Record[QPEND_IDX]
    .handler = (CSL_IntcEventHandler)&QMSS_ISR_Qpend;
      Record[QPEND_IDX]
    .arg = (void *)gem_event;
     
    CSL_intcPlugEventHandler(hIntcQmss[QPEND_IDX],&(Record[QPEND_IDX]));

      
    /* Clear the Interrupt */
     
    CSL_intcHwControl(hIntcQmss[QPEND_IDX],CSL_INTC_CMD_EVTCLEAR,NULL);

     
    /* Enable the Event & the interrupt */
     
    CSL_intcHwControl(hIntcQmss[QPEND_IDX],CSL_INTC_CMD_EVTENABLE,NULL);

    /*TEST - KICK THE INTERRUPT MANUALLY
      mask = (cp_event / 32) * 32;
      mask = 1 << (cp_event - mask);
      reg  = (Uint32 *)0x02600200 + (cp_event / 32); //get the reg offset
    *reg = mask;
    */

    }

    /*************************************************************************
    *   QMSS ISR - for the Queue Pend queues.
    * *************************************************************************/

    void QMSS_ISR_Qpend(Uint32 eventId)
    {
      Uint32 desc;

    /* Requeue the test descriptor. Since the mapping from event number
      * back to queue number is not fixed (and we're cycling through all
      * of them), we'll use a global to know which queue to pop from. */

      desc
    = qm_pop_queue(qn_num);
     
    qm_push_queue(MONO_TX_COMPLETE_Q, desc);

     
    /* Push to SYNC_Q to signal the core that the ISR was processed. */
      desc
    = qm_pop_queue(SYNC_FDQ);
     
    qm_push_queue(SYNC_Q, desc);

     
    /* Clear the GEM Interrupt */
     
    CSL_intcHwControl(hIntcQmss[QPEND_IDX],CSL_INTC_CMD_EVTCLEAR,NULL);

     
    /* Clear the CPINTC0 Interrupt */
     
    CSL_CPINTC_clearSysInterrupt(cphnd, cp_event);

     
    /* Close the handle since we are remapping with each call. */
     
    CSL_intcClose(&intcQmss);
    }

  • Problem solved.

    As you say, ISRs should be as small as possible; I never do anything in any ISR that can be done outside. Sadly, this isn't always possible due to the apparently-unlimited explosion of complexity of the devices.

    The ISR you posted is popping the queue before clearing the CIC interrupt; I was leaving that pop outside the ISR.

    I have found that simply clearing the CIC interrupt as in your code prevented further interrupts from ever happening, so I disable, clear, then re-enable the interrupt. The re-enable was provinking a further interrupt as the queue was not empty.

    This is not what I expected having seen comments like 'It will not trigger again until the next empty-to-non-empty transition'. It has been mentioned that the non-empty status is a level being input to the CIC, so disabling and then re-enabling does provoke the interrupt. I now pop the queue in the ISR (the value popped is of no interest) and the multiple interrupts have disappeared.

    As a related point on the documentation...

    In SPRUG9E, 4.1.4 about the D register, I find 'The Queue N Register D is written to add a packet to the queue and read to pop a packet off a queue. The packet is pushed or popped to/from the queue only when the Queue Register D is  written'.

    I assume the final 'written' should be 'written/read'. The same error appears in 4.1.4.4.