This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

[C66x] How to check SRIO error status in non-blocking socket and a few curious things.

Hi,

I`m new about SRIO IP. And I`m trying to build our wrapper to cover TI SRIO API in user side. 

My environment is 

 - 66k2H12 evm

 - CCS 6.2.0

 - pdk 4.0.2


I have a few question about LLD and SRIO operation. 

1. Is it possible to check error status in non-blocking socket?
 I wrote the read function such like this:

Srio_DrvBuffer srio_buf;
S32	startTime;
U8 compCode;

if(gBSP_SRIO_Initialized != 1)
        return CMN_RESULT_ERR_NOTINIT;

/* Program the DIO Destination Information. */
gBSP_SRIO_Addrinfo_Write.dio.rapidIOLSB = (U32)remote_addr;
if(data_len > BSP_SRIO_MAX_MTU)
{
	data_len = BSP_SRIO_MAX_MTU;
}

/* Writeback the contents of the cache. */
CACHE_wbL1d (pBuff, data_len, CACHE_FENCE_WAIT);

srio_buf = (Srio_DrvBuffer)l2_global_address((U32)pBuff);
System_printf("Debug(Core %d): Read DIO Data Transfer - Src 0x%p Dst 0x%p\n", gCorenum, srio_buf, remote_addr);
if(Srio_sockSend_DIO(gBSP_SRIO_Socket_Data, srio_buf, data_len, &gBSP_SRIO_Addrinfo_Read) < 0)
{
	System_printf ("Error: Unable to send payload over DIO socket\n");
	return -1;
}

/* Read the completion code filled by the ISR */
compCode = 0xFF;
if (Srio_getSockOpt(gBSP_SRIO_Socket_Data, Srio_Opt_DIO_READ_SOCK_COMP_CODE, &compCode, sizeof(U8)) < 0)
{
	System_printf ("Error: Unable to read the completion code in socket\n");
	return -1;
}

if(compCode != 0x0)
{
	System_printf ("Error: read the completion code in socket is not complete\n");
	srioDioLsuBadTransfers++;
	return -1;
}

But when I use this function, the lsu_icsr(interrupt condition status register) register shows the error in srcID0. But when I check the completionCode, It displays with no error. In this case, completion check process is useless? how can I check the error status in non-blocking socket?

2. When is the exact timing to occur LSU interrupt in read case(the case of sending NREAD send)?

- In description of Rx operation in data sheet, the interrupt is occured until the IP is ready to receive next packet. In official SRIO specification, Read operation is completed when the slave sends the "response" message with payload. So Is the LSU interrupt is occurred in NREAD packet send timing? or Response packet receiving timing?

3. How to register doorbell interrupt during device initialization?

-  My purpose is to capture dio interrupt and doorbell interrupt. So In my case, if I execute write operation, when the write operation is finished, then dio interrupt is occured. And after finishing operation, it occurs doorbell interrupt by doorbell auro-generation.  I use cpIntc and eventCombiner to register interrupt handler in BIOS side. So My process is such like this:

 1) CSL_SRIO_SetDoorbellRoute(hSrio, 0); // to capture doorbell interrupt in corepac0(INTDST16)

 2)

for (i = 0; i < 16; i++)
{
CSL_SRIO_RouteDoorbellInterrupts(hSrio, 0, i, 0);
CSL_SRIO_RouteDoorbellInterrupts(hSrio, 1, i, 0);
CSL_SRIO_RouteDoorbellInterrupts(hSrio, 2, i, 0);
CSL_SRIO_RouteDoorbellInterrupts(hSrio, 3, i, 0);
} // to capture the doorbell register of whole range in corepac0(INTDST16)

 3)/* Disable Interrupt Pacing for INTDST0 */
CSL_SRIO_DisableInterruptPacing (hSrioCSL, 0);

4) /* Route LSU0 ICR0 to INTDST0 */
CSL_SRIO_RouteLSUInterrupts (hSrioCSL, 0, 0);

5) /* Route LSU0 ICR1 to INTDST0 */
CSL_SRIO_RouteLSUInterrupts (hSrioCSL, 1, 0);

6) /* Route LSU0 ICR2 to INTDST0 */
CSL_SRIO_RouteLSUInterrupts (hSrioCSL, 2, 0);

7) /* Map the System Interrupt i.e. the Interrupt Destination 0 interrupt to the DIO ISR Handler. */
CpIntc_dispatchPlug(CSL_CIC0_SRIO_INTDST0, BSP_SRIO_Dio_Isr, (UArg)hDrvManagedSrioDrv, TRUE);

8) /* SRIO DIO: Configuration is for CPINTC0. We map system interrupt 112 to Host Interrupt 8. */
CpIntc_mapSysIntToHostInt(0, CSL_CIC0_SRIO_INTDST0, 8);

9) /* SRIO DIO: Enable the Host Interrupt. */
CpIntc_enableHostInt(0, 8);

10) /* Enable the System Interrupt */
CpIntc_enableSysInt(0, CSL_CIC0_SRIO_INTDST0);

11) /* SRIO DIO: Get the event id associated with the host interrupt & Plug the CPINTC Dispatcher. */
eventId = CpIntc_getEventId(8);
EventCombiner_dispatchPlug (eventId, CpIntc_dispatch, 8, TRUE);

12) /* Plug the DoorBell Dispatcher. */
EventCombiner_dispatchPlug (20, (CpIntc_FuncPtr)DoorBellIsr, 8, TRUE);
EventCombiner_enableEvent(20);

 Is it right procedure to register dio/doorbell interrupt? Cause, when I test it on internal loopback example(dioLoopbackExample), doorbell interrupt is not occurred during (read/write)operation. But when i forcely send doorbell packet, Interrupt is occurred. How can I configure the doorbell interrupt?

Sorry for awful question because of my english writing.

Thanks in advance.

Best regards.

Chanseok

  • Hi Chanseok,

    I've notified the SRIO experts. Feedback will be posted here.

    Best Regards,
    Yordan
  • Thanks for reply, I`ll wait for it.
  • I just waited it for a week. but nobody answered..
  • Hi,

    I've sent a reminder.

    Best Regards,
    Yordan
  • Hi Chanseok,

    >>1. Is it possible to check error status in non-blocking socket?
    Please refer to the function dioSocketsWithISR() in pdk_k2hk_4_0_2\packages\ti\drv\srio\example\SRIOLoopbackDioIsr\src\loopbackDioIsr.c, you can check the completion code after interrupt has occurred. Immediately checking the completion code followed by Srio_sockSend_DIO() for non-blocking packets may result in invalid completion code.

                 /* Wait for the interrupt to occur without touching the peripheral. */
                /* Other useful work could be done here such as by invoking a scheduler */
                startTime = TSCL;
                while((! srioLsuIsrServiced) && ((TSCL - startTime) < SRIO_DIO_LSU_ISR_TIMEOUT));

                if (! srioLsuIsrServiced) {
                  System_printf ("ISR didn't happen within set time - %d cycles. Example failed !!!\n", SRIO_DIO_LSU_ISR_TIMEOUT);
                  return -1;
                }

                 /* Read the completion code filled by the ISR */
                compCode = 0xFF;
                if (Srio_getSockOpt(srioSocket[sockIdx], Srio_Opt_DIO_READ_SOCK_COMP_CODE, &compCode, sizeof(uint8_t)) < 0)
                ...

    >>2. When is the exact timing to occur LSU interrupt in read case(the case of sending NREAD send)?
    Would you clarify which data sheet describeds "the interrupt is occured until the IP is ready to receive next packet"? Per SPRUGW1B, Serial Rapid IO (SRIO) User Guide, 2.3.9.1.1 - Direct I/O (Doorbell) Servicing Interrupts, "Interrupts are generated only after all VBUSM responses are received.", also from training.ti.com/.../keystone-srio-slides.pdf, Rx Operation, "If packet is doorbell, then complete all outstanding transactions and post interrupt." These indicate LSU interrupt is occurred after Response packet receiving for the read operation. Note "The LSU completion interrupt is given on the transmit side of things once a TX transfer has completed.  The Doorbell interrupt is on the RX side when a Doorbell packet has been received from the sender to notify a packet reception.", see e2e.ti.com/.../1087975

    >>3. How to register doorbell interrupt during device initialization?
    Will try to find an example code for configuring Doorbell interrupt. Make sure you have doorbellValid bit set.

    Regards,
    Garrett

  • Dear Garrett,

    Thanks for the reply. 

    1. yes, I already check the completion code after ISR, as you can see it in my code. there is no completion code(still 0x00) when the error interrupt is occurred.

    2. Thanks for explanation. Actually, I tried to find out how to check Rx operation in DSP side(I already have questioned about it. see the details in https://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/549207)  When the FPGA sends the packet to DSP, DSP couldn`t catch any notification(doorbell interrupt, error status etc..) Of course, when I check it through DSP internal loopback, it works fine. Any suggestion?

    3. I already figured it out. I didn`t realize the relationship between cpintc and eventcombiner. but the problem is solved.

    May I have a question about a few things? 

    1. I found the situation about sp_src_op configuration. In the example, srioDevice_init() configures that register. here is my code.

        /* Configure the source operation CAR */
        memset ((void *) &opCar, 0, sizeof (opCar));
        opCar.portWriteOperationSupport = 1;
        opCar.atomicClearSupport        = 1;
        opCar.atomicSetSupport          = 1;
        opCar.atomicDecSupport          = 1;
        opCar.atomicIncSupport          = 1;
        opCar.atomicTestSwapSupport     = 1;
        opCar.doorbellSupport           = 1;
        opCar.dataMessageSupport        = 1;
        opCar.writeResponseSupport      = 1;
        opCar.streamWriteSupport        = 1;
        opCar.writeSupport              = 0;
        opCar.readSupport               = 0;
        opCar.dataStreamingSupport      = 1;
        CSL_SRIO_SetSourceOperationCAR (hSrio, &opCar);

    Before executing this section, I confirmed that src_op = 0x4FDF4, but after passing this section, src_op = 0x4. I saw the SRIO LLD code, and realized that the code structure is same as SetDestOperationCAR(). But effect is different. Is there any precondition to set that register? One more thing to mention that, that register is described Read-Only in datasheet, But it can configure it. Is this wrong?

    2. When the FPGA sends the packet to DSP, is this the right situation to increase Inbound_AckID, described in sp_ACKID_STAT? I understand that if this value is increased, then Rx packet is received. It will be helpful to clarity this register`s rule.

    Thanks for helpful answer.

    Regards,

    Chanseok.

  • Hi Chanseok,

    1. src_op register is writable and can be confirmed by 2.3.15.3 Peripheral Initialization. It appears The CSL_SRIO_SetSourceOperationCAR function touches reserved bit- 3 of SRC_OP register that causes src_op = 0x4. If you comment out SRIO_RIO_SRC_OP_A_SWAP line in the function, see pdk_k2hk_4_0_2/packages/ti/csl/csl_srioAuxPhyLayer.h, the src op value should be correctly written into the register.

    static inline void CSL_SRIO_SetSourceOperationCAR
    (
        CSL_SrioHandle  hSrio,
        SRIO_OP_CAR*    ptrSrcOp
    )
    {
        Uint32  value = 0;

        /* Populate the fields into the Source OP */
        CSL_FINS (value, SRIO_RIO_SRC_OP_G_READ,            ptrSrcOp->gsmRead);
        ...
    //    CSL_FINS (value, SRIO_RIO_SRC_OP_A_SWAP,            ptrSrcOp->atomicSwapSupport);
        CSL_FINS (value, SRIO_RIO_SRC_OP_PORT_WR,           ptrSrcOp->portWriteOperationSupport);
        CSL_FINS (value, SRIO_RIO_SRC_OP_IMPLEMENT_DEF2,    ptrSrcOp->implnDefined2);

        hSrio->RIO_SRC_OP = value;
    }

    2. Inbound_ackID indicates the expected value of the ackID field in the next packet to be received, as described in the SPn_ACKID_STAT register. The attached document "Keystone Software Assisted Error recovery_addendum" should help understand the ACKID usage.


    Regards, GarrettKeystone Software Assisted Error recovery_addendum.pdf

  • Thanks for response.

    One more thing to be confirmed, is there any new datasheet about keystone2 SRIO? I found that current-released version of SRIO(sprugw1) is based on keystone1 device, not keystone2. So few things(i.e. register position, SerDes Configuration etc) are not matched at all. I spent most of time to find it :(

    Anyway, Thanks again.

    Regards,
    Chanseok.

  • Hi Chanseok,

    As of now, we have a common SRIO user guide for Keystone I and II however we have new SerDes user guide for keystone II devices.

    Please refer SRIO interface section for SerDes configuration detail.

    Thank you.