C6678 SRIO Transfer

Hi, this sounds very similar to the problem in this thread

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/125626.aspx

That mainly deals with the fact that the configuration of the example wasn't meant to deal with large messages so the heap needs to be adjusted.

Please let me know if increasing your heap size helps the problem (assuming you haven't done so already)

Thanks

Tim

Hi,

Please, could you tell me which is the company and model of the Micro TCA backplane that are you using?

Thanks in advance.

Shmulik.

I am not using a backplane, I am using a breakout card.

I was merely observing that your experience followed a similar pattern: transfers worked at first, but didn't work at larger sizes.

Did you confirm that you heaps aren't the problem?  They must be large enough to hold all the buffers that are allocated (the default settings allocate many buffers).

Hi, Nonos

Please, could you tell me which is the company and model of the Micro TCA that are you using?

Thanks.

Tim,

I changed the heap as you told me and is working but I'd like to understand why

1- Why I need MAX_MEM_MGR_ENTRIES 9 or more and why 9?

2- Could you explain me how do you make the calculation heapMemParams1.size = (2048+512)*1024; I mean what is 2048, 512 and 1024.

3- Do you know a simple board read to connect two EVM6678L boards?

Thanks.

Shmulik

The number of entries corresponds to the number of times that one of the routines requests an entry via one of the appropriate mallocs on the Osal API (in the example I think only the "Osal_srio***" calls used that queue.  You definitely need as many entries as you have Rx + Tx buffers, but I honestly don't know why it needs to be more than that.

 (2048+512)*1024 was just the rest of MSMC. I believe the cfg file used 1 MB for the SharedRegion0 -- and I just used 2.5 MB because some of the other PDK libraries used some extra memory of the .5MB remaining.

I'm working on inter-board SRIO myself.. I'm not having lots of success getting anything besides the default configuration to work.. but there is an example in the PDK: ti/pdk.../packages/ti/transport/ipc/examples/srioIpcChipToChipExample

Hi Tim,

Thank you for your raply.
I have tried to increase heap size that is described in the thread that you mentioned.
But, the problem was not solved.
In my understanding, DrectIO doesn't use heap area for receiving.
Payload data of NWRITE is not buffered at driver layer ,because destination address is specified in RapidIO packet directly.

Is there any advice?

Hi Shmulik,

I'm using AMC starter kit of TCSJ in Japan.
This kit is having three AMC slot and the fatpipe(4-7/8-11) is connected between slot1 and slot3.
So, I can use both SRIO and PCI Express interface of EVM in this kit.

You can see the picture of this kit in the following URL(bottom of this page).
 http://www.tcsj.co.jp/backplane/backplane-detail/micro-tca/

Thanks

Hi,

I have replaced EVM(B) with other processor board, and checked whether the SRIO packet was actually sent or not.
As the result, this failure happens even if I use another processor board as the destination of SRIO transfer.
Therefor, EVM(A) probably does not send the packet exceeding 2kBytes.
Also, I found the following situation.

When the bytecount of LSU is 1MByte, transfer data in destination memory is as follows.

    [offset addres] : transfer data
    ===============================
    0x0000 - 0x07FF : OK
    0x0800 - 0x1FFF : No data
    0x2000 - 0x27FF : OK
    0x2800 - 0x3FFF : No data
        .
        .
        .
    0xE000 - 0xE7FF : OK
    0xE800 - 0xFFFF : No data

Is there any limitation of DirectIO transfer?
Do I need to configure something when I transfer the data that exceeds 2kBytes?

Thanks,

Hi nonos,

I tried to find info in english for the AMC starter kit of TCS but I didn't find.

Please, could you tell me where to find the data sheet or more specific information for it?

Thanks.

Shmulik

BLK9_EN should always be left cleared, as that block is not currently supported.  However, the BLK6-8_EN should not have to be enabled to support 1 port mode.  Obviously, all the SerDes have to be programmed correctly and enabled, but the port 1 - 3 blk enables shouldn't be needed.  Can you verify your PATH_MODE setting?

I'm going to verify your answer since you have it working now, but I'll continue to follow the thread.

Regards,

Travis

Hi Travis,

Thank you for your reply.
I'm setting the PATH_MODE field of Port0 to "4" by following code.

#define SUPPORTED_SRIO_PORTS  1
#define SRIO_PATHMODE_X4      4

for(i = 0; i < SUPPORTED_SRIO_PORTS; i++)
{
    /* Enable 5GBaud. */
    CSL_SRIO_Enable5GBaud (hSrio, i);

    CSL_SRIO_SetPLMPortPathControlMode (hSrio, i, SRIO_PATHMODE_X4);
}

If you need any other information, please let me know.

Thanks,

Mode 4 is single port 4x mode, so that should be right.  You could verify by reading the Path_mode field of RIO_PLM_SP(n)_PATH_CTL.  Additionally, if you are in single port mode you want to set the SERDES CFGTX MSYNC setting accordingly...

Transmit channels that have a lower numbered neighbor which has different rate or width
configuration must have cfgtxi[22] set high. Failing to do so will prevent the channel operating
correctly.  For example:
Scenario 1 - A single aggregated port, four lanes wide (4X), we would set lane 0 MSYNC = 1, lanes 1, 2, 3 MSYNC = 0
Scenario 2 - Two aggregated ports each two lanes (two 2X ports), we would set lane 0 and lane 2 MSYNC = 1, lanes 1 and 3 MSYNC = 0
Scenario 3 - four single ports (four 1X ports), we would set lanes 0,1,2,and 3

Again, BLK_EN6-8 should have no bearing on this, unless you are still in loopback somehow.  How did you disable loopback, and what are your Serdes settings?  Please look at this thread which talks about disabling loopback in the SerDes and the digital loopback.  It also, has Serdes settings. http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/114940.aspx

There shouldn't be any DirectIO limitation for over 2KB transfers.  Are you using NWRITE, NWRITE_R, or SWRITE?  For simplicity, use 8B aligned payloads and addresses and you should have no problem.  What completion code are you getting from the LSU after the transfer?  Is is working correctly for you in loopback mode?  Make sure you have that working correctly first, verify payload in source and destination memory space.

Regards,

Travis

Hi Travis,

The values of RIO_PLM_SP(n)_PATH_CTL and, SP(n)_CTL in our current setting are here.

 RIO_PLM_SP(0)_PATH_CTL : 0x00000404
 RIO_PLM_SP(1)_PATH_CTL : 0x00000404
 RIO_PLM_SP(2)_PATH_CTL : 0x00000404
 RIO_PLM_SP(3)_PATH_CTL : 0x00000404

 SP(0)_CTL : 0xD0600001
 SP(1)_CTL : 0x00800001
 SP(2)_CTL : 0x00800001
 SP(3)_CTL : 0x00800001

It seems port0 is working as 4x SRIO mode properly.

Regarding MSYNC setting, I did not care this setting.
Thank you for this information.

I would like to clarify about MSYNC bit.
In SRIO User Guide(sprugw1), MSYNC bit is not assigned in Figure 2-7.
But, it is described  as bit20 at Table 2-12.
Also, it is described as bit22 (Synchronization master) at example2-1 in section 2.3.1.4 as you described.
Which is correct information?


> How did you disable loopback, and what are your Serdes settings?

Currrent SERDES settings are as follows.

    CSL_SRIO_SetNormalMode(hSrio, 0);
    CSL_SRIO_SetNormalMode(hSrio, 1);
    CSL_SRIO_SetNormalMode(hSrio, 2);
    CSL_SRIO_SetNormalMode(hSrio, 3);

    CSL_BootCfgSetSRIOSERDESConfigPLL (0x241);

    CSL_BootCfgSetSRIOSERDESRxConfig (0, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (1, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (2, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (3, 0x00440495);

    CSL_BootCfgSetSRIOSERDESTxConfig (0, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (1, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (2, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (3, 0x00180795);


> Are you using NWRITE, NWRITE_R, or SWRITE?

I'm using NWRITE transaction.

> What completion code are you getting from the LSU after the transfer?

Completion code is "0".

> Is is working correctly for you in loopback mode? 

It was not working in loopback mode too.
And, after enabling the BLK6_EN~BLK9_EN, it was working properly.

Source and destination address is aligned as at least 256 byte.

Thanks,

nonos said:

The values of RIO_PLM_SP(n)_PATH_CTL and, SP(n)_CTL in our current setting are here.

 RIO_PLM_SP(0)_PATH_CTL : 0x00000404
 RIO_PLM_SP(1)_PATH_CTL : 0x00000404
 RIO_PLM_SP(2)_PATH_CTL : 0x00000404
 RIO_PLM_SP(3)_PATH_CTL : 0x00000404

 SP(0)_CTL : 0xD0600001
 SP(1)_CTL : 0x00800001
 SP(2)_CTL : 0x00800001
 SP(3)_CTL : 0x00800001

It seems port0 is working as 4x SRIO mode properly.

Great, I would agree, this all looks correct.

nonos said:

Regarding MSYNC setting, I did not care this setting.
Thank you for this information.

I would like to clarify about MSYNC bit.
In SRIO User Guide(sprugw1), MSYNC bit is not assigned in Figure 2-7.
But, it is described  as bit20 at Table 2-12.
Also, it is described as bit22 (Synchronization master) at example2-1 in section 2.3.1.4 as you described.
Which is correct information?

The MSYNC bit affects timing by aligning the individual lane transmit clocks.  It should be set as I noted to maintain compliance to the spec.  The next release of the sprugw1 will contain more info on the MSYNC description, as well as fix the Figure 2-7 and example in section 2.3.1.4.  MSYNC is bit 20 of the CSL_BootCfgSetSRIOSERDESTxConfig.

nonos said:

> How did you disable loopback, and what are your Serdes settings?

Currrent SERDES settings are as follows.

    CSL_SRIO_SetNormalMode(hSrio, 0);
    CSL_SRIO_SetNormalMode(hSrio, 1);
    CSL_SRIO_SetNormalMode(hSrio, 2);
    CSL_SRIO_SetNormalMode(hSrio, 3);

    CSL_BootCfgSetSRIOSERDESConfigPLL (0x241);

    CSL_BootCfgSetSRIOSERDESRxConfig (0, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (1, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (2, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (3, 0x00440495);

    CSL_BootCfgSetSRIOSERDESTxConfig (0, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (1, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (2, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (3, 0x00180795);

Looks good.

nonos said:

> Are you using NWRITE, NWRITE_R, or SWRITE?

I'm using NWRITE transaction.

> What completion code are you getting from the LSU after the transfer?

Completion code is "0".

That's good.  Are you verifying payload has arrived in memory at the destination device, or are you just looking at the completion code of the LSU?  For Nwrite, the completion code is set as soon as the packet is passed to the physical layer for transmission, so it doesn't mean that it actually landed in the destination correctly.  Just want to make sure you are checking.

nonos said:

> Is is working correctly for you in loopback mode? 

It was not working in loopback mode too.
And, after enabling the BLK6_EN~BLK9_EN, it was working properly.

Source and destination address is aligned as at least 256 byte.

I just don't understand the BLK_EN6-9 need.  I'll have to do some checking from here.