66AK2H12 EVM SRIO port ok timeout and enabling internal loopback.

Jv,

Can you explain where did you get the SRIO driver and steps to build into the Linux kernel? When kernel boot-up, it looks through all 4 SRIO ports to discover any SRIO connection, the timeout seems to be 90 seconds. You need to have EVM's SRIO interface connected with each other (use a breakout card) and boot EVM cards roughly the same time to avoid this.

Regards, Eric

Hi Eric!

I'm using this example: http://processors.wiki.ti.com/index.php/MCSDK_HPC_3.x_MPI_over_SRIO

and kernel/driver from here: http://git.ti.com/keystone-linux/linux/commits/v3.8/rio-dev-dio

Where can I order/get breakout card, suitable for connecting two EVM's for debugging SRIO interface? Here http://processors.wiki.ti.com/index.php/MCSDK_HPC_3.x_Getting_Started_Guide the http://www.mistralsolutions.com/pes-products/development-platforms/rtm-break-out-card.html link is down.

Furthermore, I would like to ensure question about internal loopback in SRIO interface, especially in ARM side. Is it possible to enable SRIO interface in loopback mode without bus physically connected anything, just to debug/verify SRIO ports? Something like this: SRIO_programming_perfomance.pdf(5700.SRIO_Programming_Performance.pdf) in chapter 2.6 Loopback. I know this is for DSP, but should it work in ARM also, and without physical connection???

Thanks in advance! Like I said in earlier post that I tried to enable internal loopback, but poor results. So I would appreciate if you/TI could verify/guide how to achieve my goal, debug SRIO with or without breakout card.

br

jv

Internal SerDes loopback is available regardless of whether the ARM or DSP is driving the SRIO peripheral.  If you haven't already installed the PDK from the MCSDK, please do so:

http://software-dl.ti.com/sdoemb/sdoemb_public_sw/mcsdk/latest/index_FDS.html

Then take a look at the configuration inside the following file for enabling loopback.  It is used in all the PDK examples.

C:\ti\pdk_keystoneX_X_XX_XX_XX\packages\ti\drv\srio\device\k2h\src\device_srio_loopback.c

If interested in a SRIO breakout card, you can use this one:

https://estore.ti.com/CI2EVMBOC--P2685.aspx

Regards,

Travis

Thank you for answer Travis!

I have seen PDK example and I think that's not helping me because it is for ARM/DSP communication. My goal is to use SRIO between ARM and FPGA and kernel/driver from here: http://git.ti.com/keystone-linux/linux/commits/v3.8/rio-dev-dio.   

In my previous post you can see what I have tried. So I need to verify/debug keystone_rio.c driver functionality. Because I do not have a breakout card(or anything else connected to srio) and could not resolve timeout/port_ok errors, I tried to enable internal loopback in driver.

What is the case with internal loopback, does it need physical connection to get port working? If not, enabling internal loopback mode should resolve timeout error what I'm facing?

br,

jv

Juha Vuorivirta said:

What is the case with internal loopback, does it need physical connection to get port working?

No, it does not.  If you are doing internal loopback, nothing has to be connected to the pins.

Juha Vuorivirta said:

If not, enabling internal loopback mode should resolve timeout error what I'm facing?

I'm not real familiar with that example you are using, but from the timeout message, the port isn't getting port_ok.  In internal loopback this shouldn't be an issue.  My suggestion was to use the internal loopback configuration from the file I pointed you to, in order to make sure it was correctly configured.  Maybe even a simpler test, just to make sure the loopback is working is to use one of the DSP only examples in loopback and make sure it is working before going to the ARM/DSP example.  You could use the simplest one:

C:\ti\pdk_keystoneX_X_XX_XX_XX\packages\ti\drv\srio\example\SRIOLoopbackDioIsr

If this works fine, we know the SRIO configuration part is good, and you should be able to reuse it  for the ARM kernel/driver example next.  Eric should be able to help out a little more on this example.

Regards,

Travis

Juha,

The openmpi over SRIO doesn't support the internal loopback test, you need two EVM cards with SRIO connected via breakout card. The pdk_keystoneX_X_XX_XX_XX\packages\ti\drv\srio\example\SRIOLoopbackDioIsr example suggested is a good one for verify the SRIO configuration, it can be run on DSP cores.

Regards, Eric

Ok, thanks Eric!

I understand that kernel driver does not support SRIO loopback mode in its current state, but enabling mode could resolve timeout error and debugging ports should be possible? I need this because debugging/verifying driver functionality and in future we will implement communication between ARM(using keystone_rio.c driver) and FPGA via SRIO in custom PCB.

I executed SRIO loopback example on DSP and it runs fine, so we know the SRIO configuration part is good :)

On ARM, to reuse example is another case. I looked over CSS debugger registers during execution. At least two things came up different than in kernel keystone_rio.c driver.

1) enable loopback in PER_SET_CTL register: 0xF0 needs to be written for enabling all lines in loopback mode.

2) SerDes lane enable: kernel driver does not write to registers 0x232C200-800 like DSP example does, in the end of function.

I can not find what are actually these registers (0x232C200-800)? Debugger shows value 0x40000002 written all four registers, enabling what? I use sprugw1b - SRIO.pdf as a reference.

I tried to enable 1 & 2 to the driver like this: 

1) set per ctl registers like in DSP

/* Set control register 1 configuration */
 __raw_writel(0x000000F0, &krio_priv->regs->per_set_cntl1);

 /* Set control register (disable promiscuous) */
__raw_writel(0x00053800, &krio_priv->regs->per_set_cntl);

2) end of the function k2_rio_serdes_lane_enable()

val = 0x40000002;

__raw_writel(val, regs + 0x200 * (lane + 1));

Still same, port initialization error. There might be lots of other things that could go wrong, but on the other hand driver and DSP example SRIO initializations looks very similar. Is there any known solution, more detailed documentation or are 1 & 2 incorrect? 

Thanks in advance,

PS. I am out of office for next two weeks, but we will continue this after vacation. 

br,

jv 

Hello Eric and Travis!

Finally after debugging DSP example and finding right configuration(above and some changes to the .dts file), I got ports enabled.

[   15.453985] keystone-rapidio 2900000.rapidio: KeyStone RapidIO driver v1.2
[   15.459612] keystone-rapidio 2900000.rapidio: initializing 5.00 Gbps interface with port configuration 0
[   15.467373] keystone-rapidio 2900000.rapidio: setting lane 0 SerDes to 5GHz
[   15.473071] keystone-rapidio 2900000.rapidio: setting lane 1 SerDes to 5GHz
[   15.478764] keystone-rapidio 2900000.rapidio: setting lane 2 SerDes to 5GHz
[   15.484461] keystone-rapidio 2900000.rapidio: setting lane 3 SerDes to 5GHz
[   15.490488] keystone-rapidio 2900000.rapidio: enabling packet forwarding to port 0 for DestID 0xffff - 0xffff
[   15.498600] keystone-rapidio 2900000.rapidio: enabling packet forwarding to port 0 for DestID 0xffff - 0xffff
[   15.506714] keystone-rapidio 2900000.rapidio: enabling packet forwarding to port 0 for DestID 0xffff - 0xffff
[   15.514825] keystone-rapidio 2900000.rapidio: enabling packet forwarding to port 0 for DestID 0xffff - 0xffff
[   15.522936] keystone-rapidio 2900000.rapidio: enabling packet forwarding to port 0 for DestID 0xffff - 0xffff
[   15.531047] keystone-rapidio 2900000.rapidio: enabling packet forwarding to port 0 for DestID 0xffff - 0xffff
[   15.539158] keystone-rapidio 2900000.rapidio: enabling packet forwarding to port 0 for DestID 0xffff - 0xffff
[   15.547269] keystone-rapidio 2900000.rapidio: enabling packet forwarding to port 0 for DestID 0xffff - 0xffff
[   15.555395] keystone-rapidio 2900000.rapidio: initialization of port 0
[   15.560810] keystone-rapidio 2900000.rapidio: detected remote port 0 on port 0
[   15.566827] keystone-rapidio 2900000.rapidio: port RIO0 host_deviceid -1 registered
[   15.573096] keystone-rapidio 2900000.rapidio: initialization of port 1
[   15.578805] keystone-rapidio 2900000.rapidio: detected remote port 1 on port 1
[   15.584811] keystone-rapidio 2900000.rapidio: port RIO1 host_deviceid -1 registered
[   15.591078] keystone-rapidio 2900000.rapidio: initialization of port 2
[   15.596505] keystone-rapidio 2900000.rapidio: detected remote port 2 on port 2
[   15.602502] keystone-rapidio 2900000.rapidio: port RIO2 host_deviceid -1 registered
[   15.608767] keystone-rapidio 2900000.rapidio: initialization of port 3
[   15.614498] keystone-rapidio 2900000.rapidio: detected remote port 3 on port 3
[   15.620507] keystone-rapidio 2900000.rapidio: port RIO3 host_deviceid -1 registered
[   15.626972] RIO rio-mport-cdev: Added RIO0 mport cdev(251:0)
[   15.631694] RIO rio-mport-cdev: Added RIO1 mport cdev(251:1)
[   15.636433] RIO rio-mport-cdev: Added RIO2 mport cdev(251:2)
[   15.641152] RIO rio-mport-cdev: Added RIO3 mport cdev(251:3)

Now I  start implement loopback testing, similar as DSP.

Thank you both for your effort and I'll be back when we start working with actual HW :)

br,