About srio ackid in C6678 using Keystone_srio program

David Yang1

Hi, BrandyJ:

I am working on C6678 SRIO, using an example program Keystone_srio from the forum. I have successfully linked 2 dsp via srio, but I got stuck while trying to apply the program to connect a C6678 to a V6 fpga. The program got stuck in the function Keystone_SRIO_match_ACK_ID , I qoute:

void Keystone_SRIO_match_ACK_ID(Uint32 uiLocalPort,
Uint32 uiDestID, Uint32 uiRemotePort)
{
Uint32 uiMaintenanceValue, uiResult;
Uint32 uiLocal_In_ACK_ID, uiRemote_In_ACK_ID, uiRemote_out_ACK_ID;

//send a "restart-from-error" commond, request the ACK_ID of the other side
srioRegs->RIO_SP[uiLocalPort].RIO_SP_LM_REQ=4;

//wait for link response
while(0==(srioRegs->RIO_SP[uiLocalPort].RIO_SP_LM_RESP>>
CSL_SRIO_RIO_SP_LM_RESP_RESP_VALID_SHIFT))
asm(" nop 5");

uiRemote_In_ACK_ID= (srioRegs->RIO_SP[uiLocalPort].RIO_SP_LM_RESP&
CSL_SRIO_RIO_SP_LM_RESP_ACK_ID_STAT_MASK)>>
CSL_SRIO_RIO_SP_LM_RESP_ACK_ID_STAT_SHIFT;

//Set the local OUTBOUND_ACKID to be same as the responsed ACKID
srioRegs->RIO_SP[uiLocalPort].RIO_SP_ACKID_STAT= uiRemote_In_ACK_ID;
if(uiRemote_In_ACK_ID)
printf("match_ACK_ID SP_ACKID_STAT=0x%x\n",srioRegs->RIO_SP[uiLocalPort].RIO_SP_ACKID_STAT); //for dubug

do
{
//set the remote OUTBOUND_ACKID to be same as local INBOUND_ACKID
uiLocal_In_ACK_ID= (srioRegs->RIO_SP[uiLocalPort].RIO_SP_ACKID_STAT&
CSL_SRIO_RIO_SP_ACKID_STAT_INB_ACKID_MASK)>>
CSL_SRIO_RIO_SP_ACKID_STAT_INB_ACKID_SHIFT;

uiMaintenanceValue= ((uiRemote_In_ACK_ID+1)<<
CSL_SRIO_RIO_SP_ACKID_STAT_INB_ACKID_SHIFT)|uiLocal_In_ACK_ID;

//set the remote ACK_ID through maintenance packet
uiResult= Keystone_SRIO_Maintenance(uiLocalPort, uiLocalPort, uiDestID,
0x148+(0x20*uiRemotePort), GLOBAL_ADDR(&uiMaintenanceValue),
SRIO_PKT_TYPE_MTN_WRITE);

if(uiResult) //fail
continue;

//readback the remote ID
uiResult= Keystone_SRIO_Maintenance(uiLocalPort, uiLocalPort,
uiDestID, 0x148+(0x20*uiRemotePort), GLOBAL_ADDR(&uiMaintenanceValue),
SRIO_PKT_TYPE_MTN_READ);
uiRemote_out_ACK_ID= uiMaintenanceValue&
CSL_SRIO_RIO_SP_ACKID_STAT_OUTB_ACKID_MASK;
}while(uiResult|(uiLocal_In_ACK_ID+1 != uiRemote_out_ACK_ID));
}

It seemed that the ackid on the fpga side couldn't be set like this. I am quite confuse with how the local inbound, outbound and outstanding ack id should be set according to the RIO_SP[uiLocalPort].RIO_SP_LM_RESP, and how should the remote inbound id be set? Is the remote inbound = remote inbound +1?(it seems so according to the functoin)

And if I am gonna set the AckID on the Fpga side, how shall I change the program?

Thanks!

David Yang

over 12 years ago

0 Brandy Jabkiewicz over 12 years ago

Mastermind 6325 points

Hello Yang,

My guess, based on my experience with my FPGA, is that it is not set up to receive maintenance packets. Are you using a prefabricated FPGA core, say from Xilinx or something? When we tested with the FPGA, the FPGA engineer had a tool and could see what was being written into address space. But we were using the Xilinx core and it did not have maintenance packets automatically.

I can double check with my fpga team member to see what he says too.

If the code is working for DSP to DSP then I would say the example is correct and the FPGA is setup corrently.

Yes, I beleive the remote inbound = remote inbound + 1 becuase you need to account for the message being sent right then.

@TI - is there someone there who could comment on the DSP to FPGA link?

Thanks,

Brandy

0 David Yang1 over 12 years ago in reply to Brandy Jabkiewicz

Intellectual 435 points

Hi,Brandy.

You guys answer much more quickly then those on Chinese deyisupport forum:). We are using siro v5.6 core from Xilinx on our FPGA. If the program is correct, since I've succeeded in 2 DSPs, I belive if I am gonna apply it to FPGA, the parameters may have to change a bit. Say the command to send a maintenace request, I quote:

uiMaintenanceValue= ((uiRemote_In_ACK_ID+1)<< CSL_SRIO_RIO_SP_ACKID_STAT_INB_ACKID_SHIFT)|uiLocal_In_ACK_ID;

//set the remote ACK_ID through maintenance packet

uiResult= Keystone_SRIO_Maintenance(uiLocalPort, uiLocalPort, uiDestID, 0x148+(0x20*uiRemotePort), GLOBAL_ADDR(&uiMaintenanceValue), SRIO_PKT_TYPE_MTN_WRITE);

Is the uiMaintenanceValue equals to the ACKID_STAT(which consists inbound&outbound& outstanding ackid )I am gonna set on the other side? And if the other Srio devcie is an FPGA, how should I decide the offset ? 0x148+(0x20*uiRemotePort is the off set of SP_ACKID_STAT regiser on dsp(If i am right about that), when i come to Fpga, how shall I decide the value?

Thanks,

David Yang

0 Brandy Jabkiewicz over 12 years ago in reply to David Yang1

Mastermind 6325 points

Hi Yang,

Let me start in the middle:

David Yang1 said:
And if the other Srio devcie is an FPGA, how should I decide the offset ? 0x148+(0x20*uiRemotePort is the off set of SP_ACKID_STAT regiser on dsp(If i am right about that),

The offset is based on a standard set of registers from the global SRIO protocol (see specification on www.rapidio.org). All RIO devices are supposed to have the ACKID_STAT register at a certain address in memory. Therefore if the device is coded to the global protocol, then register will be available to be written. The uiRemotePort is is used to dictate which group of registers to access. Each port has its own set of regsiters starting at 0x140 I beleive. You can see this in SPRUGW1A: TI calls them the CSR/CAR registers and gives them a memory offset of 0xB000 but the global standard dictates where they are in the lower three nibbles.

In summary on this point, every certified SRIO device will have specific registers at value 0x000 to 0x1000, I beleive.

David Yang1 said:
when i come to Fpga, how shall I decide the value?

Now here is the tricky part, as I remember, the FPGA is not a device that meets the global protocol. You will need to look at the core and see if it has an ACKID_STAT register. If the answer is yes, then you need to look at the core documentation and see if it supports maintenaince reads and writes. If the answer is yes to both of these questions, then the code should work as is. Otherwise you need to do some work on the FPGA side.

Summary:

1. Yes the core has ACKID_STAT, yes the core supports Maintenance - no changes

2. Yes the core has ACKID_STAT, no the core does not support Maintenance - you could potentially use a DIO command to write the memory directly instead of a maintenance packet.

3. No, the core does not have ACKID_STAT - you will either always have to restart the DSP and FPGA so that the ACKIDs are aligned by default or you will have to add to the SRIO core so that you can tell it what the ACKID should be. Meaning you will have to create a register/memory location that the DSP writes using a DIO command and that the FPGA uses to acquire its ACKID.

David Yang1 said:
Is the uiMaintenanceValue equals to the ACKID_STAT(which consists inbound&outbound& outstanding ackid )I am gonna set on the other side?

In case 1 and 2 above, then yes, the uiMaintenanceValue should work as is to write to the ACKID_STAT. If the register does not exist and you have to code it yourself, then you can use whatever format you choose.

It is my best guess that you are either in case 2 or most likely case 3. Since FPGA code is so tied to hardware and number of transistors (ie. size) etc, I suspect that Xilinx did not add support for maintenance packets. Again, if I see my FPGA guy today, I will try to ask him. I just the "DSP girl" :)

Brandy

0 xin jin over 12 years ago in reply to Brandy Jabkiewicz

Prodigy 40 points

I meet the same question.

I work on the FPGA side with Yang,what should i do?

thanks!

0 David Yang1 over 12 years ago in reply to Brandy Jabkiewicz

Intellectual 435 points

Hi,Brandy,

Thank you again for your earnest responses and inspiring answers! When I change the offset to a possible address to 0x15c,which is the Local Confguration Space Base Address on the FPGA side, it seems I can calibrate the ACKid on both side, and the Keystone_SRIO_match_ACK_ID now is running OK. But when I try to send data using swrite mode, Fpga side gets noting. The SOF(start of frame)&EOF on the FPGA remain still. And I am sure the device id is right,and the outbound Ackid on the DSP side is addingIs correctly. Even when i add doorbell into my packet, the inbound ACKid adds correctly, too. But still no data. Is it the same way to send packet to DSP and to a FPGA? Do I need to care about 8/10 coding or something like Sop or Eop(end of packet)? Or I just need to do as the same like I did with 2DSPs, mind device id, source&detination address,etc, and let my co-works forcus on FPGA problems.

Gratitude to the "DSP girl":)

David Yang

0 David Yang1 over 12 years ago in reply to Brandy Jabkiewicz

Intellectual 435 points

Hi,Brandy,

I might be wrong about the Local Confguration Space Base Address. I find the same CAR/CSR register on the DSP side on sprugw1a, it's a Readonly register and it totally has different meaning compared to the ACKID_STAT. The appearance that both side AckID are aligned might just be false.I am more confused now.

0 David Yang1 over 12 years ago in reply to Brandy Jabkiewicz

Intellectual 435 points

Hi,Brandy,

Thanks for your help. I verified the offesef on Fpga side, and Fpga has successfully received my packet now!

David Yang

0 Brandy Jabkiewicz over 12 years ago in reply to David Yang1

Mastermind 6325 points

Hi Yang,

So glad you are up and running.

If you have misalignment problems and the DSP reports errors in the RIO_SP_ERR_STAT register, then TI has a special "magic number" to write to clear the errors. This routine might be helpful for you to have and call occasionally to test. Perhaps there is some more information in this post too: http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/170264.aspx

int32_t srioReadErrors(void)

{

CSL_SrioHandle hSrio;

hSrio = CSL_SRIO_Open(0);

LOG_TRACE("SP[0]_ERR_STAT = 0x%08x", hSrio->RIO_SP[0].RIO_SP_ERR_STAT);

LOG_TRACE("SP_ERR[0].RIO_SP_ERR_DET = 0x%08x", hSrio->RIO_SP_ERR[0].RIO_SP_ERR_DET);

if(hSrio->RIO_SP[0].RIO_SP_ERR_STAT & 0x00010000 || hSrio->RIO_SP[0].RIO_SP_ERR_STAT & 0x00000100) //Output Error Stop

{

hSrio->RIO_PLM[0].RIO_PLM_SP_LONG_CS_TX1 = 0x2003F044;

LOG_DEBUG("Corrected Error Stop Condition. SP[0]_ERR_STAT = 0x%08x", hSrio->RIO_SP[0].RIO_SP_ERR_STAT);

}

LOG_TRACE("Clearing Errors. ");

hSrio->RIO_SP_ERR[0].RIO_SP_ERR_DET = 0;

hSrio->RIO_SP[0].RIO_SP_ERR_STAT = 0xFFFFFFFF;

return 0;

}

Hope the rest of your project goes well!

Brandy

0 David Yang1 over 12 years ago in reply to Brandy Jabkiewicz

Intellectual 435 points

Hi,Brandy,

So glad to see you on forum again! Thanks for your tips, and I haven't encounterd misalignment problems yet, the Keystone_srio example's author comment all the function related to SP[0]_ERR_STAT & debug information, just check the port connect state. So I failed to notice the debug&err state check part. I will run this test later:)

And by the way, I test the peak speed beween 2DSPs(1*1*1*1 port mode, 5.0gbps speed), it's 3722Mbps(pure data, 8/10 coding cost not include). And the speed between DSP and V6 Fpga is 3456~3473Mbps. The Theroretcial speed should be 4gbps. Is the speed loss reasonable, or still has room for improvment?

Thanks again!

Regards,

David Yang

0 Brandy Jabkiewicz over 12 years ago in reply to David Yang1

Mastermind 6325 points

Is it through a switch perhaps? I would say there would be room for improvement, but I haven't done much profiling myself yet. Sorry. Maybe repost and see if TI knows.

0 David Yang1 over 12 years ago in reply to Brandy Jabkiewicz

Intellectual 435 points

Direct connect. I check the code again, the actual speed might be a little faster, for there are some code used for verifying data landing which cost a little cycle.

Processors

Processors forum

About srio ackid in C6678 using Keystone_srio program