Checking for LSU read/write completion

Peter Robertson

Expert 2770 points

I have a simple example that is trying to perform a trivial maintenance read from an SRIO switch.

The code I have works intermittently an apparently random number of times.

typedef unsigned int UINT32;

typedef struct {
   UINT32 R1;
   UINT32 R2;
   UINT32 R3;
   UINT32 R4;
   UINT32 R5;
} LSU_DATA;

SRIO *Srio = (SRIO *)SRIO_REGS;
LSUREG *LSU = &Srio->LSU[0];

// The next line is needed to stop the compiler over-opimising writes to the LSU registers.
#pragma FUNC_CANNOT_INLINE(SetLSU);
void SetLSU(LSU_DATA *D) {
   while (LSU->R[6]>>30) {}
   LSU->R[0] = 0;
   LSU->R[1] = D->R1;
   LSU->R[2] = D->R2;
   LSU->R[3] = D->R3;
   LSU->R[4] = D->R4;
   LSU->R[5] = D->R5;
}

UINT32 maint_read(UINT32 addr) {
   UINT32   value = byteswop(0xDEFACED);
   LSU_DATA D;
   FLUSH(&value, 4);                // flush cache
   D.R1 = addr;
   D.R2 = GLOBAL(&value);
   D.R3 = 4;                        // 4 bytes
   D.R4 = (SWITCHID<<16)
        | R4_IDSIZE16
        | R4_PRIORITY2
        | R4_INTREQ;
   D.R5 = TYPE_MAINT_READ;
   SetLSU(&D);
   UINT32 timeout = 20;
   while ((Srio->LSU_INT[0].ICSR&0x10001)==0) {
      if (--timeout==0) {
         printf("*** read time-out LSU0_ICSR=%08X LSU1_ICSR=%08X\n",
              Srio->LSU_INT[0].ICSR, Srio->LSU_INT[1].ICSR);
         break;
      }
      msDelay(50);
   }
   Srio->LSU_INT[0].ICCR = 0x10001;
   return = byteswop(value);
}

Running this in a loop trying to read with addr=0x6C returns the correct value a few times (1, 2, or 3, randomly)
then times-out for 10 or 20 times, then works, then times out, ... When it fails to work, 'value' is left unchanged
and 0xDEFACED is returned. It seems that the first read after initialising the SRIO ALWAYS works.

The examples I have seen either make no attempt to determine that LSU operations have finished or wait until the
destination has changed (an obviously useless method).

I have the same problem with all LSU transactions - sometimes it works, sometimes it doesn't; the address used seems
to make no difference to this. I have tried many variants of the code, but nothing works reliably.

All the addresses used refer to valid registers in the switch. Very, very rarely, the example runs to completion without error.

What is the correct way to determine that LSU operations have completed? The documentation is far from clear on this.
It seems that obvious way of doing this, looking at LSU R6, will simply lock the LSU and kill the next transaction.

What am I doing wrong?

over 13 years ago

0 tscheck over 13 years ago

TI__Mastermind 23525 points

Are you checking/reading the LSU_reg6 every time before programming the LSU registers for the maintenance request? You must check for FULL bit at a minimum if you are only using one CPU to program the LSU BEFORE trying to write the registers. If you have more than one CPU using a given LSU, you must check for BSY and FULL bits to lock the LSU to a given CPU.

For determining the status of the LSU transaction, what completion codes are you getting on the transactions (RIO_LSUx_STAT)? You are given an LTID and LCB when you do the LSU_reg6 read above that I mentioned. These two pieces of info tell you exactly which field to read in the RIO_LSUx_STAT to determine the completion code. Remember also, that if you are getting any other completion code than CC=000, will need to handle the error case in one of the methods below.

In case of an error the LSU is not automatically loaded with the next shadow registers. It waits for the s/w to intervene. When the CPU gets an interrupt, it reads the relevant RIO_LSU_STAT_REGx to figure out the details of the error. Now the s/w can do one of the following:
•   The s/w can choose to not do anything. In this case, a timer will expire (based on the Timeout_cnt).  CPU does nothing and waits for the Timeout_cnt to expire, the HW discards the current transaction that caused the issue. It will then reload the next shadow register on its own. If the LSU was setup for an EDMA, it will additionally send a good completion interrupt to the EDMA thus enabling the EDMA to automatically load the next transaction.
•   The s/w fixes the issue (for e.g. enables the port that is XOffed), and sets a restart bit for the specific LSU. The LSU will terminate the current transaction and load the next set of shadow registers.
•   The s/w decides to flush the transaction. It writes to the flush bit for the LSU. All transactions originating from the same SRCID that are present in the shadow registers for the specific LSU will be flushed. This can take more than one cycle to do the flush.
The flush bit and the restart bits are really the write version of the LSU_Reg6.

For each transaction the following interrupts will also be generated. It is up to the software which of these interrupts does it wants to route and use based on RIO_LSU_SETUP_REG1 and (RIO_LSU0_ICCR or RIO_LSU1_ICCR):

• Good completion per SRCID or LSU based on the setup bits in RIO_LSU_SETUP_REG1.
• Error completion per SRCID defined in LSU_Reg4

It appears from your code snippet, that you are reading the ICSR interrupt bits to determine completion. This is acceptable. Again if there is an error, the above error handling applies.

Hope that helps,

Travis

0 Peter Robertson over 13 years ago in reply to tscheck

Expert 2770 points

"Are you checking/reading the LSU_reg6 every time before programming the LSU registers for the maintenance request?"

void SetLSU(LSU_DATA *D) {
while (LSU->R[6]>>30) {}

"For determining the status of the LSU transaction, what completion codes are you getting on the transactions (RIO_LSUx_STAT)? You are given an LTID and LCB when you do the LSU_reg6 read above that I mentioned. These two pieces of info tell you exactly which field to read in the RIO_LSUx_STAT to determine the completion code."

The obvious question follows: when is the completion code valid? As there is no "not yet completed" state in the completion code, how do you know when it is valid? In the failure case, the interrupt event never happens (neither success nor an error), so it appears there is no way to know when to check the status.

0 Peter Robertson over 13 years ago in reply to Peter Robertson

Expert 2770 points

I have just run the example again.

Read #1 - worked and returned the correct answer (before the read, R6=0x00000010)

Read#2 - worked and returned the correct answer (before the read, R6 = 0x00000011)

Read#3 - timed out. LSU0_ICSR = 0, LSU1_ICSR=0

Before the read, R6 = 0x00000012

After the time-out:

LSU STAT 0 = 0x00000011 all other LSU STAT x = 0.

As far as I can see, there is no indication of an error and no completion status.

Read#4 - timed out. LSU0_ICSR = 0, LSU1_ICSR=0

Before the read, R6 = 0x00000014

After the time-out:

LSU STAT 0 = 0x00000011, all others 0.

0 Peter Robertson over 13 years ago in reply to Peter Robertson

Expert 2770 points

Running it once more gave 9 successful reads with R6 before each one being:

00000010
00000011
00000012
00000013
00000000
00000001
00000002
00000003
00000010

Before the tenth read, R6=0x00000011 and the read timed out with

LSU0_ICSR = 0 LSU_ICSR1 = 0
LSU STAT 0 = 00000001, all others 0.

0 Peter Robertson over 13 years ago in reply to Peter Robertson

Expert 2770 points

You say that the LTID and LCB give enough information to select the correct bits from the status registers.

This may be so, but I have no idea how to use that information to select the correct bits. As usual, the
documentaiton assumes you know how to do this. It states: "LCB (LSU Context Bit): This gives frame of
reference for status bits, whether it is for the current transaction or the previous/next one."
How do you use one bit to determine three possible states: current, previous, next?

Also, the documentation (table 2-12) does not bother to say which bits in the field are the completion code
and which is the LSU Context bit. I would assume that the least-significant three bits are the completion
code and the most significant bit is the LSU Context bit, but they could just as well be the other way round.

Processors

Processors forum

Checking for LSU read/write completion