TMS320C6678: 64-bit PCIe Addressing Boot Problem

Thank you, Yordan.  I'm looking forward to their response

In thread https://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/623636 

Brad,

We do not release the .out and the .map file for the boot ROM . We only provide the source for reference so you will not be able to debug the ROM code by loading symbols and setting break points. 

We provide boot examples and boot utilities to convert application to bootable binary. IF you are running into any issues, you can report the Program counter values and run the debug GEL file we provide and check the boot parameter table to see how the boot is progressing.

Regards,
Rahul

Hi Eric,

Thanks for your reply!  Please find my responses to your questions below.

Can you clarify if you used TI 6678 EVM or your own board with 6678 for PCIE boot test? It looks the latter, please confirm. 

Yes, we are using a custom board.  All the results reported are for the custom board.

Also, it looks you used the PCIE boot mode by setting the DEVSTAT register and already adapted the code so there is a stable PCIE link between C6678 and host PC, you are able to write the boot image into DSP L2 address and write the _c_int00 into MAGIC_ADDRESS. 

Yes, that's correct.  The link between host and C6678 seems stable and I am able to read/write various memory/registers.  For loading the software I write all the program data to DSP L2 and the _c_int00 address to MAGIC_ADDRESS.

I have tried issuing legacy interrupts, MSI interrupts and IPC.  Below are the methods used to trigger MSI and IPC:

/*
Issues a MSI interrupt
*/
NTSTATUS triggerMsiInterrupt(PDSP_INFO pDspInfo)
{
#define ADDR_MSI_IRQ 0x21800054
#define ADDR_MSI0_STATUS_RAW 0x21800100
#define ADDR_EP_IRQ_SET 0x21800064

UINT32 rval;
UINT32 buffer;

buffer = 0;
rval = writeDSPMemory(pDspInfo, NDX_PCIE_CFG, ADDR_MSI_IRQ, &buffer, (UINT32)sizeof(UINT32));

buffer = 1;
rval = writeDSPMemory(pDspInfo, NDX_PCIE_CFG, ADDR_MSI0_STATUS_RAW, &buffer, (UINT32)sizeof(UINT32));

buffer = 1;
rval = writeDSPMemory(pDspInfo, NDX_PCIE_CFG, ADDR_EP_IRQ_SET, &buffer, (UINT32)sizeof(UINT32));

KdPrintEx((DPFLTR_IHVDRIVER_ID, 4, "MSI interrupt triggered. %d bytes written\n", rval));

return rval;
}

NTSTATUS unlockBootConfigMMRs(PDSP_INFO pDspInfo)
{
UINT32 rval;
UINT32 buffer;

buffer = KICK0_UNLOCK;
rval = writeDSPMemory(pDspInfo, NDX_CHIP_LEVEL_REGISTERS, CHIP_LEVEL_BASE_ADDRESS+KICK0, &buffer, (UINT32)sizeof(UINT32));

buffer = KICK1_UNLOCK;
rval = writeDSPMemory(pDspInfo, NDX_CHIP_LEVEL_REGISTERS, CHIP_LEVEL_BASE_ADDRESS+KICK1, &buffer, (UINT32)sizeof(UINT32));

return rval;
}

NTSTATUS triggerIPC(PDSP_INFO pDspInfo, UINT8 coreNum)
{
UINT32 rval;
UINT32 buffer;

buffer = 1;
rval = writeDSPMemory(pDspInfo, NDX_CHIP_LEVEL_REGISTERS, CHIP_LEVEL_BASE_ADDRESS+IPCGR(coreNum), &buffer, (UINT32)sizeof(UINT32));

return rval;
}

During the boot process, the RBL executes an IDLE command on the secondary
CorePacs and keeps the secondary CorePacs waiting for an interrupt. After the
application code to be loaded in these secondary CorePacs are loaded and the
BOOT_MAGIC_ADDRESS values in individual CorePacs are populated, the
application code in the CorePac0 can trigger the IPC interrupt to wake up the
secondary cores and branch up to the address specified in the
BOOT_MAGIC_ADDRESS.

Can you try this?

So, you're suggesting to write the program data to each of the CorePacs and then send IPC to each core?  Should IPC be sent to core0, too? Should anything different be done with core0?

Note: The code I am loading is the ddrInitCode (from pcie_ddr_init_6678.h) provided with the linux bootloader example.  

Thanks!
Brad

Thank you, Rahul.  That issue looks a lot like mine. Let me adapt my code and I'll report back with status (I'm working on another issue now so it may be a day or two before I get back to this)

Thanks,
Brad

Hi Rahul,

I finally returned to the project! I have adapted my code to upon loading the code to all cores:

1. Write start address to MAGIC_ADDRESS for each core (0x1087_FFFC | (core << 24))
2. Issue MSI interrupt on the C6678 PCIe Endpoint by writing 0x1 to EP_IRQ_SET (0x2180_0064)

Upon writing 0x1 to EP_IRQ_SET, this triggers MSI interrupt on my Windows Host machine.  

C6678EvtInterruptIsr: MessageID: 0 Interrupt: Message #: 0 VECTOR: 182 IRQL: 11
C6678EvtInterruptIsr: DSP-1 matches IRQ Vector
C6678EvtInterruptIsr: DSP-1 EP_IRQ_STATUS [0x00000001]

C6678EvtInterruptIsr: MessageID: 0 Interrupt: Message #: 0 VECTOR: 101 IRQL: 6
C6678EvtInterruptIsr: DSP-0 matches IRQ Vector
C6678EvtInterruptIsr: DSP-0 EP_IRQ_STATUS [0x00000001]

So, in the Windows driver, the MSI interrupt handler:

1. Writes 0x1 to EP_IRQ_CLR (0x2180_0068)
2. Writes 0x4 to IRQ_EOI (0x2180_0050)

However, when using Blackhawk JTAG debugger, the cores are still sitting at IDLE instruntions - core0 is still at 0x20b0_1290 and cores 1-7 are at 0x20b0_02c8

I have confirmed that from in the debugger, writing 0x4 to the Interrupt Set Register (ISR) does wake the cores up from IDLE state.

Is this the correct procedure for waking up PCIe endpoint?