Compiler/AM5718: GPMC EDMA issue

Hi,

There is no GPMC driver available for AM57x device. Where did you get it? What is the software package you used on AM57x? Is it Processor SDK RTOS?

For the EDMA, there is a register called "OPT", can you provide the value programmed? It has info like SAM, DAM, SYNCDIM, FWID, etc. I want to look at. Also, how do you determine the number of transfer happened? You have a scope to look at the waveform or you enabled EDMA intermediate transfer completion interrupt and count the interrupt?

Regards, Eric

Hi

'm using "processor_sdk_rtos_am57xx_5_01_00_11".

I'm configuring the GPMC registers by my own from within C66 code, using CSL register access macros.
Attached you can find my GPMC configuration.

The OPT value is 0x00000204.

The number of transfers I can see with an external scope, viewing at the GPMC chip select signal.

Regards,
Markus

void SYS_InitGPMC(void)
{
  //Reset GPMC
  //SYS_GpmcRegs->ERR_TYPE = 0;
  CSL_FINST(SYS_GpmcRegs->SYSCONFIG,GPMC_SYSCONFIG_SOFTRESET, RESET);

  while((SYS_GpmcRegs->SYSSTATUS & CSL_GPMC_SYSSTATUS_RESETDONE_MASK) != CSL_GPMC_SYSSTATUS_RESETDONE_RSTDONE)
  {
    //wait till reset is done
  }
  //disable all irq
  SYS_GpmcRegs->IRQENABLE = 0x00000000;
  //no timeout
  SYS_GpmcRegs->TIMEOUT_CONTROL = 0x00000000;
  //wait active low and disable force posted write
  SYS_GpmcRegs->CONFIG = 0x00000000;
  //device options
  //GPMC_CLK frequency = GPMC_FCLK frequency
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_GPMCFCLKDIVIDER,DIVBY1); // div 2 = 133Mhz this must coresponde to clk_out1!!!
  //x1 latencies
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_TIMEPARAGRANULARITY,X2);
  //Nonmultiplexed attached device
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_MUXADDDATA,NONMUX);
  //NOR flash-like, asynchronous and synchronous devices
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_DEVICETYPE,NORLIKE);
  //16 bit
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_DEVICESIZE,SIXTEENBITS);
  //Wait input pin is WAIT0
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_WAITPINSELECT,W0);
  //Wait pin is monitored with valid data.
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_WAITMONITORINGTIME,RESETVAL);
  //Wait pin is not monitored for write accesses.
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_WAITWRITEMONITORING,WNOTMONIT);
  //Wait pin is not monitored for read accesses.
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_WAITREADMONITORING,WNOTMONIT);
  //device page (burst) length: 4 words
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_ATTACHEDDEVICEPAGELENGTH,FOUR);
  //First rising edge of GPMC_CLK at start access time
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_CLKACTIVATIONTIME,ATSTART);
  //Write synchronous
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_WRITETYPE,WRASYNC);
  //Single access
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_WRITEMULTIPLE,WRSINGLE);
  //Single access
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_READTYPE,RDASYNC);
  //Single access
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_READMULTIPLE,RDSINGLE);
  //Synchronous wrapping burst not supported
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG1,GPMC_CONFIG1_WRAPBURST,WRAPNOTSUPP);
  //CS Timing
  //no delays
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG2,GPMC_CONFIG2_CSONTIME,0);
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG2,GPMC_CONFIG2_CSEXTRADELAY,NOTDELAYED);
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG2,GPMC_CONFIG2_CSRDOFFTIME,GPMC_ACCESS_TIME);
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG2,GPMC_CONFIG2_CSWROFFTIME,GPMC_ACCESS_TIME);
  //ADV Timing
  //no delays
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG3,GPMC_CONFIG3_ADVEXTRADELAY,NOTDELAYED);
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG3,GPMC_CONFIG3_ADVAADMUXONTIME,0);
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG3,GPMC_CONFIG3_ADVONTIME,0);
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG3,GPMC_CONFIG3_ADVWROFFTIME,GPMC_ACCESS_TIME);
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG3,GPMC_CONFIG3_ADVRDOFFTIME,GPMC_ACCESS_TIME);
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG3,GPMC_CONFIG3_ADVAADMUXRDOFFTIME,GPMC_ACCESS_TIME);
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG3,GPMC_CONFIG3_ADVAADMUXWROFFTIME,GPMC_ACCESS_TIME);
  //OE and WE Timing
  SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG4 = 0x00000000;
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG4,GPMC_CONFIG4_OEONTIME,0);
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG4,GPMC_CONFIG4_OEOFFTIME,GPMC_ACCESS_TIME);
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG4,GPMC_CONFIG4_WEONTIME,0);
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG4,GPMC_CONFIG4_WEOFFTIME,GPMC_ACCESS_TIME);
  //RD cycle and WR cycle Timing
  //Set read and write cycle time to max for first testing
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG5,GPMC_CONFIG5_WRCYCLETIME,GPMC_ACCESS_TIME);
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG5,GPMC_CONFIG5_RDCYCLETIME,GPMC_ACCESS_TIME);
  //no pageburst and read access delays
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG5,GPMC_CONFIG5_PAGEBURSTACCESSTIME,0x0U);
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG5,GPMC_CONFIG5_RDACCESSTIME,GPMC_ACCESS_TIME-GPMC_RDDATAVALID);
  //WR accestime
  //add one clock cycle delay between reads
  SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG6 = 0;
  CSL_FINST(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG6,GPMC_CONFIG6_CYCLE2CYCLESAMECSEN,C2CDELAY);
  CSL_FINS(SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG6,GPMC_CONFIG6_CYCLE2CYCLEDELAY,0x1U);
  //set cs size to 128MiB and base address to 0x8000000 and enable cs
  SYS_GpmcRegs->CONFIG_BLOCK[GPMC_FPGA_CS_NR].CONFIG7 = 0x00000848;

}

Hi,

You are reading from the GPMC data space with range 0x0a00_806x with EDMA, ACNT=2, CCNT=1 and BCNT = 1, 2 ..... with AB SYNC. Both source and destination in address incremental mode. I can't see why actual transfer is greater than the BCNT. What data you got in the extra transfers?

Regards, Eric

Hi Eric

We had a look at the FPGA parallel Interface with the FPGA JTag Debugger.
In the attached files you can see the number of DMA read transfers depending
on Bcnt value (2, 3 and 5), resulting in 2, 4 and 8 read accesses (signal 'isl_gpmc_oe_re_n_IBUF')
on subsequent adresses (signal 'islv_gmpc_address_IBUF')

For all measures: Acnt=2 (16Bit Access), Ccnt=1.
Looks like some kind of read-ahead. But why? Where can it be turned off?

Regards,
Markus
Edma3 read access to FPGA 01.zip

Hi Markus,

Can you please provide a register dump of your GPMC registers for the chip select you are using?

What are your Cache/MMU settings?

Do you think this issue might resemble the one discussed here: https://e2e.ti.com/support/processors/f/791/t/316742?AM335x-GPMC-phantom-chip-select-generation#pi320966=1 ?

Regards,
Mark

e2e.ti.com/.../316742

Hi Mark

Attached you can find my complete GPMC register settings (I'm only using chip select 0).

I think your link to the other issue points in the right direction.
I have not set any special settings for memory address 0x0A00'0000 to 0x0AFF'FFFF (GPMC Access window for our FPGA), so I would assume SYSBIOS default settings are used.

How can I configure the MMU to access the above mentioned GPMC memory window as "device" memory (not "normal" or "strongly ordered" memory)?
Can you provide a code snipet for the SYSBIOS app.cfg file?

Regards,
Markus

Hi Eric

In the Technical Reference Manual I could see that there are many MMU devices.
- System MMU (for EDMA access)
- MPU MMU
- DSP MMU
- IPU MMU

The System MMU would be interessting to use.
How can I access it from C66 or A15 Code?

regards,
Markus

Markus,

I thought you run the GPMC test code on A15 core, you need to setup A15 MMU first level and second level translation table for the GPMC configuration register region and data region. Also as you used the EDMA to move data, you also need do the same A15 MMU for EDMA TPCC and TC0/1.

Regards, Eric

Hi Eric

I made the following adjustments to A15 app.cfg file:

========================================

Mmu.enableMMU = true;

var attrs = new Mmu.DescriptorAttrs();
Mmu.initDescAttrsMeta(attrs);
attrs.type = Mmu.DescriptorType_TABLE;
attrs.noExecute = true;
attrs.accPerm = 0;       // R/W at PL1
Mmu.setFirstLevelDescMeta(0x00000000, 0 /* ignored if table */, attrs /* ignored if table */);

/* Set FPGA Access attributes, non-cachable strongly-ordered memory */
attrs.type = Mmu.DescriptorType_BLOCK;
attrs.attrIndx = 1; // set to strongly-ordered, non-cachable
Mmu.setSecondLevelDescMeta(0x0A000000, 0x0A000000, attrs); // FPGA Access
Mmu.setSecondLevelDescMeta(0x0A200000, 0x0A200000, attrs); // FPGA Access
Mmu.setSecondLevelDescMeta(0x0A400000, 0x0A400000, attrs); // FPGA Access
Mmu.setSecondLevelDescMeta(0x0A600000, 0x0A600000, attrs); // FPGA Access
Mmu.setSecondLevelDescMeta(0x0A800000, 0x07800000, attrs); // route outside FPGA memory (to check MMU translation)
Mmu.setSecondLevelDescMeta(0x0AA00000, 0x07A00000, attrs); // route outside FPGA memory (to check MMU translation)
Mmu.setSecondLevelDescMeta(0x0AC00000, 0x07C00000, attrs); // route outside FPGA memory (to check MMU translation)
Mmu.setSecondLevelDescMeta(0x0AE00000, 0x07E00000, attrs); // route outside FPGA memory (to check MMU translation)

========================================

I still have the problem of more read transfers to FPGA than configured by EDMA (EDMA is configured by C66 code with the edma3 low level driver V2.12.5).
I checked that System EDMA is used (TPCC base address 0x4330'000) and not DSP1_EDMA (TPCC base address 0x40D1'0000).
The TRM says (MMU Overview chapter 20.1) that system EDMA only uses the MMU if configured so in CTRL_CORE_SMA_SW_7 register.
I enabled all four route-enable bits in that register (EDMA_TC0_RD, EDMA_TC0_WR, EDMA_TC1_RD, EDMA_TC1_WR). Still this does not help.

When I analyse the memory access to FPGA it seems that the MMU is not active.
See my following tests on that:

Without MMU the DMA reads between addresses 0x0800'0000 and 0x0FFF'FFFF go to FPGA (chip select 0 becomes active) -> ok
Without MMU the DMA reads between addresses 0x0000'0000 and 0x07FF'FFFF go outside FPGA (chip select 0 does not become active) -> ok
With the MMU enabled I would expect that DMA reads between addresses 0x0A80'0000 and 0x0AFF'FFFF are translated to 0x0780'0000 and 0x07FF'FFFF -> chip select 0 not active.
But instead with enabled MMU the chip select 0 still becomes active in this address range. So I would assume MMU is not active.

The A15 ROV viewer tells me that MMU is active (MMU enable property is "true"). The first and second level tables also look correct, see attached file.

In the TRM (spruhz7i.pdf) I saw the register MMU_CNTL (for MMU1 on address 0x4881 C044 and for MMU2 on address 0x4881 E044; both registers are 0).
One of those bits is named "MMUENABLE"... What does this mean?

More help is greatly appreciated.

Regards,
Markus

Markus,

Thanks for the experiment!

Without MMU the DMA reads between addresses 0x0800'0000 and 0x0FFF'FFFF go to FPGA (chip select 0 becomes active) -> ok
Without MMU the DMA reads between addresses 0x0000'0000 and 0x07FF'FFFF go outside FPGA (chip select 0 does not become active) -> ok
With the MMU enabled I would expect that DMA reads between addresses 0x0A80'0000 and 0x0AFF'FFFF are translated to 0x0780'0000 and 0x07FF'FFFF -> chip select 0 not active.
But instead with enabled MMU the chip select 0 still becomes active in this address range. So I would assume MMU is not active.

With the MMU enabled, and if you do an EDMA reads between 0x0A000000 and 0x0A200000, this will translate to the same physical address, will you see the CS0 becomes active as expected? But just give you more read than BCNT?

From the ROV MMU level 1 and level 2 view, the MMU is active and configuration is correct. The same should be observed at the address of 0xa4017000 for the nextLevelTablePtr (your 0x0000_0000 to 0x4000_0000 region).

I looked at a live system here with A15 on AM572x IDK EVM (running some other applications), I knew the MMU is working. The MMU1_CTL and MMU2_CTL are both 0.

Regards, Eric 

Hi Eric

With the MMU enabled, and if you do an EDMA reads between 0x0A000000 and 0x0A200000, this will translate to the same physical address, will you see the CS0 becomes active as expected? But just give you more read than BCNT?

Yes, same physical address but more read accesses than specified by Bcnt (as described in my pervious posts).

I made so more tests on A15 core with these MMU settings:

==========================

var attrs = new Mmu.DescriptorAttrs();
Mmu.initDescAttrsMeta(attrs);
attrs.type = Mmu.DescriptorType_BLOCK;
attrs.noExecute = true;
attrs.accPerm = 0;       // R/W at PL1

attrs.type = Mmu.DescriptorType_TABLE;
Mmu.setFirstLevelDescMeta(0x00000000, 0 /* ignored if table */, attrs /* ignored if table */);

/* Set FPGA Access attributes, non-cachable strongly-ordered memory */
attrs.type = Mmu.DescriptorType_BLOCK;
attrs.attrIndx = 1;
Mmu.setSecondLevelDescMeta(0x07000000, 0x0A000000, attrs); // route to FPGA
Mmu.setSecondLevelDescMeta(0x0A000000, 0x0A000000, attrs); // unchanged FPGA address
Mmu.setSecondLevelDescMeta(0x0A800000, 0x80000000, attrs); // route to DDR RAM

==========================

GPMC base address for FPGA access is 0x0A00'0000, memory window is 128MiB.

1)
Access via System EDMA to these (virtual) adresses:
0x0700'0000: FPGA is not accesses (chip select stays 0) -> not ok, would expect to be mapped to 0x0A00'0000 (FPGA)
0x0A00'0000: FPGA is accessed (chip select active) -> ok
0x0A80'0000: FPGA is accessed (chip select active), -> not ok, DDR RAM on address 0x8000'0000 should be accessed

2)
Direct access via HW_RD_REG32 to the same (virtual) addresses:
0x0700'0000: FPGA is accessed (chip select active) -> ok
0x0A00'0000: FPGA is accessed (chip select active) -> ok
0x0A80'0000: FPGA is not accessed (chip select inactive) -> ok,  read gives same value as on DDR RAM address 0x8000'0000

Counter-Checked register CTRL_CORE_SMA_SW_7 (address 0x4A003C18): value is 0x00003C00 -> ok, all MMU route enable bits are set.

MMU still is not enabled for System EDMA access, how can it be enabled?

Regards,
Markus

Hi

Is there any progress so far?
Does system EDMA access via MMU work on your AM571x hardware?

Regards,
Markus

Hi

Yes, reading at 0x0A000000 with MMU enabled will translate to the same physical address. But the number of transfer is not as expected.

Can you direct me to an example with EDMA access to GMPC with correct transfers?

Regards,
Markus

Hi,

Sorry we only have GPMC examples for AM335x and AM437x devices, they access a flash over GPMC, tested on TI EVMs. The examples don't use DMA. For AM57x we even don't have a GPMC example because of the EVM limitation. I still have no clue why GPMC + DMA on AM572x has more transactions than the EDMA BCNT programmed. Let me check if my colleagues have more ideas.

Regards, Eric

Hi,

Just got info from my colleague. We do have a GPMC + EDMA CSL FL example for the A15 on AM572x. Hardware modifications are required though in order to do any testing. The user guide is: http://software-dl.ti.com/processor-sdk-rtos/esd/docs/latest/rtos/index_device_drv.html#csl-fl-examples

The code example: ti\pdk_am57xx_1_0_xx\packages\ti\csl\example\gpmc\fpga_read_write. Hope this can serve as a reference.

Regards, Eric 

Hi Eric

Thank you for the example.
As I will be on vacancy for the next three weeks, my tests will have some delay.
Please keep this thread open meanwhile. Thank you.

Regards,
Markus

Hi Eric

I have installed PDK V1.0.12.
In the folder "C:\ti\pdk_am57xx_1_0_12\packages\ti\csl\example\gpmc" I can only find "nor_read_write" which does not use the DMA.
You mentionned "fpga_read_write" (instaead of "nor_read_write") do I need a newer PDK version?

Regards,
Markus