HPI on DM365 fails if used simultaneously with Codec Engine and EDMA

Problem statement: Host Port Interface fails when HPI data transfers interfere with EDMA(dm365mmap module) transfers and Codec Engine (h264dec) activity.

Hardware description: 2 evaluation modules, EVMOMAPL137 and EVMDM365 are connected by daughter card, which creates link between AEMIF expansion connector(P13) on EVMOMAPL137 and AEMIF expansion connector(J14) on EVMDM365. EVMOMAPL137 system boots and controls EVMDM365 via EMIFA/HPI link.

System settings: EVMDM365 is configured to HPI mode by setting SW4 switch to "111". AEMIF on EVMOMAPL137 is set to "Normal Mode" with "Extended wait cycles" enabled. Detailed configuration is here

1) Asynchronous Wait Cycle Configuration Register (WP0=1, MAX_EXT_WAIT=FFh)

2) Asynchronous n Configuration Register (SS = 0, EW = 1, W_SETUP = 0, W_STROBE = 0, W_HOLD = 0, R_SETUP = 0, R_STROBE = 7, R_HOLD = 0, TA = 0, ASIZE = 1)

Software: ti-dvsdk_dm365-evm_4_00_00_22 is used on DM365.

Test case: OMAP starts DM365 by uploading customized UBL, UBoot and Linux Kernel and following HPI Boot flow guidelines for DM365. After being started, DM365 runs using NFS file system. Now it can be controlled by means of linux commands issued via serial console. At the same time, OMAP system can perform data writes/reads to/from DM365's memory space.

I have made a simple program for DM365 which performs following tasks

1. decodes h264-video file (using DVSDK API);
2. performs dummy EDMA transfers (using DMAI's "Framecopy_..." routines with acceleration being activated);
3. allocates buffer which is used for holding data, transferred via HPI, and reports its physical address and size (using DMAI's "Buffer_..." routines).

Omap side application accepts remote buffer physical address and starts non-burst writes, followed by reads to ensure data integrity. After each write-read cycle system verifies "Asynchronous Timeout" bit in "EMIFA Interrupt Raw Register". If timeout occurred, application resets "Asynchronous Timeout" bit and repeats write-read cycle.

Simplified application for writing meaningless data into some buffer on DM365 is provided below:

int BufferDWSize   = 256*1024/4;    // predefined size of the remote buffer on DM365
int BufferPhysAddr = 0x84422000;    // predefined address of the remote buffer on DM365
int cntAemifErr = 0;                // Aemif errors counter
int cntReadErr = 0;                 // read-back errors counter

// data write loop
for (i = 0; i < BufferDWSize; i++) {
    int ref_value  = i;
    int test_value = 0;

    do {
        // reset Asynchronous Timeout bit and wait for some time
        if (AEMIF_EIRR & 1) {
            AEMIF_EIRR = 1;
            AEMIF_EIMR = 1;
            taskDelay(sysClkRateGet()/1000);
        }
        HPIA_WRITE = BufferPhysAddr + i*4;    // set destination address
        HPID_WOAI_WRITE = ref_value;          // write some value
        test_value = HPID_WOAI_READ;          // read value back
   
        cntAemifErr += (AEMIF_EIRR & 1);
        cntReadErr += (test_value != ref_value);

    // repeat if either timeout or read error happened
    } while ((AEMIF_EIRR & 1) || (test_value != ref_value));
}

Issues: In its original state, when running all the above-mentioned tasks, system fails within several seconds from the start. In this condition, HPI module on DM365 seems to go into some unpredictable state, because Host System (OMAP) application stays infinitely in inner loop and AEMIF timeouts occur every time after setting destination address for HPI write/read operation. At the same time, DM365 starts displaying NFS error message ("nfs: server 192.168.1.242 not responding, still trying"), stops video decoding (presumably because it can't read data from file system anymore), does not respond to keyboard input and sometimes after a minute or so displays kernel error message.

Also, read-back errors happen much more often than AEMIF timeouts. It means, it is impossible to tell that data has been delivered properly without reading it back or applying some kind of error correction.

I have performed several tests in different configurations and discovered that system works fine in these cases:

It also works fine if all EDMA-transfers are replaced by memcpy (when Framecopy job is created with acceleration deactivated). Neither read-back errors, nor AEMIF timeouts take place in this case. No errors at all. However, this mode is undesirable, because it creates heavy CPU load.

It can be seen from this table, that HPI seems to conflict with EDMA and Decode jobs only when both are active.

Questions:

1. Could you make suggestion on how EDMA transfers and H264 decoding task on DM365 can be configured in order to avoid conflicting situations with HPI transfers initiated by Host system?
2. Could you provide some guidelines on how to avoid HPI stalling conditions (how to get information about when HPI fifo is full and cannot deliver data because memory subsystem is busy and when it is empty and can accept new data) ?
3. Also, could you provide some recommendations on how to design hardware and software for safe HPI-data transfer. Because in some cases data appear to be corrupt and there is no indication about this condition to Host system because "Aemif Timeout" flag remains zero during such transfers ?
4. I may admit, that AEMIF timeouts might not always be registered by OMAP hardware and therefore read-back errors take place more often. Could you tell me then what can happen to HPI-module if supposed AEMIF timeout has been ignored and data write continues without making a pause ? Can it go to some unpredictable state and corrupt data within some arbitrary memory range ?
5. If AEMIF timeout happened, how much host application should wait before repeating transfer again ?