This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

EtherCAT with PRU Distributed Clock jitter on SYNC0

Other Parts Discussed in Thread: AM3357, SYSBIOS

Hello!

I'm using the following setup:

PC with EtherCAT master <-> EtherCAT Module with HW ET1100 <-> AM3357 with EtherCAT via PRU

So the module with the "real" ET1100 is the DC Master, the AM3357 is the DC Slave.

The EtherCAT slave application running on AM3357 is based on SDK 1.1.0.8
I have the SYNC0 pin of both Modules (DC Master, DC Slave) hooked to my oscilloscope.
I have setup the SYNC0 pins to generate a short pulse on every cycle.

Basically, it works, but I see a lot of jitter, especially when I disturb the system by sending
for example CoE requests from the master.

Then I did a trick to rule out the slave app itself:
I simply killed the slave app by Ctrl-C. The PRU continues to run and as long as the master just
sends its cyclic frame (no other requests, state changes and the like), the SYNC pulses are still
produced just like before. In this case there is only very little jitter, but if you let it run
for say 20-30 minutes, there will be a bad jitter around 1us as well.

To further check what causes the disturbance, I wrote a small program that can send a single command
to the PRU. I chose CMD_DL_USER_READ_AL_CONTROL (8). This command is usually sent after reading the
AL_CONTROL register - nothing that should disturb the DC generation in any way.
When I start the program, I get a jitter around 1.3us. It might be interesting that the jitter is not
a single spike caused by the delay that the command might cause, but rather the SYNC0 edge is wobbling
back and forth as if the DC regulator got a disturbing pulse.

After ruling out all other possible causes I am now stuck and hope TI will investigate the problem.
Any ideas?

Best regards,
Manuel Köppen

My small test program:

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>

#include "prussdrv.h"
#include "pruss_intc_mapping_ecat.h"
#include "tiescbsp.h"
#include "tieschw.h"

static volatile t_host_interface* pPRU_HostIntf;    /* 0x4a300000 */

#define PRUSS_HOST_INT                  (HOST_AL_EVENT - 20)

int main(int argc, char **argv) {
    int err;
    err = prussdrv_init();
    if (err) { printf("prussdrv_init() failed!\n"); return 1; }
    
    err = prussdrv_open(PRUSS_HOST_INT);
    if (err) { printf("prussdrv_open() failed!\n"); return 2; }

    err = prussdrv_map_prumem(PRUSS0_PRU0_DATARAM, (void**)&pPRU_HostIntf);
    if ((err) || (pPRU_HostIntf == NULL)) { printf("prussdrv_map_prumem(PRUSS0_PRU0_DATARAM, ...) failed!\n"); return 3; }
    
    pPRU_HostIntf->cmdlow = CMD_DL_USER_READ_AL_CONTROL; /* 8 */
    pPRU_HostIntf->param1low = 0;
    pPRU_HostIntf->param2low = 0;
    pPRU_HostIntf->cmdlow_ack = 1;
    prussdrv_pru_send_event(ARM_PRU_EVENT1);
    while (pPRU_HostIntf->cmdlow_ack) {
        printf(".");fflush(stdout);
    }
    pPRU_HostIntf->cmdlow = 0xFF;
    return 0;
}

  • Hi Manuel,

    I will forward this to the ISDK team.

  • Sorry for the delay, unfortunately we can't reproduce this with SYS/BIOS SDK application. There shall be no effect on jitter with doing this as AL control command just informs AL control register read by PDI to firmware . Are you using SYSTEM_TIME_PDI_CONTROLLED in this configuration ? Is this feasible to reproduce with using SYS/BIOS application in your setup?
  • It took quite some time to get something reproduceable using sysbios, but I think I got it now.
    I have used the example from sdk/protocols/ethercat_slave/ecat_appl on an ICE board.

    I edited the file sdk/protocols/ethercat_slave/ecat_appl/EcatStack/tiescutils.c

    Somewhere in the upper region, after "#define SYSTIME_TEST" I added:

    #define MANUTEST
    
    #ifdef MANUTEST
    
    volatile Uint32 g_MainFreeze;
    extern volatile t_host_interface *pHost2PruIntfc;
    
    #include "hw/soc_AM335x.h"
    #include "uart_irda_cir.h"
    
    void manuUartIsr(UArg arg)
    {
        char x;
        switch(arg) {
            case UART_INTID_RX_THRES_REACH:
            case UART_INTID_CHAR_TIMEOUT:
                do {
                    x=UARTCharGetNonBlocking((uartInstance==3)?SOC_UART_3_REGS:SOC_UART_5_REGS);
                    if (x=='f') {
                        g_MainFreeze^=1;
                    } else if ((x=='t')||(x=='T')) {
                        Uint32 cnt,timeout;
                        Uint16 ack;
                        if (x=='t') {
                            cnt=1;
                            UARTPutString(uartInstance," dummy");
                        } else {
                            cnt=100000;
                            UARTPutString(uartInstance," DUMMY");
                        }
                        while (cnt--) {
                            bsp_pruss_cmd_intfc_write_word(CMD_DL_USER_READ_AL_CONTROL,&pHost2PruIntfc->cmdlow);
                            bsp_pruss_cmd_intfc_write_word(0,&pHost2PruIntfc->param1low);
                            bsp_pruss_cmd_intfc_write_word(0,&pHost2PruIntfc->param2low);
                            bsp_pruss_cmd_intfc_write_word(1,&pHost2PruIntfc->cmdlow_ack);
                            ENABLE_PRUSS_ACCESS_FROM_HOST();
                            PRUSSDRVPruSendEvent(ARM_PRU_EVENT1);
                            ASSERT_DMB();
                            ASSERT_DSB();     
                            timeout=0;
                            do {
                                ack = bsp_pruss_cmd_intfc_read_word(&pHost2PruIntfc->cmdlow_ack);
                                if (++timeout == 100000) {
                                    UARTPutString(uartInstance," TO!!");
                                    break;
                                }
                            } while (ack);
                            bsp_pruss_cmd_intfc_write_word(0xFF,&pHost2PruIntfc->cmdlow);
    //                      bsp_send_command_to_firmware(CMD_DL_USER_READ_AL_CONTROL, 0, 0);
                        }
                    }
                } while(x!=(char)-1);
                break;
        }
    }
    #endif

    then in the mainloop I added

    #ifdef MANUTEST
                if (g_MainFreeze) {
                    UARTPutString(uartInstance,"mainloop is now frozen\r\n");
                    while(g_MainFreeze) Task_sleep(100);
                    UARTPutString(uartInstance,"mainloop now continues!!!\r\n");
                }
    #endif
    

    just before the "Task_yield();" line.

    and lastly I replaced

        UartOpen(uartInstance,NULL);

    with

    #ifdef MANUTEST
        UartOpen(uartInstance,&manuUartIsr);
    #else
        UartOpen(uartInstance,NULL);
    #endif
    

    With these changes, you can press three buttons on the serial line:
    f - freezes or unfreezes the mainloop
    t - issues one CMD_DL_USER_READ_AL_CONTROL command to PRU
    T - issues CMD_DL_USER_READ_AL_CONTROL command 100000 times to PRU

    The hardware setup is mostly the same es before:
    The ET1100 module is the DC master and first slave connected to the master,
    the ICE board is the DC slave and connected to the ET1100 module.

    First you have to get the master running and the SYNC0 on the scope,
    I recommend triggering on SYNC0 of the ET1100 module.
    There should be very little jitter.

    Then press f on the console. Notice that a litte jitter occurs!
    When pressing f again, the main loop is allowed to continue.
    Notice that considerable jitter occurs. You can press f a few times to study the phenomenon.
    Then, having frozen the mainloop, press T and you will see severe jitter (up to 800ns).
    You may have to press T several times to get the 800ns reading.
    Generally t or T should not be pressed while main-loop is running (though nothing bad really happens).

    It suprises me that the jitter is not that big as with our board and application, but it still looks
    the same - not a single spike caused by a delay, but rather the SYNC0 edge is wobbling back and forth.

    I hope this info enabled you to reproduce it.

    Best regards,
    Manuel Köppen

  • It's been a week now... is somebody working on this issue?
    Are you able to reproduce it now?

    In the meantime I have found out that the regulator which controls the DC time in the PRU seems to be pretty bad:

    1) The adjustment of the DC Clock speed seems to be very coarse.

    When I stop the master frames (simply by unplugging the cable) and thus the DC regulation, I see the SYNC0 signal on the scope drifts away very quickly (the clock seems to be up to about 100ppm off!! best value I have seen was about 2us deviation per second). On a real ET1100 the DC clock seems to be much more exact (deviation only 100ns per second) when it's in a steady state.

    2) The regulator tends to oscillate a lot and needs much more damping.

    When I restart the master frames (simply by re-plugging the cable) after the DC clock has built up 100us deviation, it takes 100 seconds for the clock to be in sync again. During the 100s the SYNC0 position oscillates back and forth (+-100us first and slowly getting better then). On an ET1100, it takes about 6s to lose 100us deviation, with another 9s jittering in ns scale until it is fully synchronized again.

    Improving the regulator could certainly improve the impact of the bug that causes a (non-existent) deviation causing the regulator to swing back and forth. But it would better to find the actual bug first.

    Best regards,
    Manuel Koeppen
  • Hi,

    This seems to indicate that something is wrong with DC filter configuration - Can you set this like following for AM335x slave

    System filter depth (0x935) = 0x0

    Speed counter filter depth (0x935) = 0xc

  • Minor correction

    PratheeshGangadhar said:

    System Time Diff  filter depth (0x934) = 0x0

    Speed counter filter depth (0x935) = 0xc

  • Reading the memory at 0x934 and 0x935 reveals that these settings seem to be the default (they already read 0 and 0x0c here).
    I wrote the same values to those registers anyway but that didn't change anything.
    I played around with the registers a litte bit,
    setting any other value than 0 to 0x934 instantly leads to an unstable oscillating system.
    Setting different values to 0x935 seems to have a minor effect only if any, not really recognizable.

    Best regards,
    Manuel Köppen