This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TMDSICE3359: EtherCAT PD watchdog expires

Part Number: TMDSICE3359

Hello,

I've got a problem while trying to run a Ethercat-Slave application synchron to the SyncManger2.

My hardware setup is as following: Master (Beckhoff TwinCAT 3.1) <---> Slave (TMDSICE3359). The ethercat slave is configured to the provided "ethercat_slave_full_AM335x_arm" example, which means it has process data output and input.

The error is reproducible with following steps:

1. Run the ethercat network in "free run" mode

2. Request OP state from the slave. The slave goes into OP.

3. Deactivate/Toggle "free run" mode in TwinCAT.

The slave now goes after 100 ms (configured PD watchdog timeout) back to SAFEOP. TwinCAT shows following error log:


 22.05.2018 11:33:20 116 ms | 'Box 2 (complex slave)' (1001): abnormal state change (from 'OP' to 'SAFEOP') with code 0x1b, 'Sync manager watchdog'. Try to go back to 'OP'...

I suspect the root cause for the error either in my configuration, or in the TI ethercat slave PRU software.

My software configuration is as following:

I obtained the slave stack code from Beckhoff and enabled ESC_SM_WD_SUPPORTED, AL_EVENT_ENABLED and DC_SUPPORTED. I generated the slave stack code and copied the files to the ethercat_slave_full_AM335x_arm project.

According to the PRU ICSS EtherCAT firmware API guide, ENABLE_PDI_TASK and ENABLE_SYNC_TASK are defined in tiescbsp.h.

The software versions of the development kits which I use are listed here in the screenshot:

Observations:

The function CheckIfEcatError(void) in ecatslv.c sets the error code 0x1b (ALSTATUSCODE_SMWATCHDOG), as a result of the expired watchdog.

      HW_EscReadWord(WdStatusOK, ESC_PD_WD_STATE);
      WdStatusOK = SWAPWORD(WdStatusOK);

      wdBuffer[wdBufferIndex] = WdStatusOK;
      wdBufferIndex++;
      wdBufferIndex = wdBufferIndex % 1023;

      if (!(WdStatusOK & ESC_PD_WD_TRIGGER_MASK) && (nPdOutputSize > 0))
      {
         /*The device is in OP state*/

         if (bEcatOutputUpdateRunning
            )
         {
            AL_ControlInd(STATE_SAFEOP, ALSTATUSCODE_SMWATCHDOG);
            return;
         }

         else
         {
            bEcatFirstOutputsReceived = FALSE;
         }
      }

However the PRU ICSS EtherCAT firmware API guide says:

"bsp_set_pdi_wd_trigger_mode
Function
  void bsp_set_pdi_wd_trigger_mode (Uint32 mode)
Parameters
•• mode: PDI_WD_TRIGGER_RX_SOF/PDI_WD_TRIGGER_LATCH_IN/PDI_WD_TRIGGER_SYNC0/1_OUT
•• Return value: none
Description Configure PDI WD trigger mode, PDI WD is triggered automatically by h/w on RX_SOF(port0/port1),
latch0/1 input high, SYNC0/1 out high. PDI WD is also triggered whenever host sends a command to firmware. PDI
WD may not expire if host stops sending commands to firmware alone, this will occur only if configured h/w events
do not occur during WD period"

For my case, the watchdog should get resetted because of the incoming cyclic data from the master, AND sending commands from A8 to PRU.

I don't understand why the watchdog still expires. Can anybody help me, or give me a hint what I can double check?

Please ask for additional required information of my environment.

  • The RTOS team have been notified. They will respond here.
  • Thomas, could you please help me to understand better what is your expected behavior, apologize my ignorance, but if I understand it better, I would be able to reproduce it.

    From your message, I understand you Deactivate/Toggle "free run" mode in TwinCAT, and after 100 ms (PD watchdog timeout) slave goes back to SAFEOP. Is this incorrect? or, should slave stays in OP?

    You also mentioned you modified Beckhoff stack to enable "ESC_SM_WD_SUPPORTED". I guess if you disable it, then you won't see a Watchdog timeout, nor an error in Twincat. Is my understanding correct?

    Thank you,
    Paula
  • Hello Paula,

    thank you very much for jumping in. Sorry when my explanation was not clear, I tried to pack any useful information in it, which could help to identify the problem.

    My expected behaviour of the TMDSICE3359 device as Ethercat slave is: I want to run synchron to the SM2 event (of course in OP state).

    The actual behaviour is: After setting the slave into OP state, after 100ms it went back to SAVEOP with the error "abnormal state change (from 'OP' to 'SAFEOP') with code 0x1b, 'Sync manager watchdog'.

    I set "ESC_SM_WD_SUPPORTED" to true, because of the description:

    "Define: ESC_SM_WD_SUPPORTED

    This switch should be set if the SyncManger watchdog provided by the ESC should be used. If reset the process data watchdog is triggered by a local timer"

    I thought the PRU is capable of this, because the PRU ICSS EtherCAT firmware API guide ( at "bsp_set_pdi_wd_trigger_mode")

    says: "Description Configure PDI WD trigger mode, PDI WD is triggered automatically by h/w on RX_SOF(port0/port1), latch0/1 input high, SYNC0/1 out high. PDI WD is also triggered whenever host sends a command to firmware. PDI WD may not expire if host stops sending commands to firmware alone, this will occur only if configured h/w events do not occur during WD period"

    Is is not correct, or is it another watchdog, compared to my problem?? I will try your suggestion, but just disabling the watchdog is not an option. My slave has to detect, when the master breaks and does not send cyclic process data anymore.

    Best regards,

    Thomas

  • Thomas, I got it now. I will ask our developers and come back to you soon
    thank you,

    Paula
  • Hi Thomas, could you please share with us wireshark logs with accurate timestaps when the issue occurs?
    thank you,
    Paula
  • Also, Thomas, have you tried with another Master different to TwinCAT?

    thank you,
    Paula
  • Hi Paula,

    I attached a Wireshark capture which contains the state transition of the ethercat slave from OP to SAFEOP, when I disable the free run mode of TwinCAT master.

    Frame No 6611 (3.493058 s), you can see the Al status register of the slave contains SAFEOP.

    When I look into the trace, I got a feeling that maybe the master could be an issue.

    I did not test it with another master yet. I will do that as next step, however could you give me an suggestion which master should I try next? 

    PDI_Watchdog.zip

  • Hi Thomas, thank you for the wireshark, we will analyze it and, we will also work in reproducing the issue with TwinCAT. About another master, I typically use acontis, as it is a more commercial master, and we have it working for our AM335x and AM572x industrial. EC-Master runs in both CPSW, PRU-ICSS Eth ports

    Acontis TI download webpage: http://www.acontis.com/eng/products/downloads/ethercat-for-ti-processors.html (just FYI, they also have a newer version v3.0, soon to be published, in case you want to request it to them).

    You can also try Igh or SOEM

    Thank you,

    Paula

  • Hi Thomas, I was able to reproduce it with TwinCAT. We will analyze it and come back to you. In the meantime, if you can check with another Master it would be helpful.

    thank you,
    Paula
  • Hi Thomas, From our Wireshark captures we see, that the next frame, after disabling FreeRun occurs after ~160ms. It seems TwinCAT master doesn’t send any process data access frames when the FreeRun mode is disabled. As expected, after 100ms (timeout default) WD expires. This behavior seems correct to us.

    Thank you,

    Paula

     

  • Hi Paula,

    I installed EC-Engineer from Acontis, and as far as I can see it does not suffer under the same problem.

    I'm not an expert neither I have deep knowledge of the Acontis Ethercat network configuration software, but it seems that there is no error when I switch to OP with synchron to SM2 event. (Which should be the same as leaving FreeRun-mode in TwinCAT).

    I also observed in my wireshark-logs the missing frames after leaving FreeRun-mode, so in other words the TI PRU Ethercat Slave firmware works as expected.

    So this problem is resolved, however I kindly ask you if you could ask your developers if this is a general problem with TwinCAT or an configuration issue? I know, you don't develop TwinCAT but TI use it as reference to test the example application ().

    Thank you for your feedback, patience and effort so far!

    Best regards,

    Thomas

  • Hi Thomas, we think is related to TwinCAT. Other master should be OK.

    thanks for confirming, I will close this thread.
    Paula