• State
  • Locked Locked
  • Replies 14 replies
  • Subscribers 33 subscribers
  • Views 293 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM2432: Sync0 drift

sahar schwartz
Prodigy 120 points
Part Number: AM2432


Tool/software:

Hi 

We encountered a scenario in which we see a drift in sync0 interrupt.

The setup using Omron controller NX701-1700 and 100 various models motion drives. ether cat cycle used is 3ms.

Our FW uses communication SDK 09.01.00.03 and we are runing the Ethercat on ICSSG0 with 2Mhz clock.

We are taking a timestamp inside the Sync0ISR, and in one case we are seeing 4 consecutive signals arrives in around 40 microseconds delay each, which causes a drift in the signal.

Interesting part is that this phenomenon happens every 37 signals. (every 38, 39, 40, 41 signal will be with delay). 

Any ideas what can cause this behavior?

  • 0
    Prodigy 120 points

    every 32 signals*

    3037.023
    3041.1
    3042.406
    3042.698
    3003.589 1
    2999.38 2
    3000.048 3
    3000.15 4
    2999.923 5
    2999.711 6
    2999.92 7
    3000.389 8
    2999.525 9
    3000.56 10
    3000.163 11
    3000.2 12
    2999.393 13
    2999.76 14
    3000.071 15
    3000.236 16
    2999.773 17
    3000.051 18
    2999.761 19
    3000.758 20
    2999.535 21
    2999.819 22
    3000.096 23
    2999.88 24
    3000.061 25
    3000.014 26
    3000.226 27
    2999.703 28
    2999.976 29
    3000.18 30
    2999.628 31
    2999.988 32
    3039.881 33
    3041.959 34
    3042.293 35
    3042.263 36
    3000.654 1
    3000.088 2
    2999.94 3
    2999.834 4
    3000.23 5
    3000.07 6
    3001.063 7
    2998.804 8
    3000.744 9
    2999.078 10
    3000.543 11
    2999.87 12
    2999.704 13
    2999.928 14
    2999.855 15
    3000.221 16
    2999.991 17
    3000.096 18
    2999.825 19
    3000.249 20
    2999.823 21
    3000.075 22
    2999.9 23
    2999.7 24
    3001.269 25
    2999.508 26
    2999.473 27
    3000.28 28
    2999.745 29
    3000.081 30
    3000.021 31
    2999.661 32
    3041.658 33
    3042.686 34
    3042.064 35
    3040.364 36
    3000.253 1
    2999.935 2
    2999.801 3
    3000.223 4
    2999.851 5
    3000.334 6
    2999.764 7
    2999.936 8
    3000.196 9
    2999.704 10
    2999.915 11
    2999.949 12
    3000.076 13
    3000.24 14
    2999.721 15
    3000.115 16
    2999.848 17
    3000.141 18
    2999.991 19
    2999.904 20
    3000.176 21
    3000.199 22
    2999.68 23
    3000.181 24
    2999.819 25
    2999.944 26
    2999.9 27
    3000.076 28
    2999.91 29
    2999.831 30
    3000.136 31
    3001.345 32
    3042.335 33
    3041.924 34
    3042.358 35
    3039.258 36
    2999.661 1
    3000.189 2
    2999.954 3
    2999.918 4
    2999.896 5
    3000.015 6
    3000.145 7
    3000.09 8
    2999.936 9
    2999.981 10
    2999.816 11
    3000.248 12
    2999.673 13
    3000.845 14
    2999.105 15
    2999.996 16
    2999.954 17
    3000.915 18
    2999.241 19
    2999.76 20
    3000.194 21
    2999.781 22
    3000.255 23
    2999.993 24
    2999.845 25
    2999.92 26
    3000.043 27
    2999.993 28
    3000.133 29
    2999.776 30
    2999.989 31
    3003.108 32
    3042.48 33
    3042.476 34
    3042.828 35
    3036.476 36
  • 0
    TI__Genius 9975 points

    Hi Sahar,

    sahar schwartz said:
    Our FW uses communication SDK 09.01.00.03 and we are runing the Ethercat on ICSSG0 with 2Mhz clock.

    I believe you mean 200MHz ?

    sahar schwartz said:
    Any ideas what can cause this behavior?

    We’ll be needing more details on the configuration and EtherCAT communication wireshark logs.

    Can you provide the complete ICSS Memory Dump (from 0x30000000 to 0x30040000) along with the EtherCAT communication wireshark logs when this issue is observed?

    Regards,
    Aaron

  • 0 in reply to Aaron Thomas
    Intellectual 880 points

    Hi Aaron,

    This issue happens quite rare, once around one month I think. We can share the memory dump when it happens again.

    But for wireshark logs, is it ok to share normal ones? As this drive is in the middle the connection, and rest drives, some of them with same firmware, operated normally.

  • 0 in reply to Jianyu Wang
    TI__Genius 9975 points

    Hi Jianya,

    Jianyu Wang said:
    We can share the memory dump when it happens again.

    Yes that will be helpful for understanding the ICSS register corruption or unintended behavior.

    Jianyu Wang said:
    But for wireshark logs, is it ok to share normal ones? As this drive is in the middle the connection, and rest drives, some of them with same firmware, operated normally.

    The normal should be fine. At least I can get an idea of the EtherCAT frame transmission and order of datagrams and operations the MainDevice is using to communicate with the SubDevices.

    Regards,
    Aaron

  • 0 in reply to Aaron Thomas
    Intellectual 880 points

    We observed drift on node 45 and 93. 

    Normal_106nodes.zip

  • 0 in reply to Jianyu Wang
    TI__Genius 9975 points

    Thank you Jianyu for the logs. We are also reviewing the EtherCAT firmware and checking on possible areas where this issue can occur. Will provide with a patch once ready.

    Regards,
    Aaron

  • 0 in reply to Aaron Thomas
    Prodigy 120 points

    Hi Aaron.

    any update?

    Thank you,

    Sahar

  • 0 in reply to sahar schwartz
    TI__Genius 9975 points

    Hi Sahar,

    We suspect the issue may be due to the i2433 Errata - ICSS: Reading the 64-bit IEP timer does not have a lock MSW logic when LSW is read, causing the issue. On reviewing the firmware, we do see a potential space where the workaround needs to be applied and checked further.

    We are also trying to reproduce the issue in our side with a lower cycle time and with some debug code within the firmware. We will update when an irregularity in the SYNC0 timing is observed.

    I would also recommend you to try with the latest EtherCAT firmware available in INDUSTRIAL-COMMUNICATIONS-SDK-AM243X and provide the ICSS memory dump if possible for the existing setup (assuming it's been a month since the first drift was observed).

    Regards,
    Aaron

  • 0 in reply to Aaron Thomas
    Intellectual 270 points

    Hi Aaron, is there any news regarding this issue ? what have you found in your tests ?

  • 0 in reply to Maor Shalom
    TI__Genius 9975 points

    Hi Maor,

    Please test with the EtherCAT firmware we have shared with Sahar over email and let us know if the issue still persists.

    Regards,
    Aaron

  • 0 in reply to Aaron Thomas
    Intellectual 270 points

    please send it also to me since Sahar is out of office for the near future.

    maor.shalom@servotronix.com

    in addition, what did you test discovered?

  • 0 in reply to Maor Shalom
    TI__Genius 9975 points

    Hi Maor,

    Maor Shalom said:
    in addition, what did you test discovered?

    We were unable to reproduce this issue; however, during code review, we identified and implemented a firmware workaround for a hardware timer errata related issue.

    Regards,
    Aaron

  • 0 in reply to Aaron Thomas
    Intellectual 270 points

    is this workaround in the FW you sent Sahar ? or is this something new?

  • 0 in reply to Maor Shalom
    TI__Genius 9975 points
    Maor Shalom said:
    is this workaround in the FW you sent Sahar ?

    Yes, the workaround has been implemented in the firmware (version 0x053F) that I had provided with Sahar.

    Regards,
    Aaron