AM2432: Profinet cyclic write problem after a reboot with S2 redundancy

Part Number: AM2432

Hi

I am using AM2432 with the Industrial SDK 11.00.00.08.

Our product has been officially certified. However, we are experiencing some unusual behavior that may be related to S2 redundancy.


The configuration we are using for testing is a ring network with:
- a redundant S7-1500 PLC (192.168.10.10 and 192.168.10.12)
- two Scalance switches (192.168.10.51 and 192.168.10.52)
- an ET200SP (192.168.10.15)
- one of our devices (192.168.10.16)

immagine.png


We noticed that after starting communication normally, if we power off and then power on our device communication resumes correctly upon restart except for cyclic writes (from PLC to our device) and TIA Portal reports an error on our device, while cyclic reads, acyclic reads and writes work correctly.

We’ve noticed that this may depend on Update Time, because a 512 ms time seems to work without issues most of the time, while shorter times always cause problems. However, we think it isn’t a Watchdog Time issue but depends only on Update Time, because:

256 ms (Update Time) * 7 (Number of accepted IO cycles without update) = 1.8s doesn't work
512 ms (Update Time) * 3 (Number of accepted IO cycles without update) = 1.5s works


I've attached the Wireshark traces of the two Scalance switches filtering only traffic between PLC and our module when it works and when it doesn’t (there is a time difference of a few seconds between the two traces) and the serial logs from the devices. What seems to be happening is that after the reboot, the write module (PLC -> device) never recover to status Good.

What could be causing this? What can we do to investigate this issue further? 

Thank you,

Best Regards,

Andrea

Profinet.zip 

  • Hi,

    are there any updates?
    I add that with a single PLC without redundancy MRP and S2, the problem doesn't occur, so I think it's specifically related to redundancy.

    Thank you,

    Best Regards,

    Andrea

  • Hi,

    are there any updates?
    Is there any information that might be helpful?

    Thank you,

    Best Regards,

    Andrea

  • Hi Andrea,

    by "except for cyclic writes" , are you referring to L3 records write requests or L2 PPMs?

    Can you help pointing out this faulty behavior with more details in the shared traces:   
    - What's the expected behavior?
    - What's the faulty behavior of the device?
    - Which parts (lines) of the traces exactly reflect this discrepancy. 

    Regards,
    Assila

  • Hi Assila,

    thank you for your reply.

    I’m referring to the L2 frames, which in Wireshark are labeled as PNIO_PS.
    In the logs Scalance 51 NOK and Scalance 52 NOK (which refer to traffic passing Switch A 192.168.1.52 and Switch B 192.168.1.51), after the PNIO-AL packets are sent, when I look at the PNIO_PS packets from the PLC to my Profinet device I always get IOPS 0x60 bad for module 1 submodule 1:

    While in the logs Scalance 51 OK and Scalance 52 OK, after the PNIO-AL packets are sent, when I look at the PNIO_PS packets from the PLC to my Profinet device I get IOPS 0x80 good for module 1 submodule 1:

    I expect that after my Profinet module is powered back on, the status for this module will also change to good, as shown in the second screenshot, but for all the update times I expect and not just for 512 ms. 

    Let me know if you need more details.

    Thank you,

    Best Regards,

    Andrea

  • Hi Andrea,

    Thanks for the follow-up.

    The  highlighted "IOPS" status is set to bad by controller, and the reason is the submodule plugged in slot 1/subslot 1 which seems to be wrong.

    I checked all module/submodule parametrization sequences from the shared traces and could not see where the mod with ID:0x5 was plugged in slot 1.

    - Where the shared traces captured at the same time from .51 and .52 ports?  
    - Could you trigger a factory reset and re-run the test if you are storing the module/submodule config in the NVRAM?
    - If possible, I would suggest you run same test with the ET200SP if it does support S2, just to confirm if this issue is showing for our device only.

    Regards,
    Assila

  • Hi Assila,

    thank you for your reply.

    The  highlighted "IOPS" status is set to bad by controller, and the reason is the submodule plugged in slot 1/subslot 1 which seems to be wrong.

    I checked all module/submodule parametrization sequences from the shared traces and could not see where the mod with ID:0x5 was plugged in slot 1.

    Could you tell me in which trace and which packet number suggests that the submodule plugged into slot 1/subslot 1 is incorrect and not accepted? Because, in fact, it shouldn't be module 0x5 but module 0x30.

    - Where the shared traces captured at the same time from .51 and .52 ports?  

    The traces were made just few seconds apart, I think a maximum of 4–5 seconds apart (just the time needed to start them on two different PCs).

    - Could you trigger a factory reset and re-run the test if you are storing the module/submodule config in the NVRAM?

    I don't save the latest configuration in flash memory (unlike what I think I saw in the demo). I can also perform a factory reset, but it doesn't delete anything related to the module configuration. In the defaultIoSubmodList I use I only enter DAP, PDEV Interface and PDev ports:

    PN_API_IOD_SubmodListEntry_t defaultIoSubmodList[]
    = {
       //api    slot    subslot     mod_id                  submod_id               in_data out_data    im0_support
       //                                                                           _len    _len
        {0,     0,      1,          PN_CFG_IDD_DAP_MOD_ID,  PN_CFG_IDD_DAP_SUB_ID,  0,      0,          PN_API_IOD_Im0Submodule + //DAP
                                                                                                        PN_API_IOD_Im0Device},
        {0,     0,      0x8000,     PN_CFG_IDD_DAP_MOD_ID,  PN_CFG_IDD_DAP_IF_ID,   0,      0,          PN_API_IOD_Im0Submodule}, //PDEV interface
        {0,     0,      0x8001,     PN_CFG_IDD_DAP_MOD_ID,  PN_CFG_IDD_DAP_P1_ID,   0,      0,          PN_API_IOD_Im0Submodule},
        //PDEV port1
        #if (PN_API_IOD_PDEV_NUM_OF_PORTS >= 2)
        {0,     0,      0x8002,     PN_CFG_IDD_DAP_MOD_ID,  PN_CFG_IDD_DAP_P2_ID,   0,      0,          PN_API_IOD_Im0Submodule},
        //PDEV port2
        #endif
        #if (PN_API_IOD_PDEV_NUM_OF_PORTS >= 3)
        {0,     0,      0x8003,     PN_CFG_IDD_DAP_MOD_ID, PN_CFG_IDD_DAP_P3_ID,    0,      0,          PN_API_eIm0Submodule},  //PDEV port3
        #endif
        #if (PN_API_IOD_PDEV_NUM_OF_PORTS >= 4)
        {0,     0,      0x8004,     PN_CFG_IDD_DAP_MOD_ID, PN_CFG_IDD_DAP_P4_ID,    0,      0,          PN_API_eIm0Submodule},  //PDEV port4
        #endif
        {0,     1,      1,          PN_CFG_IDM_MOD_IO_ID,   PN_CFG_IDS_IO_IF_ID,    0,      0,          (PN_API_IOD_Im0Submodule | //IO subslot interface
                                                                                                        PN_API_IOD_Im0Module)},
        {0,     1,      2,          PN_CFG_IDM_MOD_IO_ID,   PN_CFG_IDS_IO_T4_ID,    4,      4,          PN_API_IOD_Im0Submodule}, //IO subslot
        {0,     1,      3,          PN_CFG_IDM_MOD_IO_ID,   PN_CFG_IDS_IO_T2_ID,    2,      2,          PN_API_IOD_Im0Submodule}  //IO subslot
    };
    

    However, the problematic module is the one with ID 0x30, which is a module with the property FixedInSlots="1" in slot 1, subslot 1. Could that have something to do with it? 

    - If possible, I would suggest you run same test with the ET200SP if it does support S2, just to confirm if this issue is showing for our device only.

    I don't have a trace available, but it seems that ET200SP doesn't have the same problem.

    Thank you,
    Best Regards,
    Andrea
  • Hi,

    are there any updates?
    Is there any information that might be helpful?

    Thank you,

    Best Regards,

    Andrea

  • Hi,

    are there any updates?
    Is there any information that might be helpful?

    Thank you,

    Best Regards,

    Andrea