• State Resolved
  • Locked Locked
  • Replies 17 replies
  • Answers 1 answer
  • Subscribers 82 subscribers
  • Views 1023 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TMS320F28386D: CM-I2C config as slave holds the SDA line low after glitches in SCL. How to recover?

Agustin Jesus Gomez
Prodigy 216 points

Part Number: TMS320F28386D

Tool/software:

Hello,

I am working with the TMS320F28386D (referred to as "DSP" below), which is configured as an I2C slave on a shared bus. The Connectivity Manager (CM) receives commands via I2C (CM-I2C), while the C28x cores are running a control loop for an inverter (PWM).

Issue:

When the inverter is running, noise is introduced into the environment. This noise sometimes couples onto the I2C bus. While the noise levels are not very high, if a noise spike coincides with the rising edge of the clock (SCL), it can create false clock edges. These false edges are occasionally recognized as erroneous packets, which my software detects and reports to the master, allowing normal operation to continue.

However, sometimes the I2C peripheral in the CM becomes stuck, holding the SDA line low. When this happens, the system cannot recover without resetting the DSP.

My question:
How can I recover the I2C peripheral from this stuck state without resetting the device?

What I have tried:

  • Sending 9 clock pulses on SCL (as is kind of standard for releasing stuck slaves) did not recover the bus.

  • Using the debugger (XDS110 and CCS), I toggled the Master Enabled and Slave Enabled bits in the I2C peripheral, but this did not release the lines.

  • Changing the GPIO Mux configurations for pins 104 and 105 to their default function released the SDA line, but switching them back to I2C-CM caused SDA to be held low again, confirming the peripheral is the source of the problem.

  • Enabling the Master mode on the I2C peripheral and setting the anti-glitch filter to 8 clocks appeared to prevent the issue in my current tests, but I am concerned this is not a robust solution if the environment get noisier in the final product.

Enabling Master mode allows me to monitor the busy bit or the status of SDA/SCL and implement a watchdog as a workaround, but don't know how to reset the peripheral if the supposed watchdog triggers.

Attached is a logic analyzer trace:

  • The blue rectangle highlights a zoomed-in area with the glitch.

  • The top trace is the digital capture; the bottom is the analog capture.

  • After the glitch, SDA remains stuck low. Multiple subsequent clock edges do not cause the CM (slave) to release the SDA line.

Thanks in Advance

Let me know if you need more information or measurements.

Kind Regards

Agustin J. Gomez

  • 0
    TI__Mastermind 20695 points

    HI Agustin,

    Let me look into this further and get back to you tomorrow. In the meantime, please refer to similar threads in the forum on this topic for assistance. 

    Best Regards,

    Aishwarya

  • 0 in reply to Aishwarya Rajesh
    Prodigy 216 points

    Hi Aishwarya,

    I’ve already checked multiple threads discussing similar issues, but none quite match my specific case.

    Key difference in my setup:
    The CM-I2C is configured in slave mode and is actively holding the SDA line low itself.

    For example, in this thread: TMS320F28386D: I2C bus hangs(SDA hangs) - C2000 microcontrollers forum - C2000Tm︎ microcontrollers - TI E2E support forums

    In that case, the DSP acted as a master, and slave devices were holding the SDA line (as I understand it). The suggested solution of toggling the SCL line nine times didn’t resolve my issue when I attempted it, as mentioned in my initial message.

    This is another thread I checked: TMS320F280025: F280025 I2C slave SDA suddenly pull low, and then SCL pull high during Host send the command to F280025 I2C slave? - C2000 microcontrollers forum - C2000Tm︎ microcontrollers - TI E2E support forums

    In this thread, the problem is similar because the DSP is configured as a slave. However, it involves a C28x I2C (not the CM-I2C), and the solution was discussed privately.

    I haven’t found any thread addressing this exact scenario. At this point, I suspect it might be a specific error in the CM-I2C hardware (errata) – though I hope that’s not the case.

    Best regards,

    Agustin J. Gomez

  • 0
    TI__Mastermind 20695 points

    Agustin,

    Doing an I2C module soft reset (resetting only the peripheral not the device) is typically the best solution to recover the bus. Have you tried this instead of resetting the entire device?

    Have you also already looked into manually sending a stop condition when this glitch occurs to end the transaction and then utilize the logic you mentioned earlier to end the operation and report to the controller to take care of it accordingly? Technically, when the SDA goes from low to high and SCL is high that is a stop condition. The bus is considered free after a stop condition. Not exactly what is going on here, but a suggestion to try.

    I think in general you should look into adding an I2C glitch filter as part of the hardware setup to protect the circuit from the external noise. 

    Agustin Jesus Gomez said:
    At this point, I suspect it might be a specific error in the CM-I2C hardware (errata) – though I hope that’s not the case.

    Which error in the errata are you referring to? 

    Best Regards,

    Aishwarya

  • 0 in reply to Aishwarya Rajesh
    Prodigy 216 points

    Hi Aishwarya,

    Aishwarya Rajesh said:
    Doing an I2C module soft reset (resetting only the peripheral not the device) is typically the best solution to recover the bus

    Please tell me how to reset the peripheral. I don't find the correct bit for that in the CM-I2C.

    Aishwarya Rajesh said:
    Have you also already looked into manually sending a stop condition when this glitch occurs to end the transaction

    In that state, with the peripheral already locked, I toggled the bits for controlling the ACK bit, that is, Register I2CSACKCTL, bits ACKOVAL and ACKOEN, and didn't change the state of the peripheral. 

    Aishwarya Rajesh said:
    I think in general you should look into adding an I2C glitch filter as part of the hardware setup to protect the circuit from the external noise. 

    Thank you, we will check that. We already think in adding some capacitors if necesary. As said, the internal anti-glitch filter of the master really helped, but want to be ready for the case the peripheral dies again.

    So is my workaround, so it is clear:

        // Configure as master to use the glitch filter to protect the SCL/SDA lines
    // and avoid I2C peripheral to get stuck
    // The master functionality itself is not used.
    I2C_enableMaster(I2C0_BASE);
    I2C_initMaster(I2C0_BASE,I2C_CLK_FREQ,false);
    I2C_setSlaveAddress(I2C0_BASE,slaveAddress,I2C_MASTER_WRITE);
    I2C_configureMasterGlitchFilter(I2C0_BASE, I2C_MASTER_GLITCH_FILTER_8);
    // Configure as slave - actual use-case:
    I2C_registerInt(INT_I2C0, I2CSlaveISR);
    I2C_enableSlave(I2C0_BASE);
    I2C_setOwnSlaveAddress(I2C0_BASE, I2C_SLAVE_ADDR_PRIMARY, slaveAddress);

    Aishwarya Rajesh said:
    Which error in the errata are you referring to? 

    I meant, a new error that still not included there. But again, I hope it's not.

    Summarizing:

    How do I soft-reset the CM-I2C?

    Thank you again.

    Kind Regards

    Agustin

  • 0
    TI__Mastermind 20695 points

    Hi Agustin,

    Apologies for the delay, please give me another day to get back to you.

    Best Regards,

    Aishwarya

  • 0 in reply to Aishwarya Rajesh
    Prodigy 216 points

    Hi Rajesh,

    no problem, it is not urgent but important.

    When you have a solution or workaround, please tell me.

    Thank you.

    Kind Regards,

    Agustin

  • 0
    TI__Mastermind 20695 points

    Agustin,

    In terms of hardware, you could look into adding series resistors on both the SDA and SCL lines to help filter out noise in addition to the series resistors. 

    In terms of software, you could implement a software watchdog that monitors the I2C peripheral status. This is also a related thread on checking the CM I2C BB: TMS320F28388D: CM I2C Reading Bad Data. 

    Best Regards,

    Aishwarya

  • 0 in reply to Aishwarya Rajesh
    Prodigy 216 points

    Aishwarya,

    remember the question is how to reset the peripheral, the CM-I2C.

    Don't worry about the hardware, we can handle that, we will add those chokes you recommended me previously.

    I summarize the workaround until now, so you don't get lost:

    My application needs the slave config, but to avoid hangs or recover from them:

    * I enable the master config to use the glitch filter.

    * With the master enabled I can also check the bus status (BB) and if it is busy for a long time...

    Option 1: We soft-reset the CM-I2C (how??)

    Option 2: We reset the whole processor (worst solution)

    Remember the main issue now is how to reset the CM-I2C.

    Thanks in advance

    Regards,

    Agustin

  • 0
    TI__Mastermind 20695 points

    Agustin,

    Have you already tried enabling and disabling the CM I2C using the driverlib functions (I2C_enableSlave, I2C_disableSlave)? This would definitely reset the device. There are also the same functions for enabling/disabling the controller.

    The other information provided above was the logic to help determine when the reset is needed, though it seems you got that figured out.

    Best Regards,

    Aishwarya

  • 0 in reply to Aishwarya Rajesh
    Prodigy 216 points

    Aishwarya, 

    That seems interesting. I toggled the master and slave enable bits in the registers with the debugger and did not help. Now I checked the driverlib functions you mention and they seems different, they toggle also the bit "Device Active" from the slave control status. Let me create a test for that and come back to this issue in the next days. Maybe the end of the week. I'll let you know.

    Thank you again,

    Regards

    Agustin

  • 0 in reply to Agustin Jesus Gomez
    Prodigy 216 points

    Aishwarya,

    I took some time to test it just today. It did not work.

    I removed the glitch filter to reproduce the issue.

    void I2CSlave_config(uint8_t slaveAddress) {
        // Configure as master to use the glitch filter to protect the SCL/SDA lines
        // and avoid I2C peripheral to get stuck
        // The master functionality itself is not used.
        I2C_enableMaster(I2C0_BASE);
        I2C_initMaster(I2C0_BASE,I2C_CLK_FREQ,false);
        I2C_setSlaveAddress(I2C0_BASE,slaveAddress,I2C_MASTER_WRITE);
        // I2C_configureMasterGlitchFilter(I2C0_BASE, I2C_MASTER_GLITCH_FILTER_8);
        // Configure as slave - actual use-case:
        I2C_registerInt(INT_I2C0, I2CSlaveISR);
        I2C_enableSlave(I2C0_BASE);
        I2C_setOwnSlaveAddress(I2C0_BASE, I2C_SLAVE_ADDR_PRIMARY, slaveAddress);
    }

    void I2CSlave_reset() {
        I2C_disableSlave(I2C0_BASE);
        I2C_disableMaster(I2C0_BASE);
        I2C_enableMaster(I2C0_BASE);
        I2C_enableSlave(I2C0_BASE);
    }

    /******************************************/

    // in main.c, in a task each some miliseconds

    volatile bool reset_i2c = false; // toggle with debugger

    void task(void){

    (...)

       if (reset_i2c) {
          reset_i2c = false;
          I2CSlave_reset();
       }

    (...)

    }

    I test it just with the slave config, and also with the master config as you can see in the code.

    Another any recommendation on how to reset it?

    The new thing I noticed from the driverlib: in i2c.h exist registers "_WRITE" with the same address that other registers have but with different bit meaning.

    For example:

    #define I2C_O_MCS_WRITE 0x4U // I2C Master Control/Status
    #define I2C_O_SCSR_WRITE 0x804U // I2C Slave Control/Status

    I did not find them in the TRM. Where are they explained? 

    Kind regards,

    Agustin

  • 0 in reply to Agustin Jesus Gomez
    TI__Mastermind 20695 points

    Agustin,

    That's odd, let me check in with design experts. Thanks for your patience!

    Best Regards,

    Aishwarya

  • 0
    TI__Mastermind 20695 points

    Agustin,

    Apologies for the delay, still waiting on feedback from our other experts. 

    Agustin Jesus Gomez said:
    I2C_enableSlave(I2C0_BASE);

    Apologies for the delay, still waiting on feedback from our other experts. One thing that could help is adding a delay after this (though HW should take care of it). Try also clearing interrupt status/flags. 

    Best Regards,

    Aishwarya

  • 0
    TI__Mastermind 20695 points

    Hi Agustin,

    SYSCTL has the following register that can be used to reset the peripheral. Please give this a try and let me know. 

    Best Regards,

    Aishwarya

  • 0 in reply to Aishwarya Rajesh
    Prodigy 216 points

    Hi Aishwarya,

    Thank you. That seems promising. I'll try to test it  the next days or week, because I'm on another task right now.

    I let you know as soon as I know.

    Thanks again.

    Kind Regards,

    Agustin

  • 0
    TI__Mastermind 20695 points

    Agustin,

    Keep me updated, thanks for your patience.

    Best Regards,

    Aishwarya

  • 0 in reply to Aishwarya Rajesh
    Prodigy 216 points

    Hi Aishwarya,

    I checked quickly and can confirm the reset function works. Thank you!

    For the sake of completeness, I’m leaving the full solution here.

    First, the problem:

    The SDA line gets stuck, which is still an issue with the I2C peripheral on the CM. I believe this should be clearly mentioned in the datasheet or at least in the errata, as it can be quite critical in real-world applications.

    This issue occurs when glitches enter the clock line during communication, as shown in the images in my first question. In my case, the glitches are caused by noise when a power inverter is switching.

    The main problem is that holding the SDA line is not expected behavior from a slave device on the bus. If it does happen, sending extra clocks should resolve it. In a multimaster system, holding the SDA line would be interpreted as an arbitration loss, but that’s not the case here, and the system isn’t prepared for this scenario.

    In a simple master implementation, when the master writes, it reads the bus at the last clock, expecting an ACK/NACK response. If the SDA is held low, it will be interpreted as an ACK, so the master keeps working as if nothing is wrong.

    Prevention:
    Software:
    I saw a huge improvement after enabling the glitch filter and enabling the master in the I2C peripheral. I don’t use the peripheral as a master, but I use its features. In my lab conditions, I couldn’t reproduce the error again with the filter enabled.

    Hardware:
    You recommended adding ferrite beads (1kΩ at 100MHz) in series with the I2C inputs. That should also help prevent noise to enter. Small capacitors can also be added, but they should be analyzed together with the rest of the bus, as you don’t want to increase the bus capacitance too much and slow it down.

    Proposed solution: Watchdog (timer)

    1. If the error happens ONLY when the DSP is addressed:

      • Enable a timer after the first byte interrupt is serviced, configured for the maximum time it can take for the next byte to arrive (this defines a minimum bus frequency).

      • Reset the timer after each new byte arrives.

      • Stop the timer when the stop condition is detected.

      • In the Timer ISR, put the I2C reset function.

    2. If the error happens when the bus is written in general:

      • With the master enabled, the BUSY bit can be checked, and the states of SDA and SCL can be observed.

      • The simple solution is to have a task that counts how long the bus is busy. If the time exceeds what’s needed for the longest packet in the system, try resetting the I2C, in case the peripheral itself is holding the line.

      • This is a rudimentary solution, so I hope the first one is sufficient.

    Testing:
    I still need to test these conditions by turning on the inverter (to generate noise) and writing to other devices on the bus. If the bus doesn’t get stuck, solution 1 can be implemented.
    However, I’m afraid the error could happen during the addressing phase, when every device is listening. In that case, the interrupt might not be triggered and solution 1 wouldn’t work. Maybe both solutions should be implemented in parallel.

    Here are the initialization and reset functions:

    void I2CSlave_config(uint8_t slaveAddress) {
    // Configure as master to use the glitch filter to protect the SCL/SDA lines
    // and avoid I2C peripheral to get stuck
    // The master functionality itself is not used.
    I2C_enableMaster(I2C0_BASE);
    I2C_initMaster(I2C0_BASE,I2C_CLK_FREQ,false);
    self.slaveAddress = slaveAddress;
    I2C_setSlaveAddress(I2C0_BASE,slaveAddress,I2C_MASTER_WRITE);
    // Remove this for testing.
    // I2C_configureMasterGlitchFilter(I2C0_BASE, I2C_MASTER_GLITCH_FILTER_8);
    // Configure as slave - actual use-case:
    I2C_registerInt(INT_I2C0, I2CSlaveISR);
    I2C_enableSlave(I2C0_BASE);
    I2C_setOwnSlaveAddress(I2C0_BASE, I2C_SLAVE_ADDR_PRIMARY, slaveAddress);
    // (...)
}

void I2CSlave_reset() {
    SysCtl_resetPeripheral(SYSCTL_PERIPH_RES_I2C0);
    NOP;
    I2CSlave_config(self.slaveAddress);
}

    I am not sure if the NOP is necessary, I just suspect that after a reset some cycles are necessary.

    Thank you for your help.

    Kind regards,
    Agustin J. Gomez