• State Resolved
  • Locked Locked
  • Replies 29 replies
  • Subscribers 100 subscribers
  • Views 3568 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Strange behaviour of I2C-Module

becktho
Prodigy 205 points

Other Parts Discussed in Thread: AM3354

I'm trying to solve a problem in I2C communication.

We use three I2C interfaces on an AM3354 with our custom baremetal software. I discovered a strange behaviour on one of the three buses, where a write/read transaction is performed every 50ms (as master).

It generally works without problems, but very sporadic a kind of "clockstretch" occurs, where the SCL line is held low, after writing a value to the slave. The following image shows this behaviour:

Investigating this behaviour, I saw that always the same interrupts are handled as expected, so there is no erroneous sequence of interrupts. The last interrupt handled is always XDR (IRQ_STATUS-register reads 0x0000 4000). While digging a little deeper, I found out that DCOUNT of I2C1_CNT-register and TXSTAT of I2C1_BUFSTAT-register is not 0, although the byte was written to the bus (0x50, as visible in the image above).

The following image shows all registers at the time of the strange "clockstretch":

Generally I could issue a bus reset (which actually is what I'm doing after a certain timeout), but as there are devices connected to the bus, which do not fulfill the I2C spec regarding the bus reset and might hang after a reset, I have to find the cause for this behaviour.


So what could cause this effect? For me it seems as if writing I2C1_DATA does not cause I2C1_CNT and I2C1_BUFSTAT to update.

Any influence of a slave device, which might hold the SCL low, can be excluded, as measurements on our custom hardware clearly showed that the AM3354 holds SCL low.

If you need additional information, let me know.

Many thanks in advance

Thomas

  • 0
    Prodigy 205 points

    These are the pictures, which got lost while posting my question:

  • 0 in reply to becktho
    TI__Guru**** 393215 points

    Hi,

    A similar issue has been reported here a long time ago: https://e2e.ti.com/support/arm/sitara_arm/f/791/t/162236 I suggest you check the I2C waveforms carefully, especially whether ACK signal is actually received on the processor side.

  • 0 in reply to Biser Gatchev-XID
    Prodigy 205 points

    Hi Biser

    As you can see in the screenshot of the oscilloscope, the slave is addressed correctly and the databyte (0x50) is acknowledged. Every time the SCL gets stuck it is the same. Address the slave (a TPS65910A3 by the way), write the databyte to address the Register to be read from the slave and then it's over.

    When I halt the Debugger after a while, I see that DCOUNT=1 and TXSTAT=1. Why that?

    Regards

    Thomas

  • 0 in reply to becktho
    TI__Guru**** 393215 points

    There is a section in the AM335X TRM Rev. L: 21.3.16 I2C Behavior During Emulation. Have you checked it?

  • 0 in reply to Biser Gatchev-XID
    Prodigy 205 points
    Yes, I have.

    SYSTEST.FREE = 0 and the suspend support is not activated for all I2C modules.

    Regards
    Thomas
  • 0 in reply to becktho
    Prodigy 205 points
    Hi Biser

    Are there any more things you can come up with? Or did you pass my problem on to someone else? Or is this it?
    It's a little frustrating to just get answers like "Look at the TRM" and then nothing happens anymore...

    Regards
    Thomas
  • 0 in reply to becktho
    TI__Guru**** 393215 points

    I have asked the factory team for support on this. Feedback will be posted here when available.

  • 0 in reply to Biser Gatchev-XID
    Prodigy 205 points
    Hi Biser

    Would have been cool, if you had let me know two days ago.
    Anyway - thanks the support so far.

    Regards
    Thomas
  • 0 in reply to becktho
    TI__Guru**** 393215 points

    I didn't because normally they respond within 24h. I have sent them a reminder just now.

  • 0 in reply to Biser Gatchev-XID
    TI__Genius 15510 points
    We're looking at this and will post our findings when we know something.Steve K.
  • 0 in reply to Steve Kipisz
    Prodigy 205 points

    Hi Steve


    Two weeks have passed now - could you provide some information about the current progress?

    Regards

    Thomas

  • 0 in reply to becktho
    TI__Guru* 95361 points

    Thomas, are you saying you don't expect the clock stretching?  This is a normal part of of the I2C protocol, and a way for a slave device to "hold off" the master when it is attempting to process incoming data.  It occurs after data transmission and before the ACK stage. Sorry,  I'm not able to view the images you posted. 

    Regards,

    James

  • 0 in reply to JJD
    Prodigy 205 points

    Hi James

    I know that part of the I2C protocol and I - as you say - don't expect a clock stretch. You now may say, that there might be some processing on the slave - ok, valid objection, but in this case, it must be an error on the TI power manager we are using.

    As described above, it works as expected and out of a sudden - "boom" - I2C is until I reset the bus after 100ms. To be honest, if the slave causing this problem was a sensor, I could imagine a hold off to process data. But addressing a register to be read later on I would expect to be done immediately - and not blocking the bus for over 100ms (even if I increase the timeout to over 5s, the bus is still blocked).

    In addition, we are sure that the AM3354 blocks the bus, as we modified the pull resistors on our Hardware to see which device holds the clk line low.

    Regards

    Thomas

  • 0 in reply to becktho
    TI__Guru* 95361 points
    Hi Thomas,
    Ok, if you have exonerated the slave device, it seems that the I2C controller is getting in some weird state, especially if you see the data on the bus when DCOUNT is indicating more data should be transmitted.

    What is the FIFO depth you are using? And you using DMA? Does changing some of these parameters, or even changing the I2C clock frequency show any change in behavior?

    Unfortunately, to debug this I think we will have to try to strip down your driver to possibly see where the issue is.

    Another idea i had is that if some of your slave devices can't handle the reset, is it possible to use the SYSTEST register to sort of isolate the I2C signals temporarily until you reset the controller. With the SYSTEST register, you have manual control of the signals, which will allow you to maybe gracefully back out of the stuck situation so your slave devices can handle it.

    Regards,
    James
  • 0 in reply to JJD
    Prodigy 205 points

    Hi James


    The FIFO depth is 32Bytes and the FIFO thresholds are set to 24Bytes (RXTRSH and TXTRSH). We do not use DMA.

    Changing these parameters and/or clock frequency I first must give a try. I let you know the results as soon as I got them...

    Could there be a problem with other interrupts? We have 10 active interrupt (3x Timer, 3x I2C, 3x UART, 1x CAN) configuration and it seems as there maybe is a relation to the timer interrupts. Once I disabled the timer interrupts and then it seemed to run more stable (meaning no blocked bus in a time duration where multiple blocks occured with timer interrupts on).


    Regards

    Thomas

  • 0 in reply to becktho
    TI__Guru*** 125430 points
    I've had a quick look through the waveforms and the register screen shot. The state of the bus looks to be consistent with what you would see if I2C_CON[STP] were never set to 1 at the beginning of the transfer. In other words, if you set STT=STP=1 at the start of the transfer then when the counter gets to zero it automatically sends the stop bit and terminates the transaction. However, if for some reason you did NOT set STP=1 at the start of the transfer then at the end of the transfer the I2C controller would simply hold SCL low (like you are observing).

    So along those lines, once you get into this state can you try setting the I2C_CON[STP] bit through the register window? In your earlier screen shot you showed a value of I2C_CON = 0x8600, so my thought would be to simply poke I2C_CON=0x8602 into the register to see if it then terminates the sequence. That might at least give a bit more credence to my theory...

    And if indeed poking the STP bit terminates the transfer then I think the next order of business will be to determine if/how you end up with a transfer where you have not set the STP bit. I imagine that will involve some inspection and/or instrumentation of the code or something along those lines.
  • 0 in reply to Brad Griffis
    Prodigy 205 points

    Hi Brad and James


    I recently checked the following things:

    - Run application as usual: Multiple blocks of the bus within less than 1h

    - Run application without timer interrupts: No block of the bus within 3h

    - Run application with 50kHz I2C clock frequency: No block of the bus within 3h

    - Set STP bit, as soon as bus is blocked: This worked - after setting the bit and continuing execution of the application, the I2C communication continued successfully.


    In our driver we handle composite transactions, where after a write transaction is done, we continue with a read transaction (without setting STP bit). After completion of the read transaction the STP bit is set and therefore the bus released.

    Is this a fix requirement, that STP must be set as soon as STT is set?

  • 0 in reply to becktho
    TI__Guru*** 125430 points

    You can set the STP bit at the end of the transaction. That's fine too. My point was mainly that it appears it is not being set in the case of a failure.

  • 0 in reply to Brad Griffis
    Prodigy 205 points

    I once tracked the values of the IRQSTATUS register, where the following (repeating) sequence is visible:

    0x0000 0004

    0x0000 4000

    0x0000 1004

    0x0000 3000

    0x0000 0004

    As you can see in the screen shot of the registers above, the IRQSTATUS register has the value 0x0000 1000. So somehow it seems, as if the ARDY bit does not get set.

    In our driver we expect this bit to be set, as we then terminate the current transaction (in this specific case it would be the communication visible in the oscilloscope screen shot). After that the second transaction would be performed, where data is read from the slave and STP set afterwards.

  • 0 in reply to becktho
    TI__Guru*** 125430 points

    When you clear IRQSTATUS bits are you sure you're not potentially clearing out an ARDY event by mistake?  For example, you don't ever write IRQSTATUS_RAW = 0xFFFF do you?  My thought here is that potentially an ARDY event occurred while you're inside the ISR and it was mistakenly cleared out before it was serviced.

  • 0 in reply to Brad Griffis
    Prodigy 205 points

    I just double checked it. There is no additional clearing of ARDY, except the line where I really want to reset it.

    In addition, if this was the case, the whole communication should never run (or at least it should be clearly reproducable), but as the block of the I2C occurs randomly I think there must be something else.

  • 0 in reply to becktho
    TI__Guru*** 125430 points
    I had another customer experiencing a similar issue a few years ago. The problem ended up being that they were programming the CNT register after the CON register. I don't suppose you happen to be doing the same thing? You need to program the count before starting the transfer. Their symptoms were pretty much identical to yours.

    What OS, etc. are you using? Am I able to see the code (e.g. is it based off something from TI)?
  • 0 in reply to Brad Griffis
    Prodigy 205 points

    There is no OS in use, as we run a baremetal application with our own scheduler.


    The order of writing the registers might be an issue. I just checked in the code, where we write these registers. So is it not allowed to write any field of CON after CNT or just to set STT? The action we perform on these registers could be simplified to the following sequence:

    1. Configure CON (e.g. clear XOA0, clear XSA, set MST)

    2. Set the CNT register to the number of bytes to transfer/receive

    3. Start (set STT in CON register)

    So is this wrong or not allowed?


     

  • 0 in reply to becktho
    TI__Guru*** 125430 points
    That looks fine. The I2C_CNT register mentions the following, "Note that DCOUNT must not be reconfigured after I2C_CON.STT was enabled and before ARDY is received." That's what I wanted to make sure was not the issue. It looks like you're programming CNT before setting STT which is correct.
  • 0 in reply to Brad Griffis
    Prodigy 205 points
    Ok - we do not change I2C_CNT after setting STT, except for a repeated start. In this case we have twice the sequence of my previous post without STP in between. But if my problem occurs, we have not yet reached the code, which modifies these registers.
    Other ideas?
  • 0 in reply to becktho
    TI__Guru*** 125430 points
    Can you share any of your code?
  • 0 in reply to Brad Griffis
    Prodigy 205 points
    Can I send you an e-Mail with the code?
  • 0 in reply to Brad Griffis
    TI__Guru*** 125430 points
    I know I already asked you about interrupts, but one other thing... Are you certain your code is doing a straight-up WRITE to the IRQSTATUS register to clear an interrupt? In other words, can you please verify you're *not* doing a read-modify-write of that register? A read-modify-write would result in all bits getting cleared. There are subtle things like this that can be observed directly in the code, but I'm just blindly guessing otherwise.
  • 0 in reply to Brad Griffis
    Prodigy 205 points
    For all who might have a similar problem, mine was cause by a wrong access of the IRQSTATUS register.
    Ensure that accessing the IRQSTATUS register is not done with read-modify-write ("|=") but with "=" only.
    Thanks to Brad who found this coding error.