This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM5748: AM5748 I2C2 bus intermittently hangs

Part Number: AM5748
Other Parts Discussed in Thread: TPD12S015, PMP, DRA752, TFP410, TMP102

Team, 

Can you  help my customer identify the cause of I2C2 issue described below?:

Device AM5748ABZXAS

98P10EQ

941  ABZ

This issue has been seen on multiple AM devices of the same batch.

The AM device is used on a custom module.

All five I2C buses are configured, only I2C2 displays this issues:

   Occasionally I2C will power up with I2C2_SDA stuck low, especially when cold

   Sometimes, when SDA is not stuck low, from a u-boot prompt, attempting to access a device results in the following error:

=> i2c dev 1

Setting bus to 1

=> i2c probe

Valid chip addresses:Timed out in wait_for_event: status=0000

Check if pads/pull-ups of bus are properly configured

Timed out in wait_for_event: status=0000

   (and the above two lines are repeated . . .)

   When all seems to be well, the above probe works fine:

=> i2c dev 1

Setting bus to 1

=> i2c probe

Valid chip addresses: 10 20 48 49 4A 4B 57

   Signal Integrity is good.

   When all seems to be fine, running the following as a script will hang the SDA low after a small number of attempts (1 to 6 or so)

      Script completes:

                => i2c mm 4b 100.2

00000100: 00 ? 00

00000101: 00 ? 00

00000102: 00 ? 00

00000103: 00 ? 00

00000104: 00 ? 00

00000105: 00 ? 00

00000106: 00 ? 00

00000107: 00 ? 00

00000108: 00 ? 00

00000109: 00 ? 00

0000010a: 00 ? 00

0000010b: 00 ? 00

0000010c: 00 ? ^c

=>

      Script fails:

> i2c mm 4b 100.2

00000100: 00 ? 00

00000101: 00 ? 00

00000102: 00 ? 00

00000103: 00 ? 00

00000104: 00 ? 00

00000105: 00 ? 00

00000106: 00 ? 00

00000107:Timed out in wait_for_event: status=1000

Check if pads/pull-ups of bus are properly configured

Error reading the chip: -121

=>

   Here is a scope capture of the failing transaction. It should be a write of 0x0107 to device at 0x96 (4B write) followed by a single data read at device 0x97 (4B read). The write is successful but the read fails as shown.

 

   I have a previous version of our module with the same device at 4B and uses an AM5748 from the same batch. Difference is that the 4B device is on I2C4 on that version. On that previous version, I2C2 was used but had devices at 10, 4A, and 57; I2C4 had devices at 38, 48, 49, and 4B. In this older version we have never seen any of these issues and the script runs flawlessly.

 

   In both the old and the new modules, I2C pullups are 2.21K and I2C buses are run at 3.3V.

    The lead SW person and I (HW lead) have scrubbed the registers and compared the configuration between the various I2C buses and found nothing of any significance.

    Anyone have any ideas what might be happening or ever seen anything like this?

Let me know if additional information is needed.

Thanks!

Viktorija

  • Viktorija,

    Do you know which device is holding SDA low?  For example, if you change the i2c2_sda pinmux to "driver off" that will give you definitive information as to whether the AM57xx is responsible for holding the line low.  Since you said this is a read, I suspect it is the other device holding the line low.  Assuming that changing the AM57xx pin mux does not release the line, you still should try to confirm which device is holding the line.  Is there a "reset" pin for the device on the other side?  It would seem most likely that the device from which you are reading is the device having an issue, but we should try to prove that is really the case.  Another way to do it (more invasive) would be to physically remove the other devices from the board and confirm that you can still recreate the same issue with only a single device on the bus.

    Are you able to post a more zoomed in version of that same picture?  When performing an I2C read, the AM57xx is expected to NACK the last transfer such that the other device knows not to try and send more data.  You can run into these lockup situations if the AM57xx is not NACKing that transaction.  I thought that all happened in hardware, but it would be good to see a zoomed in photo to confirm that the AM57xx is indeed NACKing the byte before the bus hangs.

    Also -- what software is running where you see this issue?  Is it u-boot?  Have you been able to boot all the way to the kernel?  Do you ever experience the same issue in Linux or does it strictly occur when u-boot is running?

    Best regards,
    Brad

  • Hi Brad,

       I'm Ron, the guy with the problem. I will verify which device is holding SDA low but my strong suspicion is it is the salve device at 0x4b. Since the 5748 ACKd the first data beat, the slave device is still offering data but the master is not sending any clocks. I know the data at this location is 0x00 so the SDA will be asserted low. I believe the real issue is why the 5748 is ACKing the first data when the read is a 1-byte transaction.

       Will get back to you on the definitive answer of who is holding the SDA low.

           Many thanks, Ron.

  • HI Brad,

       Confirming that the slave device at 4B is holding SDA low. On asserting the chip reset, SDA releases and the 5748 recovers. So the real issue is why the 5748 is ACKing the data.

       

       Thanks, Ron.

  • Hi Ron,

    You're sure the processor is ACKing that data?  Is that true of all reads, but in cases where the next bit is 1 the bus doesn't hang?  What is the software being used?  It sounds like the ACK is the root cause of the failure so we need to look into the software.

    Best regards,
    Brad

  • HI Brad,

       Here is an expanded view of the failing transaction showing the transaction is NAKd. Ron

  • The picture didn't come through.  If the transaction was NACK'd that is the proper behavior.  Does the processor ACK or NACK just before the bus hang?

  • HI Brad,

       Not sure if you can see the picture or not. Anyway, We are using u-boot. I need to ask the SW person for details. Should I have him zip up the I2C files and can I send them somewhere?

    Thanks, Ron.

  • My bad, transaction is ACKd, the incorrect behavior. Sorry! Ron.

  • Rather than sending me code, I would rather just know the details of the release.  Is this u-boot that came from a TI SDK?  If so, which version?

  • I received your picture.  Thanks.  We already knew the issue, but it's good to have the substantiating evidence.  Here's my markup:

    Please send details on the software so we can understand that better.  Also, what is this specific device you're talking to?  I may need to look at its associated driver.

  • HI Brad,

       Here is what our SW person tells me:

    It was pulled from an SDK. It’s been so long. I thinks it’s one of these two versions.

    ti-processor-sdk-linux-am57xx-evm-04.01.00.06-Linux-x86-Install

    ti-processor-sdk-linux-am57xx-evm-05.01.00.11-Linux-x86-Install

    The u-boot version is U-Boot 2018.03-rc3

    The device at 4B is an AD9545. Don't forget the same device runs OK on a previous version of our module.

    Thanks, Ron

  • It must be the 5.01 version because that's the one on the 2018 branch.  The 4.01 used a u-boot 2017 branch.

    I had a quick look at the I2C driver bundled there.  Surprisingly there have been very few patches between then and now.  I don't see any patches to the TI I2C driver that look similar to this issue.  Can you tell me where the AD9545 comes from, i.e. is the driver already in the u-boot tree?  If not, where did it come from?  Perhaps the issue is in the way the AD9545 is trying to perform the I2C transactions?

    I acknowledge there are a lot of inconsistencies...  If there was an issue in the AD9545 driver, I would not have expected it to work on the other I2C.  We need to keep digging for some new data points until we uncover one that helps lead us to the answer...

    So do you see this same issue on all devices (of the same board revision)?  And can you always reproduce the issue by reading the exact same address?  Have you tried reading that same address on the older hardware revision where a different I2C bus was used?

  • Hi Brad,

       We have no specific driver for the 9545 in u-boot so I suppose it uses the generic driver. The failure is not seen at any definite point in the script. The only consistency is the mode of failure; a read that does not complete as seen above. But it always happens early in the script, after less that twenty or thirty transactions. It smells of a race condition: either the master is asserting ACK instead of NAK or the slave thinks it is a write and is asserting ACK (or just asserting SDA for no good reason). Ideas on how to separate that out? My impression is that the master thinks it has completed the transaction since I can ask it to do another transaction and it complains it cannot but after the SDA release, it can successfully.

       Assuming the master read state machine is well behaved, this error would indicate that the instruction was read wrong and it thinks it is a multi-byte read. is it possible that the memory used for the transaction is being used by something else? I've seen this sort of thing where different processes get allocated overlapping memory. Results are really messy.

       I have asked our SW guy to run my script on his board and see what happens. I know that we do share the issue where occasionally I2C2 cones up with SDA already low and where SDA is not low but transactions fail.

       That's the story so far . . .

    Many thanks, Ron

  • Can you distill your script down to a handful of commands that you can make fail repeatedly?  At that point, can you share it here for review?

  • You and I are heading in the same direction! if I perform "i2c md 4b 100.2 1" one at a time, I can get it to fail after as few as two attempts (usually quite a few more). And I did an interesting thing; I upped the gain on the SDA to hopefully reveal who was performing the "ACK" and, voila, it looks like the 5748. Observe the low level of the address and the low level of the read data. My interpretation would be that the master is asserting the ACK. 

    You can see the assertion of SDA by the master to end the cycle but the slave is asserting low also, causing the difference, and when the master releases, the slave takes over slightly raising the low voltage. Ron

  • Ronald Edgar said:
    You can see the assertion of SDA by the master to end the cycle but the slave is asserting low also, causing the difference, and when the master releases, the slave takes over slightly raising the low voltage.

    You read my mind on verifying that it was the processor that performed the ACK.  I assumed that to be the case, but it's good to check assumptions in these types of situations.

    Has the bus speed changed at all between the working and non-working scenarios?

  • HI Brad,

       No, the bus speed is 400KHz (well, ~384KHz) and that is the same for current and previous modules. BTW, reducing the speed to 100KHz makes no difference to the issue. I have asked our production crew to lend me another module so I can have two at home to play with. Never know!

       A  segway to something else. Could I have a physical issue on this board, like a pad missing and not providing a ground or power or something else? Clearly not the I2C pins but something else? Suggestions? If a state machine is making a mistake SOMETIMES, it might be internal SI if a GND is missing. A28 is a GND and that is in the corner. Corners are vulnerable to not soldering properly. Having said that, we have a very good track record of soldering the AM5748 chips down (we do our own manufacturing), And it looks OK but hard to tell with lead free and no microscope at home. But a thought . . .

    Ron

  • Ronald Edgar said:
     No, the bus speed is 400KHz (well, ~384KHz) and that is the same for current and previous modules. BTW, reducing the speed to 100KHz makes no difference to the issue.

    Hmmm.  Hopefully I'm not making a silly mistake, but looking at your last screenshot, I would not have guessed 400 kHz.  It looked like 50us/division.  Over a span of 2 divisions (100 us) I counted almost exactly 5 clock periods.  I was estimating 20us/clock which corresponds to 50 kHz.  Am I making a mistake in my estimate?  Or perhaps is the actual speed not what you were expecting?

    You said this was I2C2 right?  I'd like to have you read a couple of registers after a failure, but I want to provide you the correct address.

  • Look at trace Z4, the zoomed clock. In the green box below you will see 6.5us/div. 18 clock cycles take 7 and a bit divisions = ~46ns makes ~2.56nS per clock make ~391KHz. The zoom gets me every time too.

    Let me know what regs you want. I'm assuming somewhere in the 0x48072000 group (I2C2) but maybe not.

    Ron.

  • . . . of course I mean uS, not nS, sorry!

  • Possibly jumping the gun but here is what I think you might be interested in.

    => i2c md 4b 100.2 1
    Timed out in wait_for_event: status=1000
    Check if pads/pull-ups of bus are properly configured
    Error reading the chip: -121
    => md.w 0x48072000 0x80
    48072000: 080c 0000 5040 0000 0000 0000 0000 0000 ....@P..........
    48072010: 0001 0000 0000 0000 0000 0000 0000 0000 ................
    48072020: 0000 0000 1000 0000 1000 0000 0000 0000 ................
    48072030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    48072040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    48072050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    48072060: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    48072070: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    48072080: 0000 0000 0000 0000 1000 0000 0000 0000 ................
    48072090: 0001 0000 0000 0000 0000 0000 0000 0000 ................
    480720a0: 0000 0000 8402 0000 0000 0000 004b 0000 ............K...
    480720b0: 0009 0000 0009 0000 0003 0000 01a0 0000 ................
    480720c0: 4000 0000 0000 0000 0000 0000 0000 0000 .@..............
    480720d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    480720e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    480720f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    => md.w 0x48073000 0x80
    48073000: 6439 0020 0000 0000 0000 0000 0000 0000 9d .............
    48073010: 0000 0000 0000 0000 0001 0600 5449 0000 ............IT..
    48073020: 0200 0000 0000 0000 0000 0000 0000 0000 ................
    48073030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    48073040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    48073050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    48073060: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    48073070: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    48073080: 6439 0020 0000 0000 0000 0000 0000 0000 9d .............
    48073090: 0000 0000 0000 0000 0001 0600 5449 0000 ............IT..
    480730a0: 0200 0000 0000 0000 0000 0000 0000 0000 ................
    480730b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    480730c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    480730d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    480730e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    480730f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    =>

  • Ronald Edgar said:
    => i2c md 4b 100.2 1
    Timed out in wait_for_event: status=1000
    Check if pads/pull-ups of bus are properly configured
    Error reading the chip: -121
    => md.w 0x48072000 0x80

    Were you watching the scope as you did these transactions?  In particular I'm wondering if you saw the above error messages on the iteration where the bus locked up.  Or did the bus lockup first, and on the following attempt to read from the bus you got the above error?

    Are you able to have your software team add some instrumentation into the I2C driver?

    File:drivers/i2c/omap24xx_i2c.c

    Function: __omap24_i2c_read()

    In particular:

    1. Can you print out the arguments that were passed into the functions?

    2. This is a bit of a shot in the dark based on past I2C issues, but can you stick a delay such as udelay(1000) between the "write" and "read" portions of the transaction?  If my file is the same as yours, I see a comment "Set slave address" at line 526.  That represents the beginning of the receive logic for the I2C read.  A delay right there would be perfect.

    3. Just a few lines after the above delay there's a while(1) loop for reading the data.  Just before that loop please print the values of  CNT and CON registers (i.e. just after you write them).

    To give you insight into why I've requested these specific items, I've seen in the past that the state machine timing can be very finicky.  What I think could possibly be happening is that if somehow the write of CNT=1 isn't registering properly with the hardware, that it might still be attempting to use a CNT=2 (i.e. from when the address was written). I'm sure there are plenty of holes in my theory, but some observations from your side based on these suggestions would be a big help for starters.

    Thanks!

    Brad

  • HI Brad,

       We tried putting in the delay but that did not help. We are working on the debug stuff now so hopefully more feedback later today. Ron.

  • Hi Brad,

       Here is what our SW person said:

    Here is something to test with. The output doesn’t make much sense but that could be because alen is reporting zero, which might be where the problem lies. That’s why the second debug output he requested is not being displayed.

      Here is where the script fell off the cliff with a few instruction before (ignore the unknown command stuff):

    => i2c mm 4b f.2
    0000000f:
    __omap24_i2c_read waitdelay->50 chip->4b addr->0000 alen->0 len->1
    00 ? 00
    00000010:
    __omap24_i2c_read waitdelay->50 chip->4b addr->0000 alen->0 len->1
    00 ? ^c
    => //serial_port_extensions
    Unknown command '//serial_port_extensions' - try 'help'
    => i2c mm 4b 10.2
    00000010:
    __omap24_i2c_read waitdelay->50 chip->4b addr->0000 alen->0 len->1
    Timed out in wait_for_event: status=1000
    Check if pads/pull-ups of bus are properly configured
    Error reading the chip: -121

  • HI Brad,

       From Bill, our SW guru:

    I think they need to take a look at this function in that file

     

    static int omap_i2c_xfer(struct udevice *bus, struct i2c_msg *msg, int nmsgs)

     

    it calls the read/write function but zeroes addr and alen

     

    static int omap_i2c_xfer(struct udevice *bus, struct i2c_msg *msg, int nmsgs)

    {

            struct omap_i2c *priv = dev_get_priv(bus);

            int ret;

     

            debug("i2c_xfer: %d messages\n", nmsgs);

            for (; nmsgs > 0; nmsgs--, msg++) {

                    debug("i2c_xfer: chip=0x%x, len=0x%x\n", msg->addr, msg->len);

                    if (msg->flags & I2C_M_RD) {

                            ret = __omap24_i2c_read(priv->regs                                               

         priv->waitdelay,

                                                    msg->addr, 0, 0, msg->buf,

                                                    msg->len);

                    } else {

                            ret = __omap24_i2c_write(priv->regs,

                                                     priv->waitdelay,

                                                     msg->addr, 0, 0, msg->buf,

                                                     msg->len);

                    }

                    if (ret) {

                            debug("i2c_write: error sending\n");

                            return -EREMOTEIO;

                    }

            }

     

            return 0;

    }

     

  • Ronald Edgar said:

                                                    msg->addr, 0, 0, msg->buf,

                                                    msg->len);

                    } else {

                            ret = __omap24_i2c_write(priv->regs,

                                                     priv->waitdelay,

                                                     msg->addr, 0, 0, msg->buf,

                                                     msg->len);

    Ah, yes.  It looks like an upper level of the driver is implicitly breaking down a "read" command into two messages, a write followed by a read.  That's fine.  There's one other place where there might be a timing issue.  Can you also try adding a udelay(1000) at the rd_exit label, i.e. just before the flush_fifo command?

    On a related note, are you able to perform similar tests from Linux using i2cset/i2cget commands?  In general I expect the Linux drivers to be more robust than the u-boot drivers.  If everything works flawlessly all the time from Linux that will at least confirm that we've got an issue in the u-boot drivers somewhere.

  • HI Brad,

       We're making progress. Adding the last delay fixes our u-boot issues. Unfortunately we are seeing similar issues when we boot the kernel. Where do we add the delays for that?

       Many thanks, Ron

  • That's great news!  By the way, I think we can clean up the u-boot fix slightly.  First, I think the location should be slightly different.  I think the delay should be after the flush_fifo and not before.  Second, rather than using udelay(1000), I recommend polling for I2C_CON[10] (MST) to clear.    The MST bit self-clears when the transaction is complete.  I've seen this in the past on other devices, i.e. for back-to-back transactions you need to make sure MST has cleared before initiating the next transaction.  Will you please try that out and let me know if it continues to work?  I think it's a bit more correct than simply inserting a delay which feels admittedly like a hack.

  • HI Brad,

       We will try that. Also, what and where should the same fix for the kernel go? Thanks, ROn.

  • Hi Brad,

       SW say they'd rather not go backwards at this moment but can we please have the kernel fix? Many thanks, Ron.

  • I was looking at the kernel code in parallel.  Can you tell me more about how you recreate the issue for example:

    1. Are you using i2cset/i2cget commands from a script?

    2. Does putting a delay between commands "fix" the issue?

    3. Are you seeing the same failure, e.g. can you see the controller ACKing the final transaction when acting as master receiver?

    The Linux kernel driver is a bit more complex than the u-boot driver.  It supports both interrupts and polling.  It will take some study to determine the appropriate fix. I need to know if MST self-clears when the transaction ends.  I'm pretty sure it does, but if your software team could please spend a few minutes to re-run that other test, it would help clarify the path forward.

  • HI Brad,

       About 4 seconds or so into the kernel boot, the same I2C device that formally failed the script initialization in u-boot gets initialized by some code we wrote to initialize the device. So there is a huge flurry of transactions, similar to the script. After about 10mS or more, one of the write transactions fails and it looks identical to what I previously sent you. So the I2C bus gets stuck with the SDA line low, just the same. 

           Ron.

  • HI Brad,

       Let me know if you need any more info. Ron.

  • For now I'll assume that the MST bit self-clears as I recall.  The issue of too little time between transfers only happens when there are back-to-back transfers.  While you could enforce this at the end of the transaction, from a CPU perspective I think it is better to check it at the start of the transaction.  

    This would be my first thought of how to fix it:

    static int omap_i2c_xfer_msg(struct i2c_adapter *adap,
    struct i2c_msg *msg, int stop, bool polling)
    {
    struct omap_i2c_dev *omap = i2c_get_adapdata(adap);
    unsigned long timeout;
    u16 w, con_reg;
    int ret;

    dev_dbg(omap->dev, "addr: 0x%04x, len: %d, flags: 0x%x, stop: %d\n",
    msg->addr, msg->len, msg->flags, stop);

    con_reg = omap_i2c_read_reg(omap, OMAP_I2C_CON_REG);
    if ( con_reg & OMAP_I2C_CON_MST )
         return -EAGAIN;

    omap->receiver = !!(msg->flags & I2C_M_RD);
    omap_i2c_resize_fifo(omap, msg->len, omap->receiver);

    Please try it out.

  • HI Brad,

       Sorry, that did not work. Here is the code as returned from our SW people. Hopefully we did what you asked:

    here is the code snippet

    /*

     * Low level master read/write transaction.

     */

    static int omap_i2c_xfer_msg(struct i2c_adapter *adap,

                  struct i2c_msg *msg, int stop)

    {

       struct omap_i2c_dev *omap = i2c_get_adapdata(adap);

       unsigned long timeout;

       u16 w, con_reg;

     

       dev_dbg(omap->dev, "addr: 0x%04x, len: %d, flags: 0x%x, stop: %d\n",

          msg->addr, msg->len, msg->flags, stop);

     

       if (msg->len == 0)

          return -EINVAL;

     

       con_reg = omap_i2c_read_reg(omap, OMAP_I2C_CON_REG);

       if ( con_reg & OMAP_I2C_CON_MST )

          return -EAGAIN;

     

       omap->receiver = !!(msg->flags & I2C_M_RD);

       omap_i2c_resize_fifo(omap, msg->len, omap->receiver);

     

       omap_i2c_write_reg(omap, OMAP_I2C_SA_REG, msg->addr);

     

       /* REVISIT: Could the STB bit of I2C_CON be used with probing? */

       omap->buf = msg->buf;

       omap->buf_len = msg->len;

     

       /* make sure writes to omap->buf_len are ordered */

       barrier();

     

       omap_i2c_write_reg(omap, OMAP_I2C_CNT_REG, omap->buf_len);

    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    Here is what I see on the i2c bus:

    When I monitor the accesses to the AD9545 on I2C2, I see an initial access of a 2-byte write (successful) but the SCL line is now jammed low!!! ~ 1 second later I see one high/low pulse on the clock line and the data line SDA goes low and stays there for ~ 2.6 seconds when both SDA and SCL go high (presumably a reset of some kind). Something like 6mS later there is another transaction: it’s a 1-byte write to location (7-bit) 0x20 with data 0x0a (successful). The transaction ends the same as previous with the data line held low.

       So, if anything, things got a lot worse.

           Ron.

  • Hi Brad,

       More info from the SW people:

    Dumped con_reg. Here is the output prior to crash.

     

    [    1.662009] tpd12s015 encoder0: failed to find video source

    [    1.662276] connector-dvi connector1: failed to find video source

    [    1.662534] connector-hdmi connector0: failed to find video source

    [    1.662857] panel-dpi display: failed to find video source

    [    1.687313] omap_i2c_xfer_msg con_reg->8000

    [    1.687632] omap_i2c_xfer_msg con_reg->8600

    [    2.710287] random: fast init done

    [    2.866118] random: crng init done

    [    4.022993] Serial: 8250/16550 driver, 10 ports, IRQ sharing disabled

    [    4.028434] 4806a000.serial: ttyS0 at MMIO 0x4806a000 (irq = 301, base_baud = 3000000) is a 8250

    [    4.078152] ------------[ cut here ]------------

    [    4.078168] WARNING: CPU: 0 PID: 20 at drivers/bus/omap_l3_noc.c:147 l3_interrupt_handler+0x25c/0x378

    [    4.078173] 44000000.ocp:L3 Custom Error: MASTER MPU TARGET L4_PER1_P3 (Read): Data Access in User mode during Functional access

  • Ok, maybe that was too fancy.  Let's try something a bit simpler and more straight forward.  Further down in omap_i2c_xfer_msg is the following:

    if (timeout == 0) {
    dev_err(omap->dev, "controller timed out\n");
    omap_i2c_reset(omap);
    __omap_i2c_init(omap);
    return -ETIMEDOUT;
    }

    if (likely(!omap->cmd_err))
    return 0;

    I believe that "return 0" statement is the normal place where the function would return.  Can you try putting a delay right there?  Maybe this:

    if (likely(!omap->cmd_err))
    {
    mdelay(1);
    return 0;
    }

    Let's see if it works first.  If it does, perhaps we can do something a little cleaner like watching for the MST bit to transition.  I just want to be careful not to eat up a bunch of CPU polling a bit. 

    Brad

  • HI Brad,

       Sorry, that didn't help either. But it didn't change the character of the original failure either. Still ending up with SDA stuck low. And for a warm board, all the way up with no errors. 

     

       Next?

           Thanks, Ron.

  • Can you put a couple printk's in there?  Maybe 1 following the mdelay(1) and another after that "if" statement, i.e. so you can see how often you actually are going beyond that part of the code?

  • Hi Brad,

       Here is the console spew with the prints:


    U-Boot SPL 2018.03-rc3 (Jan 13 2021 - 12:57:09 -0500)
    DRA762-HS ES1.0 ABZ package
    SDRAM test results:
    Tests [00000000] Not run
    Cf=00000007 ECCrange now 3fff0000
    EMIF1 A=00000000 D=00000000 M=00000000
    EA=00000000 ED=00000000 EM=00000000
    EMIF2 A=00000000 D=00000000 M=00000000
    Trying to boot from MMC1
    no pinctrl state for default mode
    Authentication passed
    Authentication passed


    U-Boot 2018.03-rc3 (Jan 18 2021 - 10:34:03 -0500)

    CPU : DRA762-HS ES1.0 ABZ package
    Model: Compunetix AM5748 ID16P
    Board: Compunetix ID16P REV 00
    DRAM: 2 GiB
    MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1
    Loading Environment from MMC... OK
    GUID Partition Table Header signature is wrong: 0x0 != 0x5452415020494645
    part_get_info_efi: *** ERROR: Invalid GPT ***
    GUID Partition Table Header signature is wrong: 0x0 != 0x5452415020494645
    part_get_info_efi: *** ERROR: Invalid Backup GPT ***
    compu1160_dvi_enable: DVI Monitor Not Detected: 1!
    Model Type up to date unknown
    We're red. Let's reset the usb interface to be safe
    SCSI: SATA link 0 timeout.
    AHCI 0001.0300 32 slots 1 ports 3 Gbps 0x1 impl SATA mode
    flags: 64bit ncq stag pm led clo only pmp pio slum part ccc apst
    scanning bus for devices...
    Found 0 device(s).
    Net: eth0: ethernet@48484000
    Hit any key to stop autoboot: 0
    => boot
    switch to partitions #0, OK
    mmc0 is current device
    SD/MMC found on device 0
    4120148 bytes read in 183 ms (21.5 MiB/s)
    ## Loading kernel from FIT Image at 90000000 ...
    Using 'conf@1' configuration
    Trying 'kernel@1' kernel subimage
    Description: Vanilla Linux kernel
    Type: Kernel Image
    Compression: uncompressed
    Data Start: 0x900000ec
    Data Size: 3899744 Bytes = 3.7 MiB
    Architecture: ARM
    OS: Linux
    Load Address: 0x82000000
    Entry Point: 0x82000000
    Hash algo: sha256
    Hash value: bf4d256db7b5095969c01b1aee34a0011c321c4c1f952baf0d638b63ccb0400a
    Verifying Hash Integrity ... sha256+ OK
    Authentication passed
    ## Loading fdt from FIT Image at 90000000 ...
    Using 'conf@1' configuration
    Trying 'fdt@1' fdt subimage
    Description: Flattened Device Tree blob
    Type: Flat Device Tree
    Compression: uncompressed
    Data Start: 0x903b834c
    Data Size: 109168 Bytes = 106.6 KiB
    Architecture: ARM
    Hash algo: sha256
    Hash value: ced626bcfa4fbe9e4ba2505b480660a3ba849178de7f54eca88ced79646e5353
    Verifying Hash Integrity ... sha256+ OK
    Authentication passed
    Booting using the fdt blob at 0x903b834c
    Loading Kernel Image ... OK
    Loading Device Tree to 8ffe2000, end 8ffff954 ... OK
    get_voltrail_opp
    get_voltrail_opp
    get_voltrail_opp

    Starting kernel ...

    [ 0.000000] Booting Linux on physical CPU 0x0
    [ 0.000000] Linux version 4.9.41 (wkanczes@slash) (gcc version 7.3.0 (crosstool-NG crosstool-ng-1.23.0-319-gaca85cb) ) #8 SMP PREEMPT Tue Jan 19 14:10:26 EST 2021
    [ 0.000000] CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=30c5387d
    [ 0.000000] CPU: div instructions available: patching division code
    [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
    [ 0.000000] OF: fdt:Machine model: COMPUNETIX AM5748 ID16P
    [ 0.000000] efi: Getting EFI parameters from FDT:
    [ 0.000000] efi: UEFI not found.
    [ 0.000000] Reserved memory: created CMA memory pool at 0x0000000095800000, size 56 MiB
    [ 0.000000] OF: reserved mem: initialized node ipu2_cma@95800000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created CMA memory pool at 0x0000000099000000, size 64 MiB
    [ 0.000000] OF: reserved mem: initialized node dsp1_cma@99000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created CMA memory pool at 0x000000009d000000, size 32 MiB
    [ 0.000000] OF: reserved mem: initialized node ipu1_cma@9d000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created CMA memory pool at 0x000000009f000000, size 8 MiB
    [ 0.000000] OF: reserved mem: initialized node dsp2_cma@9f000000, compatible id shared-dma-pool
    [ 0.000000] cma: Reserved 24 MiB at 0x00000000fc800000
    [ 0.000000] Memory policy: Data cache writealloc
    [ 0.000000] OMAP4: Map 0x00000000ffd00000 to fe600000 for dram barrier
    [ 0.000000] dra7xxx_check_revision: unknown idcode=0x0bb5002f (hawkeye=0x0000bb50,rev=0x0)
    [ 0.000000] DRA752 ES2.0
    [ 0.000000] percpu: Embedded 13 pages/cpu @eed79000 s22156 r8192 d22900 u53248
    [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 465216
    [ 0.000000] Kernel command line: console=ttyO0,115200n8 omapdss.def_disp=hdmi omapfb.mode=hdmi:1920x1080MR-32@60 usbcore.autosuspend=-1 root=PARTUUID=ce1452b7-02 rw rootfstype=ext4 rootwait 2
    [ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
    [ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
    [ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
    [ 0.000000] Memory: 1646244K/1867776K available (8192K kernel code, 323K rwdata, 2516K rodata, 2048K init, 302K bss, 33116K reserved, 188416K cma-reserved, 1253376K highmem)
    [ 0.000000] Virtual kernel memory layout:
    [ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
    [ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
    [ 0.000000] vmalloc : 0xf0800000 - 0xff800000 ( 240 MB)
    [ 0.000000] lowmem : 0xc0000000 - 0xf0000000 ( 768 MB)
    [ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
    [ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
    [ 0.000000] .text : 0xc0008000 - 0xc0a00000 (10208 kB)
    [ 0.000000] .init : 0xc0e00000 - 0xc1000000 (2048 kB)
    [ 0.000000] .data : 0xc1000000 - 0xc1050e18 ( 324 kB)
    [ 0.000000] .bss : 0xc1052000 - 0xc109d8f4 ( 303 kB)
    [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
    [ 0.000000] Preemptible hierarchical RCU implementation.
    [ 0.000000] Build-time adjustment of leaf fanout to 32.
    [ 0.000000] NR_IRQS:16 nr_irqs:16 16
    [ 0.000000] OMAP clockevent source: timer1 at 32786 Hz
    [ 0.000000] arm_arch_timer: Architected cp15 timer(s) running at 6.14MHz (virt).
    [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x16af5adb9, max_idle_ns: 440795202250 ns
    [ 0.000004] sched_clock: 56 bits at 6MHz, resolution 162ns, wraps every 4398046511023ns
    [ 0.000015] Switching to timer-based delay loop, resolution 162ns
    [ 0.000333] clocksource: 32k_counter: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58327039986419 ns
    [ 0.000341] OMAP clocksource: 32k_counter at 32768 Hz
    [ 0.000776] Console: colour dummy device 80x30
    [ 0.000794] WARNING: Your 'console=ttyO0' has been replaced by 'ttyS0'
    [ 0.000800] This ensures that you still see kernel messages. Please
    [ 0.000806] update your kernel commandline.
    [ 0.000823] Calibrating delay loop (skipped), value calculated using timer frequency.. 12.29 BogoMIPS (lpj=61475)
    [ 0.000836] pid_max: default: 32768 minimum: 301
    [ 0.000937] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
    [ 0.000947] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
    [ 0.001585] CPU: Testing write buffer coherency: ok
    [ 0.001794] /cpus/cpu@0 missing clock-frequency property
    [ 0.001810] /cpus/cpu@1 missing clock-frequency property
    [ 0.001821] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
    [ 0.001838] Setting up static identity map for 0x80200000 - 0x80200060
    [ 0.080048] EFI services will not be available.
    [ 0.170199] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
    [ 0.170287] Brought up 2 CPUs
    [ 0.170299] SMP: Total of 2 processors activated (24.59 BogoMIPS).
    [ 0.170306] CPU: All CPU(s) started in SVC mode.
    [ 0.170846] devtmpfs: initialized
    [ 0.198986] VFP support v0.3: implementor 41 architecture 4 part 30 variant f rev 0
    [ 0.199227] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
    [ 0.199244] futex hash table entries: 512 (order: 3, 32768 bytes)
    [ 0.203032] pinctrl core: initialized pinctrl subsystem
    [ 0.203858] NET: Registered protocol family 16
    [ 0.204751] DMA: preallocated 256 KiB pool for atomic coherent allocations
    [ 0.205791] omap_hwmod: l3_main_2 using broken dt data from ocp
    [ 0.420333] cpuidle: using governor ladder
    [ 0.450358] cpuidle: using governor menu
    [ 0.458596] GPIO line 16 (tfp410_reset_n) hogged as output/high
    [ 0.460276] OMAP GPIO hardware version 0.1
    [ 0.463034] GPIO line 64 (dvi_ls_oe_n) hogged as output/high
    [ 0.463053] GPIO line 65 (dvi_ct_hpd_n) hogged as output/high
    [ 0.463070] GPIO line 88 (usbhub1_reset_n) hogged as output/high
    [ 0.471023] GPIO line 197 (usbhub1_vbus2_n) hogged as output/low
    [ 0.471041] GPIO line 198 (usbhub1_vbus3_n) hogged as output/low
    [ 0.471057] GPIO line 199 (usbhub1_vbus4_n) hogged as output/low
    [ 0.474700] irq: no irq domain found for /ocp/l4@4a000000/scm@2000/pinmux@1400 !
    [ 0.504537] No ATAGs?
    [ 0.504562] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
    [ 0.504571] hw-breakpoint: maximum watchpoint size is 8 bytes.
    [ 0.504925] omap4_sram_init:Unable to allocate sram needed to handle errata I688
    [ 0.504935] omap4_sram_init:Unable to get sram pool needed to handle errata I688
    [ 0.505481] OMAP DMA hardware revision 0.0
    [ 0.552670] omap-dma-engine 4a056000.dma-controller: OMAP DMA engine driver (LinkedList1/2/3 supported)
    [ 0.554002] edma 43300000.edma: memcpy is disabled
    [ 0.558685] edma 43300000.edma: TI EDMA DMA engine driver
    [ 0.563155] omap-iommu 40d01000.mmu: 40d01000.mmu registered
    [ 0.563348] omap-iommu 40d02000.mmu: 40d02000.mmu registered
    [ 0.563504] omap-iommu 58882000.mmu: 58882000.mmu registered
    [ 0.563656] omap-iommu 55082000.mmu: 55082000.mmu registered
    [ 0.563922] omap-iommu 41501000.mmu: 41501000.mmu registered
    [ 0.564117] omap-iommu 41502000.mmu: 41502000.mmu registered
    [ 0.564553] usbcore: registered new interface driver usbfs
    [ 0.564609] usbcore: registered new interface driver hub
    [ 0.564692] usbcore: registered new device driver usb
    [ 0.565638] palmas 0-0058: Irq flag is 0x00000008
    [ 0.590790] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.592035] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.593371] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.594607] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.595850] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.597183] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.598410] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.599651] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.600983] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.602210] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.603453] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.604784] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.606010] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.607253] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.608598] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.609842] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.611074] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.612316] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.613549] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.614789] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.616019] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.617261] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.618506] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.619753] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.620981] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.622226] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.623240] palmas 0-0058: Muxing GPIO 2b, PWM 0, LED 0
    [ 0.623571] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.625376] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.626620] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.627851] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.629096] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.630325] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.631570] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.632905] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.634134] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.635379] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.636608] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.637852] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.639093] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.640336] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.641573] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.642817] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.644150] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.645381] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.646626] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.647909] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.649154] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.650605] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.651850] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.653186] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.654413] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.655656] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.656885] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.658128] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.659370] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.660614] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.661849] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.663093] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.664375] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.665620] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.666831] vtt_fixed: supplied by smps3
    [ 0.667098] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.668342] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.669589] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.670834] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.672062] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.673306] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.674639] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.675868] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.677112] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.678340] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.679584] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.680828] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.682074] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.683308] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.684551] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.685884] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.687115] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.688358] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.689636] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.690880] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.692307] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.693552] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.694781] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.696024] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.697359] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.698587] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.699831] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.701059] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.702304] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.703547] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.704790] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.706026] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.707271] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.708604] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.709835] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.711079] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.712360] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.713606] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.715040] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.716284] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.717619] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.718848] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.720093] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.721323] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.722566] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.723813] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.725056] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.726291] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.727536] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.728869] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.730098] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.731345] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.732621] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.733867] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.735321] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.736566] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.737795] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.739040] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.740374] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.741604] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.742849] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.744078] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.745321] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.746564] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.747808] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.749042] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.750272] random: fast init done
    [ 0.750297] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.751632] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.752863] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.754106] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.755382] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.756628] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.758093] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.759337] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.760670] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.761900] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.763145] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.764374] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.765619] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.766863] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.768107] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.769343] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.770587] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.771922] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.773199] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.774445] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.775928] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.777174] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.778406] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.779649] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.781013] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.782257] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.783533] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.784778] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.786259] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.787503] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.788838] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.790083] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.791332] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.792562] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.793806] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.801088] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.802334] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.803824] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.805067] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.806402] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.807648] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.808893] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.810125] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.811369] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.821121] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.822366] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.823886] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.825131] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.826466] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.827713] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.828957] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.830189] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.831432] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.841152] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.842397] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.843921] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.845166] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.846501] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.847747] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.848991] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.850262] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.851507] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.853060] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.854305] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.855576] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.856822] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.858401] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.859647] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.860927] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.862172] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.863764] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.865009] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.866280] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.867525] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.869140] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.870384] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.871613] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.872857] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.874088] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.875333] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.881202] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.882446] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.884074] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.885320] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.886653] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.887902] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.889146] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.890377] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.891623] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.901233] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.902478] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.904119] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.905366] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.906702] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.907951] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.909194] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.910427] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.911673] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.921266] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.922511] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.924177] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.925421] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.926754] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.927991] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.929237] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.930407] vdd_3v3: supplied by regen1
    [ 0.930673] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.931920] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.932965] aic_dvdd_fixed: supplied by vdd_3v3
    [ 0.933498] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.934743] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.938410] omap_i2c 48070000.i2c: bus 0 rev0.12 at 100 kHz
    [ 0.939447] omap_i2c 48072000.i2c: bus 1 rev0.12 at 100 kHz
    [ 0.939916] omap_i2c 48060000.i2c: bus 2 rev0.12 at 400 kHz
    [ 0.940272] omap_i2c 4807a000.i2c: bus 3 rev0.12 at 400 kHz
    [ 0.940641] omap_i2c 4807c000.i2c: bus 4 rev0.12 at 400 kHz
    [ 0.940832] media: Linux media interface: v0.10
    [ 0.940879] Linux video capture interface: v2.00
    [ 0.940913] pps_core: LinuxPPS API ver. 1 registered
    [ 0.940920] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
    [ 0.940939] PTP clock support registered
    [ 0.940968] EDAC MC: Ver: 3.0.0
    [ 0.941693] omap-mailbox 4883c000.mailbox: omap mailbox rev 0x400
    [ 0.941986] omap-mailbox 4883e000.mailbox: omap mailbox rev 0x400
    [ 0.942269] omap-mailbox 48840000.mailbox: omap mailbox rev 0x400
    [ 0.942552] omap-mailbox 48842000.mailbox: omap mailbox rev 0x400
    [ 0.942741] omap-mailbox 48846000.mailbox: omap mailbox rev 0x400
    [ 0.943086] Advanced Linux Sound Architecture Driver Initialized.
    [ 0.943782] clocksource: Switched to clocksource arch_sys_counter
    [ 0.953616] NET: Registered protocol family 2
    [ 0.954151] TCP established hash table entries: 8192 (order: 3, 32768 bytes)
    [ 0.954214] TCP bind hash table entries: 8192 (order: 4, 65536 bytes)
    [ 0.954339] TCP: Hash tables configured (established 8192 bind 8192)
    [ 0.954388] UDP hash table entries: 512 (order: 2, 16384 bytes)
    [ 0.954419] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
    [ 0.954615] NET: Registered protocol family 1
    [ 0.954951] RPC: Registered named UNIX socket transport module.
    [ 0.954960] RPC: Registered udp transport module.
    [ 0.954967] RPC: Registered tcp transport module.
    [ 0.954974] RPC: Registered tcp NFSv4.1 backchannel transport module.
    [ 0.955949] hw perfevents: enabled with armv7_cortex_a15 PMU driver, 7 counters available
    [ 0.958826] workingset: timestamp_bits=14 max_order=19 bucket_order=5
    [ 0.966247] squashfs: version 4.0 (2009/01/31) Phillip Lougher
    [ 0.966959] NFS: Registering the id_resolver key type
    [ 0.966979] Key type id_resolver registered
    [ 0.966986] Key type id_legacy registered
    [ 0.967028] ntfs: driver 2.1.32 [Flags: R/O].
    [ 0.968204] bounce: pool size: 64 pages
    [ 0.968353] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246)
    [ 0.968363] io scheduler noop registered
    [ 0.968371] io scheduler deadline registered
    [ 0.968499] io scheduler cfq registered (default)
    [ 0.970918] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.972166] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.973769] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.975043] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.977865] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.979111] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.981107] pinctrl-single 4a003400.pinmux: 282 pins at pa fc003400 size 1128
    [ 0.984332] omap_dss_probe compunetix offset value 0
    [ 0.986389] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.987635] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.988809] OMAP DSS rev 6.1
    [ 0.989106] omapdss_dss 58000000.dss: bound 58001000.dispc (ops dispc_component_ops)
    [ 0.989754] omapdss_dss 58000000.dss: bound 58040000.encoder (ops hdmi5_component_ops)
    [ 0.992534] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 0.993809] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.023866] omapdss APPLY error: manager lcd is already connected to an output
    [ 1.024558] fbcvt: 1920x1080@60: CVT Name - 2.073M9-R
    [ 1.053731] Console: switching to colour frame buffer device 240x67
    [ 1.069543] omapfb omapfb: using display 'hdmi' mode 1920x1080
    [ 1.094125] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.095376] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.096701] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.097952] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.098965] clk-ctx160 1-004b: Found AD9545.
    [ 1.098975] clk-ctx160 1-004b: Vers. 1.002
    [ 1.099045] clk-ctx160 1-004b: XO source is board_49M152_osc.
    [ 1.099126] clk-ctx160 1-004b: REF_BB source is board_20M_clk.
    [ 1.099134] clk-ctx160 1-004b: Found 2 source clocks.
    [ 1.099147] clk-ctx160 1-004b: Creating output clock[0] clk_plogic_ref.
    [ 1.099243] clk-ctx160 1-004b: Creating output clock[1] clk_aux_sys.
    [ 1.099308] clk-ctx160 1-004b: Creating output clock[2] clk_audio_sys.
    [ 1.099351] clk-ctx160 1-004b: Creating output clock[3] clk_otherside_ref.
    [ 1.099390] clk-ctx160 1-004b: Creating output clock[4] clk_audio_framea.
    [ 1.099429] clk-ctx160 1-004b: Creating output clock[5] clk_audio_frameb.
    [ 1.099886] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.101319] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.103738] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.106342] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.107770] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.110010] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.112159] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.114228] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.116827] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.118795] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.121392] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.123362] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.125963] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.127932] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.130531] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.132499] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.135105] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.136803] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.139403] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.141101] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.143700] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.145400] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.147999] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.149698] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.152298] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.154004] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.156602] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.158301] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.160899] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.162596] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.165203] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.166902] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.169320] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.171738] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.174344] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.175955] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.177653] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.179531] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.181679] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.183834] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.185983] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.188133] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.190281] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.192429] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.195035] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.196552] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.199149] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.200668] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.203266] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.204786] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.207384] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.208902] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.211501] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.213017] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.215616] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.217134] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.219732] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.221341] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.223040] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 1.224432] omap_i2c 48072000.i2c: Arbitration lost
    [ 1.224455] omap_i2c_xfer_msg omap->cmd_err->00000001
    [ 2.462198] random: crng init done
    [ 4.313824] omap_i2c 48072000.i2c: controller timed out
    [ 4.374939] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.377092] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.379242] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.381839] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.383358] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.385959] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.387479] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.390079] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.391598] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.394204] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.395723] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.398322] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.399840] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.402440] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.403966] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.405845] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.407903] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.409961] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.412289] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.414627] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.416142] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.417567] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 4.418959] omap_i2c 48072000.i2c: Arbitration lost
    [ 4.418979] omap_i2c_xfer_msg omap->cmd_err->00000001
    [ 5.433822] clk-ctx160 1-004b: Added provider.
    [ 7.513803] omap_i2c 48072000.i2c: controller timed out
    [ 7.574197] omap_i2c 48072000.i2c: Arbitration lost
    [ 7.574222] omap_i2c_xfer_msg omap->cmd_err->00000001
    [ 10.633804] omap_i2c 48072000.i2c: controller timed out
    [ 10.694179] omap_i2c 48072000.i2c: Arbitration lost
    [ 10.694204] omap_i2c_xfer_msg omap->cmd_err->00000001
    [ 13.753803] omap_i2c 48072000.i2c: controller timed out
    [ 13.814113] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.815366] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.821689] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.822939] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.829265] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.830517] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.836843] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.838095] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.844421] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.845672] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.851994] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.853242] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.859570] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.860821] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.867141] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.868388] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.874714] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.875966] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.882290] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.883539] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.889864] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.891115] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.897441] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.898690] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.905015] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.906265] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 13.912589] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 14.953801] omap_i2c 48072000.i2c: controller timed out
    [ 17.033799] omap_i2c 48072000.i2c: controller timed out
    [ 17.094114] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 17.095364] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 17.101690] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 17.102941] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 17.109266] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 17.110518] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 17.116844] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 17.118094] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 17.124418] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 17.125669] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 17.131994] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 17.133244] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 17.139677] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 17.141070] omap_i2c 48072000.i2c: Arbitration lost
    [ 17.141094] omap_i2c_xfer_msg omap->cmd_err->00000001
    [ 20.233803] omap_i2c 48072000.i2c: controller timed out
    [ 20.294179] omap_i2c 48072000.i2c: Arbitration lost
    [ 20.294203] omap_i2c_xfer_msg omap->cmd_err->00000001
    [ 23.353802] omap_i2c 48072000.i2c: controller timed out
    [ 23.414112] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.415362] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.421687] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.422936] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.429262] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.430513] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.436840] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.438091] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.444417] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.445667] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.451989] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.453240] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.459563] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.460812] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.467135] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.468387] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.474713] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.475962] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.482286] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.483537] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.489863] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.491114] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 23.497417] omap_i2c 48072000.i2c: Arbitration lost
    [ 23.497440] omap_i2c_xfer_msg omap->cmd_err->00000001
    [ 26.553818] omap_i2c 48072000.i2c: controller timed out
    [ 26.614090] omap_i2c 48072000.i2c: Arbitration lost
    [ 26.614115] omap_i2c_xfer_msg omap->cmd_err->00000001
    [ 29.673816] omap_i2c 48072000.i2c: controller timed out
    [ 29.703804] clk-ctx160 1-004b: calibrated.
    [ 29.763737] Serial: 8250/16550 driver, 10 ports, IRQ sharing disabled
    [ 29.767170] 4806a000.serial: ttyS0 at MMIO 0x4806a000 (irq = 301, base_baud = 3000000) is a 8250
    [ 32.956251] console [ttyS0] enabled
    [ 32.961429] [drm] Initialized
    [ 32.975004] brd: module loaded
    [ 32.983240] loop: module loaded
    [ 32.986979] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 32.993290] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 32.999543] at24 0-0050: 256 byte 24c02 EEPROM, writable, 1 bytes/write
    [ 33.006532] ctx160-plogic v0.001: probing
    [ 33.011321] ctx160-plogic ctx160-plogic: Found 8-wide dipswitch.
    [ 33.017547] ctx161-plogic 1-0020: Driver established
    [ 33.044023] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.050341] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.056661] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.062978] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.069296] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.075631] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.081930] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.088262] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.094339] ctx161-plogic 1-0020: Hardware found, version=00000000
    [ 33.100770] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.107102] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.113504] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.119746] ctx161-codec 0-0010: assume MAX98091
    [ 33.124664] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.130982] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.137081] ctx161-codec 0-0010: Vers. 1.002 Rev. 52
    [ 33.142484] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.148904] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.155325] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.161726] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.168147] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.174642] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.181042] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.187551] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.194048] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.200446] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.206886] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.213287] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.219708] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.226128] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.232621] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.239403] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.246184] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.252769] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.259371] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.265883] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.272285] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.278452] ctx161-codec 1-0010: assume MAX98091
    [ 33.283347] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.289681] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.295777] ctx161-codec 1-0010: Vers. 1.002 Rev. 52
    [ 33.301179] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.307605] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.314017] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.320419] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.326840] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.333331] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.339753] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.346263] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.352755] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.359177] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.365599] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.372000] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.378421] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.384823] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.391314] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.398095] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.404857] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.411436] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.418058] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.424553] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.430955] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.437138] ctx161-codec 2-0010: assume MAX98091
    [ 33.463883] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.470035] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.476131] ctx161-codec 2-0010: Vers. 1.002 Rev. 52
    [ 33.481311] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.487511] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.493692] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.499891] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.506088] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.512291] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.518490] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.524693] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.530894] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.537092] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.543273] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.549474] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.555672] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.561852] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.568074] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.574345] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.580613] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.586857] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.593082] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.599307] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.605506] omap_i2c_xfer_msg omap->cmd_err->00000000
    [ 33.611952] mc60-audio16 v0.007: probing
    [ 33.615938] mc60-audio16 mc60-audio16: Found DSP clock.
    [ 33.621216] mc60-audio16 mc60-audio16: Found IPU clock.
    [ 33.626522] mc60-audio16 mc60-audio16: Found DSP MMU0 @40d01000.
    [ 33.632595] mc60-audio16 mc60-audio16: Found DSP MMU1 @40d02000.
    [ 33.638701] mc60-audio16 mc60-audio16: Found IPU MMU @58882000.
    [ 33.644662] mc60-audio16 mc60-audio16: Assuming IPU Unicache @58880000.
    [ 33.651477] mc60-audio16 mc60-audio16: Found BIOS timer @fa820000.
    [ 33.657831] mc60-audio16 mc60-audio16: Found BIOS timer @fa088000.
    [ 33.664158] mc60-audio16 mc60-audio16: Reusing frame clock gpio.
    [ 33.670202] mc60-audio16 mc60-audio16: VOIP memory @adc00000 = edc00000 + 00000000
    [ 33.677823] mc60-audio16 mc60-audio16: Xfer memory @adc10000 = edc00000 + 00010000
    [ 33.686053] davinci-mcasp 48460000.mcasp: Configured tdm_slots=2
    [ 33.692091] davinci-mcasp 48460000.mcasp: Counted num_serializer=16
    [ 33.698435] davinci-mcasp 48460000.mcasp: Assigned num_serializer=16
    [ 33.707426] omap_hwmod: mcasp1: _wait_target_ready failed: -16
    [ 33.713290] davinci-mcasp 48460000.mcasp: use pm_runtime_put_sync_suspend() in driver?
    [ 33.721264] ------------[ cut here ]------------
    [ 33.725911] WARNING: CPU: 0 PID: 1 at drivers/bus/omap_l3_noc.c:147 l3_interrupt_handler+0x25c/0x378
    [ 33.735084] 44000000.ocp:L3 Custom Error: MASTER MPU TARGET L4_PER2_P3 (Read): Data Access in User mode during Functional access
    [ 33.746695] Modules linked in:
    [ 33.749773] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.9.41 #8
    [ 33.755717] Hardware name: Generic DRA74X (Flattened Device Tree)
    [ 33.761835] Backtrace:
    [ 33.764308] [<c020b3c4>] (dump_backtrace) from [<c020b6a0>] (show_stack+0x18/0x1c)
    [ 33.771912] r7:00000009 r6:60000193 r5:00000000 r4:c1022a50
    [ 33.777600] [<c020b688>] (show_stack) from [<c04cd4fc>] (dump_stack+0x90/0xa4)
    [ 33.784857] [<c04cd46c>] (dump_stack) from [<c022e34c>] (__warn+0xec/0x104)
    [ 33.791848] r7:00000009 r6:c0bcc164 r5:00000000 r4:ee87baf8
    [ 33.797532] [<c022e260>] (__warn) from [<c022e3a4>] (warn_slowpath_fmt+0x40/0x48)
    [ 33.805047] r9:0000001a r8:eea18b10 r7:c0bcbfd0 r6:00000002 r5:c0bcc090 r4:c0bcc134
    [ 33.812826] [<c022e368>] (warn_slowpath_fmt) from [<c04fcc84>] (l3_interrupt_handler+0x25c/0x378)
    [ 33.821735] r3:eea18980 r2:c0bcc134
    [ 33.825322] r4:80080003
    [ 33.827872] [<c04fca28>] (l3_interrupt_handler) from [<c027f954>] (__handle_irq_event_percpu+0x6c/0x134)
    [ 33.837394] r10:c101a92f r9:eea19300 r8:00000017 r7:ee87bc1c r6:00000000 r5:eea19300
    [ 33.845255] r4:eea18e80
    [ 33.847803] [<c027f8e8>] (__handle_irq_event_percpu) from [<c027fa40>] (handle_irq_event_percpu+0x24/0x60)
    [ 33.857499] r10:c0e4883c r9:ee87a000 r8:ee808000 r7:00000001 r6:c100906c r5:eea19300
    [ 33.865359] r4:eea19300
    [ 33.867905] [<c027fa1c>] (handle_irq_event_percpu) from [<c027fabc>] (handle_irq_event+0x40/0x64)
    [ 33.876813] r5:eea19360 r4:eea19300
    [ 33.880407] [<c027fa7c>] (handle_irq_event) from [<c0283108>] (handle_fasteoi_irq+0xc4/0x194)
    [ 33.888968] r7:00000001 r6:c100906c r5:eea19360 r4:eea19300
    [ 33.894653] [<c0283044>] (handle_fasteoi_irq) from [<c027ebb4>] (generic_handle_irq+0x2c/0x3c)
    [ 33.903302] r7:00000001 r6:00000000 r5:00000000 r4:c0e5dddc
    [ 33.908988] [<c027eb88>] (generic_handle_irq) from [<c027f13c>] (__handle_domain_irq+0x64/0xbc)
    [ 33.917726] [<c027f0d8>] (__handle_domain_irq) from [<c020146c>] (gic_handle_irq+0x40/0x7c)
    [ 33.926114] r9:ee87a000 r8:fa213000 r7:fa212000 r6:ee87bcd8 r5:fa21200c r4:c1003424
    [ 33.933892] [<c020142c>] (gic_handle_irq) from [<c020c238>] (__irq_svc+0x58/0x8c)
    [ 33.941406] Exception stack(0xee87bcd8 to 0xee87bd20)
    [ 33.946478] bcc0: eeac7c10 00000005
    [ 33.954692] bce0: 00000000 eeac7ce8 eeac7c10 00000005 eeac7c00 00000000 00000000 00000000
    [ 33.962906] bd00: c0e4883c ee87bd44 ee87bd48 ee87bd28 c0643e98 c061d13c 20000013 ffffffff
    [ 33.971120] r9:ee87a000 r8:00000000 r7:ee87bd0c r6:ffffffff r5:20000013 r4:c061d13c
    [ 33.978905] [<c061d118>] (__pm_runtime_idle) from [<c0643e98>] (ctx_mcasp_probe+0x9a0/0xbdc)
    [ 33.987379] r7:00000000 r6:eeac7c00 r5:eeac7c10 r4:ee37b410
    [ 33.993067] [<c06434f8>] (ctx_mcasp_probe) from [<c06151b4>] (platform_drv_probe+0x58/0xb4)
    [ 34.001455] r10:c0e4883c r9:00000000 r8:00000000 r7:fffffdfb r6:c1037168 r5:ffffffed
    [ 34.009316] r4:eeac7c10
    [ 34.011863] [<c061515c>] (platform_drv_probe) from [<c06135dc>] (driver_probe_device+0x1e4/0x2d0)
    [ 34.020772] r7:c1037168 r6:00000000 r5:c10906ec r4:eeac7c10
    [ 34.026457] [<c06133f8>] (driver_probe_device) from [<c0613774>] (__driver_attach+0xac/0xb0)
    [ 34.034931] r9:c1052000 r8:c0e48834 r7:00000000 r6:eeac7c44 r5:c1037168 r4:eeac7c10
    [ 34.042711] [<c06136c8>] (__driver_attach) from [<c06118e4>] (bus_for_each_dev+0x54/0xa4)
    [ 34.050921] r7:00000000 r6:c06136c8 r5:c1037168 r4:00000000
    [ 34.056608] [<c0611890>] (bus_for_each_dev) from [<c0612fc4>] (driver_attach+0x24/0x28)
    [ 34.064646] r6:c10357e8 r5:ee30c780 r4:c1037168
    [ 34.069286] [<c0612fa0>] (driver_attach) from [<c0612b8c>] (bus_add_driver+0x198/0x21c)
    [ 34.077327] [<c06129f4>] (bus_add_driver) from [<c06140d0>] (driver_register+0x80/0xfc)
    [ 34.085364] r7:c1052000 r6:ffffe000 r5:c0e2d160 r4:c1037168
    [ 34.091051] [<c0614050>] (driver_register) from [<c061510c>] (__platform_driver_register+0x48/0x50)
    [ 34.100133] r5:c0e2d160 r4:00000000
    [ 34.103731] [<c06150c4>] (__platform_driver_register) from [<c0e2d17c>] (ctx_mcasp_driver_init+0x1c/0x20)
    [ 34.113343] [<c0e2d160>] (ctx_mcasp_driver_init) from [<c020179c>] (do_one_initcall+0x4c/0x178)
    [ 34.122084] [<c0201750>] (do_one_initcall) from [<c0e00f54>] (kernel_init_freeable+0x1e4/0x27c)
    [ 34.130821] r8:c0e48834 r7:c1052000 r6:00000007 r5:c0e58fcc r4:c0c738c0
    [ 34.137557] [<c0e00d70>] (kernel_init_freeable) from [<c09193c0>] (kernel_init+0x10/0x118)
    [ 34.145859] r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c09193b0
    [ 34.153718] r4:00000000
    [ 34.156267] [<c09193b0>] (kernel_init) from [<c0207dc8>] (ret_from_fork+0x14/0x2c)
    [ 34.163866] r5:c09193b0 r4:00000000
    [ 34.167458] ---[ end trace 422aca20e4f32de0 ]---
    [ 34.172186] Unhandled fault: asynchronous external abort (0x1211) at 0x00000000
    [ 34.179526] pgd = c0003000
    [ 34.182244] [00000000] *pgd=80000080004003, *pmd=00000000
    [ 34.187687] Internal error: : 1211 [#1] PREEMPT SMP ARM
    [ 34.192933] Modules linked in:
    [ 34.196008] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W 4.9.41 #8
    [ 34.203170] Hardware name: Generic DRA74X (Flattened Device Tree)
    [ 34.209291] task: ee890000 task.stack: ee87a000
    [ 34.213842] PC is at __pm_runtime_idle+0x24/0x88
    [ 34.218477] LR is at ctx_mcasp_probe+0x9a0/0xbdc
    [ 34.223114] pc : [<c061d13c>] lr : [<c0643e98>] psr: 20000013
    [ 34.223114] sp : ee87bd28 ip : ee87bd48 fp : ee87bd44
    [ 34.234641] r10: c0e4883c r9 : 00000000 r8 : 00000000
    [ 34.239888] r7 : 00000000 r6 : eeac7c00 r5 : 00000005 r4 : eeac7c10
    [ 34.246442] r3 : eeac7ce8 r2 : 00000000 r1 : 00000005 r0 : eeac7c10
    [ 34.252998] Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
    [ 34.260161] Control: 30c5387d Table: 80003000 DAC: fffffffd
    [ 34.265931] Process swapper/0 (pid: 1, stack limit = 0xee87a210)
    [ 34.271962] Stack: (0xee87bd28 to 0xee87c000)
    [ 34.276339] bd20: ee37b410 eeac7c10 eeac7c00 00000000 ee87bdd4 ee87bd48
    [ 34.284553] bd40: c0643e98 c061d124 00000000 c0bdc794 00000002 00000000 ee87bd7c 00000002
    [ 34.292766] bd60: eed9d134 00000002 00000080 00000001 ee87bd9c 00000000 ee3bdb90 c0be2af8
    [ 34.300979] bd80: ee3bdc30 eeacacd0 00000001 c0e4883c ee87bdc4 ee87bda0 c0398c9c c039528c
    [ 34.309193] bda0: eeac7c10 00000000 eeac7c18 eeac7c10 ffffffed c1037168 fffffdfb 00000000
    [ 34.317404] bdc0: 00000000 c0e4883c ee87bdf4 ee87bdd8 c06151b4 c0643504 eeac7c10 c10906ec
    [ 34.325617] bde0: 00000000 c1037168 ee87be1c ee87bdf8 c06135dc c0615168 eeac7c10 c1037168
    [ 34.333828] be00: eeac7c44 00000000 c0e48834 c1052000 ee87be3c ee87be20 c0613774 c0613404
    [ 34.342041] be20: 00000000 c1037168 c06136c8 00000000 ee87be64 ee87be40 c06118e4 c06136d4
    [ 34.350253] be40: ee86465c eeac8534 ee864670 c1037168 ee30c780 c10357e8 ee87be74 ee87be68
    [ 34.358466] be60: c0612fc4 c061189c ee87be9c ee87be78 c0612b8c c0612fac c0bef40c ee87be88
    [ 34.366680] be80: c1037168 c0e2d160 ffffe000 c1052000 ee87beb4 ee87bea0 c06140d0 c0612a00
    [ 34.374894] bea0: 00000000 c0e2d160 ee87bec4 ee87beb8 c061510c c061405c ee87bed4 ee87bec8
    [ 34.383107] bec0: c0e2d17c c06150d0 ee87bf4c ee87bed8 c020179c c0e2d16c c0a21f04 effff69e
    [ 34.391321] bee0: 000000d2 c0c74928 ee87bf00 ee87bef8 c024a570 c0e00624 ee87bf34 c0ba87c4
    [ 34.399532] bf00: c0ba8810 00000000 00000006 00000006 c0ba80c8 c0c05bbc c0c738c0 00000000
    [ 34.407744] bf20: 000004dc c0c738c0 00000007 c0c738c0 c0e58fcc 00000007 c1052000 c0e48834
    [ 34.415957] bf40: ee87bf94 ee87bf50 c0e00f54 c020175c 00000006 00000006 00000000 c0e00618
    [ 34.424169] bf60: c0e00618 000000d2 00000000 00000000 c09193b0 00000000 00000000 00000000
    [ 34.432382] bf80: 00000000 00000000 ee87bfac ee87bf98 c09193c0 c0e00d7c 00000000 c09193b0
    [ 34.440595] bfa0: 00000000 ee87bfb0 c0207dc8 c09193bc 00000000 00000000 00000000 00000000
    [ 34.448807] bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
    [ 34.457019] bfe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
    [ 34.465227] Backtrace:
    [ 34.467695] [<c061d118>] (__pm_runtime_idle) from [<c0643e98>] (ctx_mcasp_probe+0x9a0/0xbdc)
    [ 34.476171] r7:00000000 r6:eeac7c00 r5:eeac7c10 r4:ee37b410
    [ 34.481860] [<c06434f8>] (ctx_mcasp_probe) from [<c06151b4>] (platform_drv_probe+0x58/0xb4)
    [ 34.490248] r10:c0e4883c r9:00000000 r8:00000000 r7:fffffdfb r6:c1037168 r5:ffffffed
    [ 34.498110] r4:eeac7c10
    [ 34.500659] [<c061515c>] (platform_drv_probe) from [<c06135dc>] (driver_probe_device+0x1e4/0x2d0)
    [ 34.509570] r7:c1037168 r6:00000000 r5:c10906ec r4:eeac7c10
    [ 34.515255] [<c06133f8>] (driver_probe_device) from [<c0613774>] (__driver_attach+0xac/0xb0)
    [ 34.523729] r9:c1052000 r8:c0e48834 r7:00000000 r6:eeac7c44 r5:c1037168 r4:eeac7c10
    [ 34.531510] [<c06136c8>] (__driver_attach) from [<c06118e4>] (bus_for_each_dev+0x54/0xa4)
    [ 34.539722] r7:00000000 r6:c06136c8 r5:c1037168 r4:00000000
    [ 34.545409] [<c0611890>] (bus_for_each_dev) from [<c0612fc4>] (driver_attach+0x24/0x28)
    [ 34.553447] r6:c10357e8 r5:ee30c780 r4:c1037168
    [ 34.558087] [<c0612fa0>] (driver_attach) from [<c0612b8c>] (bus_add_driver+0x198/0x21c)
    [ 34.566127] [<c06129f4>] (bus_add_driver) from [<c06140d0>] (driver_register+0x80/0xfc)
    [ 34.574165] r7:c1052000 r6:ffffe000 r5:c0e2d160 r4:c1037168
    [ 34.579851] [<c0614050>] (driver_register) from [<c061510c>] (__platform_driver_register+0x48/0x50)
    [ 34.588934] r5:c0e2d160 r4:00000000
    [ 34.592531] [<c06150c4>] (__platform_driver_register) from [<c0e2d17c>] (ctx_mcasp_driver_init+0x1c/0x20)
    [ 34.602143] [<c0e2d160>] (ctx_mcasp_driver_init) from [<c020179c>] (do_one_initcall+0x4c/0x178)
    [ 34.610884] [<c0201750>] (do_one_initcall) from [<c0e00f54>] (kernel_init_freeable+0x1e4/0x27c)
    [ 34.619620] r8:c0e48834 r7:c1052000 r6:00000007 r5:c0e58fcc r4:c0c738c0
    [ 34.626356] [<c0e00d70>] (kernel_init_freeable) from [<c09193c0>] (kernel_init+0x10/0x118)
    [ 34.634658] r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c09193b0
    [ 34.642518] r4:00000000
    [ 34.645066] [<c09193b0>] (kernel_init) from [<c0207dc8>] (ret_from_fork+0x14/0x2c)
    [ 34.652665] r5:c09193b0 r4:00000000
    [ 34.656258] Code: e1a04000 0a00000a f57ff05b e28030d8 (f593f000)
    [ 34.662382] ---[ end trace 422aca20e4f32de1 ]---
    [ 34.667102] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
    [ 34.667102]
    [ 34.676286] CPU1: stopping
    [ 34.679008] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D W 4.9.41 #8
    [ 34.686173] Hardware name: Generic DRA74X (Flattened Device Tree)
    [ 34.692290] Backtrace:
    [ 34.694762] [<c020b3c4>] (dump_backtrace) from [<c020b6a0>] (show_stack+0x18/0x1c)
    [ 34.702366] r7:fa212000 r6:20000193 r5:00000000 r4:c1022a50
    [ 34.708053] [<c020b688>] (show_stack) from [<c04cd4fc>] (dump_stack+0x90/0xa4)
    [ 34.715312] [<c04cd46c>] (dump_stack) from [<c020e990>] (handle_IPI+0x1b8/0x1cc)
    [ 34.722740] r7:fa212000 r6:00000001 r5:00000000 r4:c0e5dddc
    [ 34.728426] [<c020e7d8>] (handle_IPI) from [<c02014a4>] (gic_handle_irq+0x78/0x7c)
    [ 34.736028] r6:ee8b7f40 r5:fa21200c r4:c1003424
    [ 34.740665] [<c020142c>] (gic_handle_irq) from [<c020c238>] (__irq_svc+0x58/0x8c)
    [ 34.748178] Exception stack(0xee8b7f40 to 0xee8b7f88)
    [ 34.753253] 7f40: 00000001 00000000 00000000 c0219640 ffffe000 c100303c 00000002 c10030a4
    [ 34.761465] 7f60: 00000000 00000000 c10030ac ee8b7f9c ee8b7fa0 ee8b7f90 c0208848 c020884c
    [ 34.769674] 7f80: 60000013 ffffffff
    [ 34.773178] r9:ee8b6000 r8:00000000 r7:ee8b7f74 r6:ffffffff r5:60000013 r4:c020884c
    [ 34.780961] [<c020880c>] (arch_cpu_idle) from [<c091e174>] (default_idle_call+0x28/0x34)
    [ 34.789092] [<c091e14c>] (default_idle_call) from [<c026ea88>] (cpu_startup_entry+0x19c/0x218)
    [ 34.797742] [<c026e8ec>] (cpu_startup_entry) from [<c020e550>] (secondary_start_kernel+0x170/0x17c)
    [ 34.806825] r7:c1052250
    [ 34.809371] [<c020e3e0>] (secondary_start_kernel) from [<8020154c>] (0x8020154c)
    [ 34.816800] r7:c1052250 r6:30c0387d r5:00000000 r4:ae85e340
    [ 34.822488] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
    [ 34.822488]

  • That looks mostly like I expected, i.e. generally no errors reported.  I'm surprised that didn't work.

    I think it would be nearly the same, but another change would be inside omap_i2c_xfer to add a delay/sleep there.  For example:

    omap_i2c_wait_for_bb(omap);

    mdelay(1);

    if (omap->set_mpu_wkup_lat != NULL)
    omap->set_mpu_wkup_lat(omap->dev, -1);

    out:
    pm_runtime_mark_last_busy(omap->dev);
    pm_runtime_put_autosuspend(omap->dev);

    If that still doesn't work can you please print the value of the CON register immediately following that mdelay?

  • Morning Brad,

       No joy so far. Hopefully this is what you need. Many thanks, Ron.


    U-Boot 2018.03-rc3 (Jan 18 2021 - 10:34:03 -0500)

    CPU : DRA762-HS ES1.0 ABZ package
    Model: Compunetix AM5748 ID16P
    Board: Compunetix ID16P REV 00
    DRAM: 2 GiB
    MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1
    Loading Environment from MMC... OK
    GUID Partition Table Header signature is wrong: 0x0 != 0x5452415020494645
    part_get_info_efi: *** ERROR: Invalid GPT ***
    GUID Partition Table Header signature is wrong: 0x0 != 0x5452415020494645
    part_get_info_efi: *** ERROR: Invalid Backup GPT ***
    compu1160_dvi_enable: DVI Monitor Not Detected: 1!
    Model Type up to date unknown
    We're red. Let's reset the usb interface to be safe
    Timed out in wait_for_event: status=0000
    Check if pads/pull-ups of bus are properly configured
    compu1160_reset_usb: HIP board detect failed: 1!
    SCSI: SATA link 0 timeout.
    AHCI 0001.0300 32 slots 1 ports 3 Gbps 0x1 impl SATA mode
    flags: 64bit ncq stag pm led clo only pmp pio slum part ccc apst
    scanning bus for devices...
    Found 0 device(s).
    Net: eth0: ethernet@48484000
    Hit any key to stop autoboot: 0
    => boot
    switch to partitions #0, OK
    mmc0 is current device
    SD/MMC found on device 0
    4118628 bytes read in 183 ms (21.5 MiB/s)
    ## Loading kernel from FIT Image at 90000000 ...
    Using 'conf@1' configuration
    Trying 'kernel@1' kernel subimage
    Description: Vanilla Linux kernel
    Type: Kernel Image
    Compression: uncompressed
    Data Start: 0x900000ec
    Data Size: 3898224 Bytes = 3.7 MiB
    Architecture: ARM
    OS: Linux
    Load Address: 0x82000000
    Entry Point: 0x82000000
    Hash algo: sha256
    Hash value: b3f458e5b27fad3c3e31a3821e5bf5ee1946e336788a2d079288d245de757974
    Verifying Hash Integrity ... sha256+ OK
    Authentication passed
    ## Loading fdt from FIT Image at 90000000 ...
    Using 'conf@1' configuration
    Trying 'fdt@1' fdt subimage
    Description: Flattened Device Tree blob
    Type: Flat Device Tree
    Compression: uncompressed
    Data Start: 0x903b7d5c
    Data Size: 109168 Bytes = 106.6 KiB
    Architecture: ARM
    Hash algo: sha256
    Hash value: ced626bcfa4fbe9e4ba2505b480660a3ba849178de7f54eca88ced79646e5353
    Verifying Hash Integrity ... sha256+ OK
    Authentication passed
    Booting using the fdt blob at 0x903b7d5c
    Loading Kernel Image ... OK
    Loading Device Tree to 8ffe2000, end 8ffff954 ... OK
    get_voltrail_opp
    get_voltrail_opp
    get_voltrail_opp

    Starting kernel ...

    [ 0.000000] Booting Linux on physical CPU 0x0
    [ 0.000000] Linux version 4.9.41 (wkanczes@slash) (gcc version 7.3.0 (crosstool-NG crosstool-ng-1.23.0-319-gaca85cb) ) #11 SMP PREEMPT Wed Jan 20 09:09:05 EST 2021
    [ 0.000000] CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=30c5387d
    [ 0.000000] CPU: div instructions available: patching division code
    [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
    [ 0.000000] OF: fdt:Machine model: COMPUNETIX AM5748 ID16P
    [ 0.000000] efi: Getting EFI parameters from FDT:
    [ 0.000000] efi: UEFI not found.
    [ 0.000000] Reserved memory: created CMA memory pool at 0x0000000095800000, size 56 MiB
    [ 0.000000] OF: reserved mem: initialized node ipu2_cma@95800000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created CMA memory pool at 0x0000000099000000, size 64 MiB
    [ 0.000000] OF: reserved mem: initialized node dsp1_cma@99000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created CMA memory pool at 0x000000009d000000, size 32 MiB
    [ 0.000000] OF: reserved mem: initialized node ipu1_cma@9d000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created CMA memory pool at 0x000000009f000000, size 8 MiB
    [ 0.000000] OF: reserved mem: initialized node dsp2_cma@9f000000, compatible id shared-dma-pool
    [ 0.000000] cma: Reserved 24 MiB at 0x00000000fc800000
    [ 0.000000] Memory policy: Data cache writealloc
    [ 0.000000] OMAP4: Map 0x00000000ffd00000 to fe600000 for dram barrier
    [ 0.000000] dra7xxx_check_revision: unknown idcode=0x0bb5002f (hawkeye=0x0000bb50,rev=0x0)
    [ 0.000000] DRA752 ES2.0
    [ 0.000000] percpu: Embedded 13 pages/cpu @eed79000 s22156 r8192 d22900 u53248
    [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 465216
    [ 0.000000] Kernel command line: console=ttyO0,115200n8 omapdss.def_disp=hdmi omapfb.mode=hdmi:1920x1080MR-32@60 usbcore.autosuspend=-1 root=PARTUUID=ce1452b7-02 rw rootfstype=ext4 rootwait 2
    [ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
    [ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
    [ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
    [ 0.000000] Memory: 1646244K/1867776K available (8192K kernel code, 323K rwdata, 2516K rodata, 2048K init, 302K bss, 33116K reserved, 188416K cma-reserved, 1253376K highmem)
    [ 0.000000] Virtual kernel memory layout:
    [ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
    [ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
    [ 0.000000] vmalloc : 0xf0800000 - 0xff800000 ( 240 MB)
    [ 0.000000] lowmem : 0xc0000000 - 0xf0000000 ( 768 MB)
    [ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
    [ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
    [ 0.000000] .text : 0xc0008000 - 0xc0a00000 (10208 kB)
    [ 0.000000] .init : 0xc0e00000 - 0xc1000000 (2048 kB)
    [ 0.000000] .data : 0xc1000000 - 0xc1050e18 ( 324 kB)
    [ 0.000000] .bss : 0xc1052000 - 0xc109d8f4 ( 303 kB)
    [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
    [ 0.000000] Preemptible hierarchical RCU implementation.
    [ 0.000000] Build-time adjustment of leaf fanout to 32.
    [ 0.000000] NR_IRQS:16 nr_irqs:16 16
    [ 0.000000] OMAP clockevent source: timer1 at 32786 Hz
    [ 0.000000] arm_arch_timer: Architected cp15 timer(s) running at 6.14MHz (virt).
    [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x16af5adb9, max_idle_ns: 440795202250 ns
    [ 0.000004] sched_clock: 56 bits at 6MHz, resolution 162ns, wraps every 4398046511023ns
    [ 0.000015] Switching to timer-based delay loop, resolution 162ns
    [ 0.000323] clocksource: 32k_counter: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58327039986419 ns
    [ 0.000331] OMAP clocksource: 32k_counter at 32768 Hz
    [ 0.000762] Console: colour dummy device 80x30
    [ 0.000780] WARNING: Your 'console=ttyO0' has been replaced by 'ttyS0'
    [ 0.000786] This ensures that you still see kernel messages. Please
    [ 0.000792] update your kernel commandline.
    [ 0.000810] Calibrating delay loop (skipped), value calculated using timer frequency.. 12.29 BogoMIPS (lpj=61475)
    [ 0.000823] pid_max: default: 32768 minimum: 301
    [ 0.000925] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
    [ 0.000935] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
    [ 0.001571] CPU: Testing write buffer coherency: ok
    [ 0.001782] /cpus/cpu@0 missing clock-frequency property
    [ 0.001798] /cpus/cpu@1 missing clock-frequency property
    [ 0.001809] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
    [ 0.001825] Setting up static identity map for 0x80200000 - 0x80200060
    [ 0.080052] EFI services will not be available.
    [ 0.170203] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
    [ 0.170292] Brought up 2 CPUs
    [ 0.170305] SMP: Total of 2 processors activated (24.59 BogoMIPS).
    [ 0.170312] CPU: All CPU(s) started in SVC mode.
    [ 0.170848] devtmpfs: initialized
    [ 0.199219] VFP support v0.3: implementor 41 architecture 4 part 30 variant f rev 0
    [ 0.199461] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
    [ 0.199478] futex hash table entries: 512 (order: 3, 32768 bytes)
    [ 0.203290] pinctrl core: initialized pinctrl subsystem
    [ 0.204127] NET: Registered protocol family 16
    [ 0.205015] DMA: preallocated 256 KiB pool for atomic coherent allocations
    [ 0.206069] omap_hwmod: l3_main_2 using broken dt data from ocp
    [ 0.420331] cpuidle: using governor ladder
    [ 0.450358] cpuidle: using governor menu
    [ 0.458540] GPIO line 16 (tfp410_reset_n) hogged as output/high
    [ 0.460260] OMAP GPIO hardware version 0.1
    [ 0.463037] GPIO line 64 (dvi_ls_oe_n) hogged as output/high
    [ 0.463055] GPIO line 65 (dvi_ct_hpd_n) hogged as output/high
    [ 0.463072] GPIO line 88 (usbhub1_reset_n) hogged as output/high
    [ 0.471149] GPIO line 197 (usbhub1_vbus2_n) hogged as output/low
    [ 0.471167] GPIO line 198 (usbhub1_vbus3_n) hogged as output/low
    [ 0.471183] GPIO line 199 (usbhub1_vbus4_n) hogged as output/low
    [ 0.474886] irq: no irq domain found for /ocp/l4@4a000000/scm@2000/pinmux@1400 !
    [ 0.504624] No ATAGs?
    [ 0.504647] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
    [ 0.504656] hw-breakpoint: maximum watchpoint size is 8 bytes.
    [ 0.505011] omap4_sram_init:Unable to allocate sram needed to handle errata I688
    [ 0.505020] omap4_sram_init:Unable to get sram pool needed to handle errata I688
    [ 0.505567] OMAP DMA hardware revision 0.0
    [ 0.552837] omap-dma-engine 4a056000.dma-controller: OMAP DMA engine driver (LinkedList1/2/3 supported)
    [ 0.554170] edma 43300000.edma: memcpy is disabled
    [ 0.558898] edma 43300000.edma: TI EDMA DMA engine driver
    [ 0.563364] omap-iommu 40d01000.mmu: 40d01000.mmu registered
    [ 0.563555] omap-iommu 40d02000.mmu: 40d02000.mmu registered
    [ 0.563710] omap-iommu 58882000.mmu: 58882000.mmu registered
    [ 0.563862] omap-iommu 55082000.mmu: 55082000.mmu registered
    [ 0.564125] omap-iommu 41501000.mmu: 41501000.mmu registered
    [ 0.564320] omap-iommu 41502000.mmu: 41502000.mmu registered
    [ 0.564760] usbcore: registered new interface driver usbfs
    [ 0.564817] usbcore: registered new interface driver hub
    [ 0.564901] usbcore: registered new device driver usb
    [ 0.565855] palmas 0-0058: Irq flag is 0x00000008
    [ 0.594017] omap_i2c_xfer con_reg->8000
    [ 0.596356] omap_i2c_xfer con_reg->8200
    [ 0.599824] omap_i2c_xfer con_reg->8000
    [ 0.602158] omap_i2c_xfer con_reg->8200
    [ 0.605620] omap_i2c_xfer con_reg->8000
    [ 0.607956] omap_i2c_xfer con_reg->8200
    [ 0.611415] omap_i2c_xfer con_reg->8000
    [ 0.613749] omap_i2c_xfer con_reg->8200
    [ 0.617213] omap_i2c_xfer con_reg->8000
    [ 0.620793] omap_i2c_xfer con_reg->8000
    [ 0.624265] omap_i2c_xfer con_reg->8000
    [ 0.627732] omap_i2c_xfer con_reg->8000
    [ 0.631198] omap_i2c_xfer con_reg->8000
    [ 0.634687] omap_i2c_xfer con_reg->8000
    [ 0.638146] omap_i2c_xfer con_reg->8000
    [ 0.638158] palmas 0-0058: Muxing GPIO 2b, PWM 0, LED 0
    [ 0.640490] omap_i2c_xfer con_reg->8200
    [ 0.644547] omap_i2c_xfer con_reg->8000
    [ 0.648007] omap_i2c_xfer con_reg->8000
    [ 0.651469] omap_i2c_xfer con_reg->8000
    [ 0.653805] omap_i2c_xfer con_reg->8200
    [ 0.657266] omap_i2c_xfer con_reg->8000
    [ 0.660725] omap_i2c_xfer con_reg->8000
    [ 0.664201] omap_i2c_xfer con_reg->8000
    [ 0.667667] omap_i2c_xfer con_reg->8000
    [ 0.669999] omap_i2c_xfer con_reg->8200
    [ 0.673466] omap_i2c_xfer con_reg->8000
    [ 0.676982] omap_i2c_xfer con_reg->8000
    [ 0.680672] omap_i2c_xfer con_reg->8000
    [ 0.683007] omap_i2c_xfer con_reg->8200
    [ 0.686472] omap_i2c_xfer con_reg->8000
    [ 0.689930] omap_i2c_xfer con_reg->8000
    [ 0.693408] omap_i2c_xfer con_reg->8000
    [ 0.696875] omap_i2c_xfer con_reg->8000
    [ 0.700388] omap_i2c_xfer con_reg->8000
    [ 0.700597] vtt_fixed: supplied by smps3
    [ 0.704101] omap_i2c_xfer con_reg->8000
    [ 0.707577] omap_i2c_xfer con_reg->8000
    [ 0.711037] omap_i2c_xfer con_reg->8000
    [ 0.713372] omap_i2c_xfer con_reg->8200
    [ 0.716832] omap_i2c_xfer con_reg->8000
    [ 0.720291] omap_i2c_xfer con_reg->8000
    [ 0.723769] omap_i2c_xfer con_reg->8000
    [ 0.727237] omap_i2c_xfer con_reg->8000
    [ 0.729567] omap_i2c_xfer con_reg->8200
    [ 0.733032] omap_i2c_xfer con_reg->8000
    [ 0.736541] omap_i2c_xfer con_reg->8000
    [ 0.740200] omap_i2c_xfer con_reg->8000
    [ 0.743664] omap_i2c_xfer con_reg->8000
    [ 0.745997] omap_i2c_xfer con_reg->8200
    [ 0.749456] omap_i2c_xfer con_reg->8000
    [ 0.752918] omap_i2c_xfer con_reg->8000
    [ 0.756395] omap_i2c_xfer con_reg->8000
    [ 0.759861] omap_i2c_xfer con_reg->8000
    [ 0.762193] omap_i2c_xfer con_reg->8200
    [ 0.765659] omap_i2c_xfer con_reg->8000
    [ 0.769171] omap_i2c_xfer con_reg->8000
    [ 0.772841] omap_i2c_xfer con_reg->8000
    [ 0.775175] omap_i2c_xfer con_reg->8200
    [ 0.778636] omap_i2c_xfer con_reg->8000
    [ 0.782096] omap_i2c_xfer con_reg->8000
    [ 0.785576] omap_i2c_xfer con_reg->8000
    [ 0.789043] omap_i2c_xfer con_reg->8000
    [ 0.791374] omap_i2c_xfer con_reg->8200
    [ 0.794836] omap_i2c_xfer con_reg->8000
    [ 0.798345] omap_i2c_xfer con_reg->8000
    [ 0.802033] omap_i2c_xfer con_reg->8000
    [ 0.805495] omap_i2c_xfer con_reg->8000
    [ 0.807828] omap_i2c_xfer con_reg->8200
    [ 0.811288] omap_i2c_xfer con_reg->8000
    [ 0.814750] omap_i2c_xfer con_reg->8000
    [ 0.818227] omap_i2c_xfer con_reg->8000
    [ 0.819679] random: fast init done
    [ 0.821700] omap_i2c_xfer con_reg->8000
    [ 0.824033] omap_i2c_xfer con_reg->8200
    [ 0.827496] omap_i2c_xfer con_reg->8000
    [ 0.831003] omap_i2c_xfer con_reg->8000
    [ 0.834700] omap_i2c_xfer con_reg->8000
    [ 0.837034] omap_i2c_xfer con_reg->8200
    [ 0.840493] omap_i2c_xfer con_reg->8000
    [ 0.843956] omap_i2c_xfer con_reg->8000
    [ 0.847432] omap_i2c_xfer con_reg->8000
    [ 0.850898] omap_i2c_xfer con_reg->8000
    [ 0.853232] omap_i2c_xfer con_reg->8200
    [ 0.856739] omap_i2c_xfer con_reg->8000
    [ 0.860456] omap_i2c_xfer con_reg->8000
    [ 0.863923] omap_i2c_xfer con_reg->8000
    [ 0.874345] omap_i2c_xfer con_reg->8000
    [ 0.877851] omap_i2c_xfer con_reg->8000
    [ 0.881564] omap_i2c_xfer con_reg->8000
    [ 0.883899] omap_i2c_xfer con_reg->8200
    [ 0.887378] omap_i2c_xfer con_reg->8000
    [ 0.890841] omap_i2c_xfer con_reg->8000
    [ 0.894419] omap_i2c_xfer con_reg->8000
    [ 0.898143] omap_i2c_xfer con_reg->8000
    [ 0.900475] omap_i2c_xfer con_reg->8200
    [ 0.903953] omap_i2c_xfer con_reg->8000
    [ 0.907419] omap_i2c_xfer con_reg->8000
    [ 0.914446] omap_i2c_xfer con_reg->8000
    [ 0.918195] omap_i2c_xfer con_reg->8000
    [ 0.920529] omap_i2c_xfer con_reg->8200
    [ 0.924008] omap_i2c_xfer con_reg->8000
    [ 0.927471] omap_i2c_xfer con_reg->8000
    [ 0.934478] omap_i2c_xfer con_reg->8000
    [ 0.938229] omap_i2c_xfer con_reg->8000
    [ 0.940563] omap_i2c_xfer con_reg->8200
    [ 0.944043] omap_i2c_xfer con_reg->8000
    [ 0.947545] omap_i2c_xfer con_reg->8000
    [ 0.951332] omap_i2c_xfer con_reg->8000
    [ 0.954836] omap_i2c_xfer con_reg->8000
    [ 0.958644] omap_i2c_xfer con_reg->8000
    [ 0.962148] omap_i2c_xfer con_reg->8000
    [ 0.965976] omap_i2c_xfer con_reg->8000
    [ 0.969480] omap_i2c_xfer con_reg->8000
    [ 0.973329] omap_i2c_xfer con_reg->8000
    [ 0.976792] omap_i2c_xfer con_reg->8000
    [ 0.980257] omap_i2c_xfer con_reg->8000
    [ 0.984540] omap_i2c_xfer con_reg->8000
    [ 0.988407] omap_i2c_xfer con_reg->8000
    [ 0.990740] omap_i2c_xfer con_reg->8200
    [ 0.994221] omap_i2c_xfer con_reg->8000
    [ 0.997686] omap_i2c_xfer con_reg->8000
    [ 1.004571] omap_i2c_xfer con_reg->8000
    [ 1.008443] omap_i2c_xfer con_reg->8000
    [ 1.010777] omap_i2c_xfer con_reg->8200
    [ 1.014261] omap_i2c_xfer con_reg->8000
    [ 1.017725] omap_i2c_xfer con_reg->8000
    [ 1.024599] omap_i2c_xfer con_reg->8000
    [ 1.028495] omap_i2c_xfer con_reg->8000
    [ 1.030827] omap_i2c_xfer con_reg->8200
    [ 1.034297] omap_i2c_xfer con_reg->8000
    [ 1.034469] vdd_3v3: supplied by regen1
    [ 1.037970] omap_i2c_xfer con_reg->8000
    [ 1.038010] aic_dvdd_fixed: supplied by vdd_3v3
    [ 1.041779] omap_i2c_xfer con_reg->8000
    [ 1.044455] omap_i2c 48070000.i2c: bus 0 rev0.12 at 100 kHz
    [ 1.045495] omap_i2c 48072000.i2c: bus 1 rev0.12 at 100 kHz
    [ 1.045967] omap_i2c 48060000.i2c: bus 2 rev0.12 at 400 kHz
    [ 1.046325] omap_i2c 4807a000.i2c: bus 3 rev0.12 at 400 kHz
    [ 1.046691] omap_i2c 4807c000.i2c: bus 4 rev0.12 at 400 kHz
    [ 1.046885] media: Linux media interface: v0.10
    [ 1.046932] Linux video capture interface: v2.00
    [ 1.046966] pps_core: LinuxPPS API ver. 1 registered
    [ 1.046973] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
    [ 1.046993] PTP clock support registered
    [ 1.047024] EDAC MC: Ver: 3.0.0
    [ 1.047729] omap-mailbox 4883c000.mailbox: omap mailbox rev 0x400
    [ 1.048023] omap-mailbox 4883e000.mailbox: omap mailbox rev 0x400
    [ 1.048310] omap-mailbox 48840000.mailbox: omap mailbox rev 0x400
    [ 1.048595] omap-mailbox 48842000.mailbox: omap mailbox rev 0x400
    [ 1.048784] omap-mailbox 48846000.mailbox: omap mailbox rev 0x400
    [ 1.049128] Advanced Linux Sound Architecture Driver Initialized.
    [ 1.049833] clocksource: Switched to clocksource arch_sys_counter
    [ 1.060225] NET: Registered protocol family 2
    [ 1.060730] TCP established hash table entries: 8192 (order: 3, 32768 bytes)
    [ 1.060793] TCP bind hash table entries: 8192 (order: 4, 65536 bytes)
    [ 1.060918] TCP: Hash tables configured (established 8192 bind 8192)
    [ 1.060967] UDP hash table entries: 512 (order: 2, 16384 bytes)
    [ 1.060999] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
    [ 1.061196] NET: Registered protocol family 1
    [ 1.061531] RPC: Registered named UNIX socket transport module.
    [ 1.061540] RPC: Registered udp transport module.
    [ 1.061547] RPC: Registered tcp transport module.
    [ 1.061553] RPC: Registered tcp NFSv4.1 backchannel transport module.
    [ 1.062534] hw perfevents: enabled with armv7_cortex_a15 PMU driver, 7 counters available
    [ 1.065420] workingset: timestamp_bits=14 max_order=19 bucket_order=5
    [ 1.072950] squashfs: version 4.0 (2009/01/31) Phillip Lougher
    [ 1.073662] NFS: Registering the id_resolver key type
    [ 1.073683] Key type id_resolver registered
    [ 1.073690] Key type id_legacy registered
    [ 1.073730] ntfs: driver 2.1.32 [Flags: R/O].
    [ 1.074908] bounce: pool size: 64 pages
    [ 1.075056] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246)
    [ 1.075066] io scheduler noop registered
    [ 1.075074] io scheduler deadline registered
    [ 1.075202] io scheduler cfq registered (default)
    [ 1.080852] omap_i2c_xfer con_reg->8000
    [ 1.084695] omap_i2c_xfer con_reg->8000
    [ 1.089754] omap_i2c_xfer con_reg->8000
    [ 1.090821] pinctrl-single 4a003400.pinmux: 282 pins at pa fc003400 size 1128
    [ 1.094000] omap_dss_probe compunetix offset value 0
    [ 1.099286] omap_i2c_xfer con_reg->8000
    [ 1.099459] OMAP DSS rev 6.1
    [ 1.099756] omapdss_dss 58000000.dss: bound 58001000.dispc (ops dispc_component_ops)
    [ 1.100434] omapdss_dss 58000000.dss: bound 58040000.encoder (ops hdmi5_component_ops)
    [ 1.106457] omap_i2c_xfer con_reg->8000
    [ 1.129918] omapdss APPLY error: manager lcd is already connected to an output
    [ 1.130610] fbcvt: 1920x1080@60: CVT Name - 2.073M9-R
    [ 1.159819] Console: switching to colour frame buffer device 240x67
    [ 1.175664] omapfb omapfb: using display 'hdmi' mode 1920x1080
    [ 1.203420] omap_i2c_xfer con_reg->8000
    [ 1.206987] omap_i2c_xfer con_reg->8000
    [ 1.207000] clk-ctx160 1-004b: Found AD9545.
    [ 1.207009] clk-ctx160 1-004b: Vers. 1.002
    [ 1.207080] clk-ctx160 1-004b: XO source is board_49M152_osc.
    [ 1.207161] clk-ctx160 1-004b: REF_BB source is board_20M_clk.
    [ 1.207170] clk-ctx160 1-004b: Found 2 source clocks.
    [ 1.207182] clk-ctx160 1-004b: Creating output clock[0] clk_plogic_ref.
    [ 1.207281] clk-ctx160 1-004b: Creating output clock[1] clk_aux_sys.
    [ 1.207345] clk-ctx160 1-004b: Creating output clock[2] clk_audio_sys.
    [ 1.207386] clk-ctx160 1-004b: Creating output clock[3] clk_otherside_ref.
    [ 1.207425] clk-ctx160 1-004b: Creating output clock[4] clk_audio_framea.
    [ 1.207463] clk-ctx160 1-004b: Creating output clock[5] clk_audio_frameb.
    [ 1.209920] omap_i2c_xfer con_reg->8200
    [ 1.212353] omap_i2c_xfer con_reg->8200
    [ 1.215775] omap_i2c_xfer con_reg->8200
    [ 1.219374] omap_i2c_xfer con_reg->8200
    [ 1.221805] omap_i2c_xfer con_reg->8200
    [ 1.225047] omap_i2c_xfer con_reg->8200
    [ 1.228195] omap_i2c_xfer con_reg->8200
    [ 1.231257] omap_i2c_xfer con_reg->8200
    [ 1.234859] omap_i2c_xfer con_reg->8200
    [ 1.237826] omap_i2c_xfer con_reg->8200
    [ 1.241426] omap_i2c_xfer con_reg->8200
    [ 1.244396] omap_i2c_xfer con_reg->8200
    [ 1.247995] omap_i2c_xfer con_reg->8200
    [ 1.250964] omap_i2c_xfer con_reg->8200
    [ 1.254566] omap_i2c_xfer con_reg->8200
    [ 1.257535] omap_i2c_xfer con_reg->8200
    [ 1.261134] omap_i2c_xfer con_reg->8200
    [ 1.263838] omap_i2c_xfer con_reg->8200
    [ 1.267438] omap_i2c_xfer con_reg->8200
    [ 1.270135] omap_i2c_xfer con_reg->8200
    [ 1.273736] omap_i2c_xfer con_reg->8200
    [ 1.276435] omap_i2c_xfer con_reg->8200
    [ 1.280033] omap_i2c_xfer con_reg->8200
    [ 1.282733] omap_i2c_xfer con_reg->8200
    [ 1.286333] omap_i2c_xfer con_reg->8200
    [ 1.289032] omap_i2c_xfer con_reg->8200
    [ 1.292640] omap_i2c_xfer con_reg->8200
    [ 1.295341] omap_i2c_xfer con_reg->8200
    [ 1.298940] omap_i2c_xfer con_reg->8200
    [ 1.301639] omap_i2c_xfer con_reg->8200
    [ 1.305239] omap_i2c_xfer con_reg->8200
    [ 1.307941] omap_i2c_xfer con_reg->8200
    [ 1.311360] omap_i2c_xfer con_reg->8200
    [ 1.314782] omap_i2c_xfer con_reg->8200
    [ 1.318381] omap_i2c_xfer con_reg->8200
    [ 1.320989] omap_i2c_xfer con_reg->8200
    [ 1.323689] omap_i2c_xfer con_reg->8200
    [ 1.326571] omap_i2c_xfer con_reg->8200
    [ 1.329719] omap_i2c_xfer con_reg->8200
    [ 1.332871] omap_i2c_xfer con_reg->8200
    [ 1.336022] omap_i2c_xfer con_reg->8200
    [ 1.339169] omap_i2c_xfer con_reg->8200
    [ 1.342327] omap_i2c_xfer con_reg->8200
    [ 1.345473] omap_i2c_xfer con_reg->8200
    [ 1.349069] omap_i2c_xfer con_reg->8200
    [ 1.351583] omap_i2c_xfer con_reg->8200
    [ 1.355188] omap_i2c_xfer con_reg->8200
    [ 1.357710] omap_i2c_xfer con_reg->8200
    [ 1.361310] omap_i2c_xfer con_reg->8200
    [ 1.363830] omap_i2c_xfer con_reg->8200
    [ 1.367430] omap_i2c_xfer con_reg->8200
    [ 1.369949] omap_i2c_xfer con_reg->8200
    [ 1.373551] omap_i2c_xfer con_reg->8200
    [ 1.376071] omap_i2c_xfer con_reg->8200
    [ 1.379669] omap_i2c_xfer con_reg->8200
    [ 1.382197] omap_i2c_xfer con_reg->8200
    [ 1.385800] omap_i2c_xfer con_reg->8200
    [ 1.388408] omap_i2c_xfer con_reg->8200
    [ 1.391110] omap_i2c_xfer con_reg->8200
    [ 1.391502] omap_i2c 48072000.i2c: Arbitration lost
    [ 2.410875] omap_i2c_xfer con_reg->8200
    [ 2.625237] random: crng init done
    [ 4.479874] omap_i2c 48072000.i2c: controller timed out
    [ 4.510849] omap_i2c_xfer con_reg->8000
    [ 4.542994] omap_i2c_xfer con_reg->8200
    [ 4.546147] omap_i2c_xfer con_reg->8200
    [ 4.549295] omap_i2c_xfer con_reg->8200
    [ 4.552902] omap_i2c_xfer con_reg->8200
    [ 4.555425] omap_i2c_xfer con_reg->8200
    [ 4.559024] omap_i2c_xfer con_reg->8200
    [ 4.561544] omap_i2c_xfer con_reg->8200
    [ 4.565146] omap_i2c_xfer con_reg->8200
    [ 4.567666] omap_i2c_xfer con_reg->8200
    [ 4.571263] omap_i2c_xfer con_reg->8200
    [ 4.573784] omap_i2c_xfer con_reg->8200
    [ 4.577382] omap_i2c_xfer con_reg->8200
    [ 4.579900] omap_i2c_xfer con_reg->8200
    [ 4.583499] omap_i2c_xfer con_reg->8200
    [ 4.586020] omap_i2c_xfer con_reg->8200
    [ 4.588899] omap_i2c_xfer con_reg->8200
    [ 4.591966] omap_i2c_xfer con_reg->8200
    [ 4.595027] omap_i2c_xfer con_reg->8200
    [ 4.598355] omap_i2c_xfer con_reg->8200
    [ 4.601682] omap_i2c_xfer con_reg->8200
    [ 4.604204] omap_i2c_xfer con_reg->8200
    [ 4.606632] omap_i2c_xfer con_reg->8200
    [ 4.609059] omap_i2c_xfer con_reg->8200
    [ 4.609074] clk-ctx160 1-004b: Added provider.
    [ 4.611501] omap_i2c_xfer con_reg->8200
    [ 4.613931] omap_i2c_xfer con_reg->8200
    [ 4.616361] omap_i2c_xfer con_reg->8200
    [ 4.616751] omap_i2c 48072000.i2c: Arbitration lost
    [ 5.630866] omap_i2c_xfer con_reg->8200
    [ 7.679852] omap_i2c 48072000.i2c: controller timed out
    [ 7.710850] omap_i2c_xfer con_reg->8000
    [ 7.743408] omap_i2c_xfer con_reg->8000
    [ 7.751977] omap_i2c_xfer con_reg->8000
    [ 7.760543] omap_i2c_xfer con_reg->8000
    [ 7.769110] omap_i2c_xfer con_reg->8000
    [ 7.777677] omap_i2c_xfer con_reg->8000
    [ 7.786241] omap_i2c_xfer con_reg->8000
    [ 7.794808] omap_i2c_xfer con_reg->8000
    [ 7.803372] omap_i2c_xfer con_reg->8000
    [ 8.879854] omap_i2c 48072000.i2c: controller timed out
    [ 8.910868] omap_i2c_xfer con_reg->8000
    [ 10.959854] omap_i2c 48072000.i2c: controller timed out
    [ 10.990852] omap_i2c_xfer con_reg->8000
    [ 11.023402] omap_i2c_xfer con_reg->8000
    [ 11.031970] omap_i2c_xfer con_reg->8000
    [ 11.040538] omap_i2c_xfer con_reg->8000
    [ 11.049105] omap_i2c_xfer con_reg->8000
    [ 11.057671] omap_i2c_xfer con_reg->8000
    [ 11.066239] omap_i2c_xfer con_reg->8000
    [ 12.079865] omap_i2c 48072000.i2c: controller timed out
    [ 12.110853] omap_i2c_xfer con_reg->8000
    [ 14.159876] omap_i2c 48072000.i2c: controller timed out
    [ 14.190851] omap_i2c_xfer con_reg->8000
    [ 14.223406] omap_i2c_xfer con_reg->8000
    [ 14.231975] omap_i2c_xfer con_reg->8000
    [ 14.240541] omap_i2c_xfer con_reg->8000
    [ 14.247971] omap_i2c_xfer con_reg->8200
    [ 14.250401] omap_i2c_xfer con_reg->8200
    [ 14.252832] omap_i2c_xfer con_reg->8200
    [ 14.255259] omap_i2c_xfer con_reg->8200
    [ 14.258823] omap_i2c_xfer con_reg->8000
    [ 14.267396] omap_i2c_xfer con_reg->8000
    [ 14.275961] omap_i2c_xfer con_reg->8000
    [ 14.284526] omap_i2c_xfer con_reg->8000
    [ 15.359854] omap_i2c 48072000.i2c: controller timed out
    [ 15.390850] omap_i2c_xfer con_reg->8000
    [ 17.439854] omap_i2c 48072000.i2c: controller timed out
    [ 17.470852] omap_i2c_xfer con_reg->8000
    [ 17.503404] omap_i2c_xfer con_reg->8000
    [ 17.505834] omap_i2c_xfer con_reg->8200
    [ 17.508263] omap_i2c_xfer con_reg->8200
    [ 17.508563] omap_i2c 48072000.i2c: Arbitration lost
    [ 18.520877] omap_i2c_xfer con_reg->8200
    [ 20.569868] omap_i2c 48072000.i2c: controller timed out
    [ 20.600853] omap_i2c_xfer con_reg->8000
    [ 20.600865] clk-ctx160 1-004b: calibrated.
    [ 20.661200] Serial: 8250/16550 driver, 10 ports, IRQ sharing disabled
    [ 20.664596] 4806a000.serial: ttyS0 at MMIO 0x4806a000 (irq = 301, base_baud = 3000000) is a 8250
    [ 22.803509] console [ttyS0] enabled
    [ 22.808688] [drm] Initialized
    [ 22.942522] brd: module loaded
    [ 23.000886] loop: module loaded
    [ 23.007849] omap_i2c_xfer con_reg->8000
    [ 23.011838] at24 0-0050: 256 byte 24c02 EEPROM, writable, 1 bytes/write
    [ 23.018811] ctx160-plogic v0.001: probing
    [ 23.023624] ctx160-plogic ctx160-plogic: Found 8-wide dipswitch.
    [ 23.029802] ctx161-plogic 1-0020: Driver established
    [ 23.063319] omap_i2c_xfer con_reg->8000
    [ 23.070643] omap_i2c_xfer con_reg->8000
    [ 23.077964] omap_i2c_xfer con_reg->8000
    [ 23.085300] omap_i2c_xfer con_reg->8000
    [ 23.089156] ctx161-plogic 1-0020: Hardware found, version=00000000
    [ 23.098848] omap_i2c_xfer con_reg->8000
    [ 23.105049] omap_i2c_xfer con_reg->8200
    [ 23.109023] ctx161-codec 0-0010: assume MAX98091
    [ 23.117191] omap_i2c_xfer con_reg->8000
    [ 23.121100] ctx161-codec 0-0010: Vers. 1.002 Rev. 52
    [ 23.128506] omap_i2c_xfer con_reg->8200
    [ 23.134708] omap_i2c_xfer con_reg->8200
    [ 23.140891] omap_i2c_xfer con_reg->8200
    [ 23.147075] omap_i2c_xfer con_reg->8200
    [ 23.153275] omap_i2c_xfer con_reg->8200
    [ 23.159548] omap_i2c_xfer con_reg->8200
    [ 23.165750] omap_i2c_xfer con_reg->8200
    [ 23.172029] omap_i2c_xfer con_reg->8200
    [ 23.178302] omap_i2c_xfer con_reg->8200
    [ 23.184504] omap_i2c_xfer con_reg->8200
    [ 23.190685] omap_i2c_xfer con_reg->8200
    [ 23.196872] omap_i2c_xfer con_reg->8200
    [ 23.203074] omap_i2c_xfer con_reg->8200
    [ 23.209256] omap_i2c_xfer con_reg->8200
    [ 23.215547] omap_i2c_xfer con_reg->8200
    [ 23.222098] omap_i2c_xfer con_reg->8200
    [ 23.228643] omap_i2c_xfer con_reg->8200
    [ 23.235024] omap_i2c_xfer con_reg->8200
    [ 23.241388] omap_i2c_xfer con_reg->8200
    [ 23.247658] omap_i2c_xfer con_reg->8200
    [ 23.253878] omap_i2c_xfer con_reg->8200
    [ 23.257804] ctx161-codec 1-0010: assume MAX98091
    [ 23.265956] omap_i2c_xfer con_reg->8000
    [ 23.269813] ctx161-codec 1-0010: Vers. 1.002 Rev. 52
    [ 23.277232] omap_i2c_xfer con_reg->8200
    [ 23.283434] omap_i2c_xfer con_reg->8200
    [ 23.289615] omap_i2c_xfer con_reg->8200
    [ 23.295817] omap_i2c_xfer con_reg->8200
    [ 23.302007] omap_i2c_xfer con_reg->8200
    [ 23.308280] omap_i2c_xfer con_reg->8200
    [ 23.314483] omap_i2c_xfer con_reg->8200
    [ 23.320754] omap_i2c_xfer con_reg->8200
    [ 23.327025] omap_i2c_xfer con_reg->8200
    [ 23.333226] omap_i2c_xfer con_reg->8200
    [ 23.339408] omap_i2c_xfer con_reg->8200
    [ 23.345606] omap_i2c_xfer con_reg->8200
    [ 23.351788] omap_i2c_xfer con_reg->8200
    [ 23.357969] omap_i2c_xfer con_reg->8200
    [ 23.364258] omap_i2c_xfer con_reg->8200
    [ 23.370799] omap_i2c_xfer con_reg->8200
    [ 23.377339] omap_i2c_xfer con_reg->8200
    [ 23.383720] omap_i2c_xfer con_reg->8200
    [ 23.390081] omap_i2c_xfer con_reg->8200
    [ 23.396353] omap_i2c_xfer con_reg->8200
    [ 23.402553] omap_i2c_xfer con_reg->8200
    [ 23.406494] ctx161-codec 2-0010: assume MAX98091
    [ 23.443029] omap_i2c_xfer con_reg->8000
    [ 23.446885] ctx161-codec 2-0010: Vers. 1.002 Rev. 52
    [ 23.454079] omap_i2c_xfer con_reg->8200
    [ 23.460037] omap_i2c_xfer con_reg->8200
    [ 23.466000] omap_i2c_xfer con_reg->8200
    [ 23.471978] omap_i2c_xfer con_reg->8200
    [ 23.477940] omap_i2c_xfer con_reg->8200
    [ 23.483943] omap_i2c_xfer con_reg->8200
    [ 23.489903] omap_i2c_xfer con_reg->8200
    [ 23.495887] omap_i2c_xfer con_reg->8200
    [ 23.501875] omap_i2c_xfer con_reg->8200
    [ 23.507837] omap_i2c_xfer con_reg->8200
    [ 23.513814] omap_i2c_xfer con_reg->8200
    [ 23.519774] omap_i2c_xfer con_reg->8200
    [ 23.525751] omap_i2c_xfer con_reg->8200
    [ 23.531712] omap_i2c_xfer con_reg->8200
    [ 23.537699] omap_i2c_xfer con_reg->8200
    [ 23.543769] omap_i2c_xfer con_reg->8200
    [ 23.549819] omap_i2c_xfer con_reg->8200
    [ 23.555843] omap_i2c_xfer con_reg->8200
    [ 23.561855] omap_i2c_xfer con_reg->8200
    [ 23.567836] omap_i2c_xfer con_reg->8200
    [ 23.573817] omap_i2c_xfer con_reg->8200
    [ 23.578053] mc60-audio16 v0.007: probing
    [ 23.582039] mc60-audio16 mc60-audio16: Found DSP clock.
    [ 23.587315] mc60-audio16 mc60-audio16: Found IPU clock.
    [ 23.592602] mc60-audio16 mc60-audio16: Found DSP MMU0 @40d01000.
    [ 23.598674] mc60-audio16 mc60-audio16: Found DSP MMU1 @40d02000.
    [ 23.604781] mc60-audio16 mc60-audio16: Found IPU MMU @58882000.
    [ 23.610741] mc60-audio16 mc60-audio16: Assuming IPU Unicache @58880000.
    [ 23.617554] mc60-audio16 mc60-audio16: Found BIOS timer @fa820000.
    [ 23.623905] mc60-audio16 mc60-audio16: Found BIOS timer @fa088000.
    [ 23.630233] mc60-audio16 mc60-audio16: Reusing frame clock gpio.
    [ 23.636277] mc60-audio16 mc60-audio16: VOIP memory @adc00000 = edc00000 + 00000000
    [ 23.643898] mc60-audio16 mc60-audio16: Xfer memory @adc10000 = edc00000 + 00010000
    [ 23.652133] davinci-mcasp 48460000.mcasp: Configured tdm_slots=2
    [ 23.658169] davinci-mcasp 48460000.mcasp: Counted num_serializer=16
    [ 23.664515] davinci-mcasp 48460000.mcasp: Assigned num_serializer=16
    [ 23.673529] omap_hwmod: mcasp1: _wait_target_ready failed: -16
    [ 23.679393] davinci-mcasp 48460000.mcasp: use pm_runtime_put_sync_suspend() in driver?
    [ 23.687368] ------------[ cut here ]------------
    [ 23.692016] WARNING: CPU: 0 PID: 1 at drivers/bus/omap_l3_noc.c:147 l3_interrupt_handler+0x25c/0x378
    [ 23.701188] 44000000.ocp:L3 Custom Error: MASTER MPU TARGET L4_PER2_P3 (Read): Data Access in User mode during Functional access
    [ 23.712800] Modules linked in:
    [ 23.715877] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.9.41 #11
    [ 23.721908] Hardware name: Generic DRA74X (Flattened Device Tree)
    [ 23.728024] Backtrace:
    [ 23.730497] [<c020b3c4>] (dump_backtrace) from [<c020b6a0>] (show_stack+0x18/0x1c)
    [ 23.738101] r7:00000009 r6:60000193 r5:00000000 r4:c1022a50
    [ 23.743787] [<c020b688>] (show_stack) from [<c04cd4fc>] (dump_stack+0x90/0xa4)
    [ 23.751045] [<c04cd46c>] (dump_stack) from [<c022e34c>] (__warn+0xec/0x104)
    [ 23.758038] r7:00000009 r6:c0bcc164 r5:00000000 r4:ee87baf8
    [ 23.763723] [<c022e260>] (__warn) from [<c022e3a4>] (warn_slowpath_fmt+0x40/0x48)
    [ 23.771239] r9:0000001a r8:eea18b10 r7:c0bcbfd0 r6:00000002 r5:c0bcc090 r4:c0bcc134
    [ 23.779018] [<c022e368>] (warn_slowpath_fmt) from [<c04fcc84>] (l3_interrupt_handler+0x25c/0x378)
    [ 23.787927] r3:eea18980 r2:c0bcc134
    [ 23.791514] r4:80080003
    [ 23.794062] [<c04fca28>] (l3_interrupt_handler) from [<c027f954>] (__handle_irq_event_percpu+0x6c/0x134)
    [ 23.803583] r10:c101a92f r9:eea19300 r8:00000017 r7:ee87bc1c r6:00000000 r5:eea19300
    [ 23.811445] r4:eea18e80
    [ 23.813992] [<c027f8e8>] (__handle_irq_event_percpu) from [<c027fa40>] (handle_irq_event_percpu+0x24/0x60)
    [ 23.823689] r10:c0e4883c r9:ee87a000 r8:ee808000 r7:00000001 r6:c100906c r5:eea19300
    [ 23.831549] r4:eea19300
    [ 23.834095] [<c027fa1c>] (handle_irq_event_percpu) from [<c027fabc>] (handle_irq_event+0x40/0x64)
    [ 23.843003] r5:eea19360 r4:eea19300
    [ 23.846598] [<c027fa7c>] (handle_irq_event) from [<c0283108>] (handle_fasteoi_irq+0xc4/0x194)
    [ 23.855159] r7:00000001 r6:c100906c r5:eea19360 r4:eea19300
    [ 23.860846] [<c0283044>] (handle_fasteoi_irq) from [<c027ebb4>] (generic_handle_irq+0x2c/0x3c)
    [ 23.869494] r7:00000001 r6:00000000 r5:00000000 r4:c0e5dddc
    [ 23.875179] [<c027eb88>] (generic_handle_irq) from [<c027f13c>] (__handle_domain_irq+0x64/0xbc)
    [ 23.883918] [<c027f0d8>] (__handle_domain_irq) from [<c020146c>] (gic_handle_irq+0x40/0x7c)
    [ 23.892306] r9:ee87a000 r8:fa213000 r7:fa212000 r6:ee87bcd8 r5:fa21200c r4:c1003424
    [ 23.900085] [<c020142c>] (gic_handle_irq) from [<c020c238>] (__irq_svc+0x58/0x8c)
    [ 23.907599] Exception stack(0xee87bcd8 to 0xee87bd20)
    [ 23.912671] bcc0: eeac7c10 00000005
    [ 23.920884] bce0: 00000000 eeac7ce8 eeac7c10 00000005 eeac7c00 00000000 00000000 00000000
    [ 23.929098] bd00: c0e4883c ee87bd44 ee87bd48 ee87bd28 c0643e98 c061d13c 20000013 ffffffff
    [ 23.937311] r9:ee87a000 r8:00000000 r7:ee87bd0c r6:ffffffff r5:20000013 r4:c061d13c
    [ 23.945095] [<c061d118>] (__pm_runtime_idle) from [<c0643e98>] (ctx_mcasp_probe+0x9a0/0xbdc)
    [ 23.953566] r7:00000000 r6:eeac7c00 r5:eeac7c10 r4:ee37b410
    [ 23.959254] [<c06434f8>] (ctx_mcasp_probe) from [<c06151b4>] (platform_drv_probe+0x58/0xb4)
    [ 23.967640] r10:c0e4883c r9:00000000 r8:00000000 r7:fffffdfb r6:c1037168 r5:ffffffed
    [ 23.975501] r4:eeac7c10
    [ 23.978049] [<c061515c>] (platform_drv_probe) from [<c06135dc>] (driver_probe_device+0x1e4/0x2d0)
    [ 23.986958] r7:c1037168 r6:00000000 r5:c10906ec r4:eeac7c10
    [ 23.992645] [<c06133f8>] (driver_probe_device) from [<c0613774>] (__driver_attach+0xac/0xb0)
    [ 24.001120] r9:c1052000 r8:c0e48834 r7:00000000 r6:eeac7c44 r5:c1037168 r4:eeac7c10
    [ 24.008900] [<c06136c8>] (__driver_attach) from [<c06118e4>] (bus_for_each_dev+0x54/0xa4)
    [ 24.017113] r7:00000000 r6:c06136c8 r5:c1037168 r4:00000000
    [ 24.022799] [<c0611890>] (bus_for_each_dev) from [<c0612fc4>] (driver_attach+0x24/0x28)
    [ 24.030835] r6:c10357e8 r5:ee30c780 r4:c1037168
    [ 24.035475] [<c0612fa0>] (driver_attach) from [<c0612b8c>] (bus_add_driver+0x198/0x21c)
    [ 24.043516] [<c06129f4>] (bus_add_driver) from [<c06140d0>] (driver_register+0x80/0xfc)
    [ 24.051555] r7:c1052000 r6:ffffe000 r5:c0e2d160 r4:c1037168
    [ 24.057243] [<c0614050>] (driver_register) from [<c061510c>] (__platform_driver_register+0x48/0x50)
    [ 24.066326] r5:c0e2d160 r4:00000000
    [ 24.069923] [<c06150c4>] (__platform_driver_register) from [<c0e2d17c>] (ctx_mcasp_driver_init+0x1c/0x20)
    [ 24.079534] [<c0e2d160>] (ctx_mcasp_driver_init) from [<c020179c>] (do_one_initcall+0x4c/0x178)
    [ 24.088276] [<c0201750>] (do_one_initcall) from [<c0e00f54>] (kernel_init_freeable+0x1e4/0x27c)
    [ 24.097012] r8:c0e48834 r7:c1052000 r6:00000007 r5:c0e58fcc r4:c0c738c0
    [ 24.103750] [<c0e00d70>] (kernel_init_freeable) from [<c0919368>] (kernel_init+0x10/0x118)
    [ 24.112050] r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c0919358
    [ 24.119909] r4:00000000
    [ 24.122458] [<c0919358>] (kernel_init) from [<c0207dc8>] (ret_from_fork+0x14/0x2c)
    [ 24.130057] r5:c0919358 r4:00000000
    [ 24.133649] ---[ end trace 33ad25b351a6eb4e ]---
    [ 24.138364] Unhandled fault: asynchronous external abort (0x1211) at 0x00000000
    [ 24.145705] pgd = c0003000
    [ 24.148423] [00000000] *pgd=80000080004003, *pmd=00000000
    [ 24.153863] Internal error: : 1211 [#1] PREEMPT SMP ARM
    [ 24.159109] Modules linked in:
    [ 24.162186] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W 4.9.41 #11
    [ 24.169439] Hardware name: Generic DRA74X (Flattened Device Tree)
    [ 24.175559] task: ee890000 task.stack: ee87a000
    [ 24.180109] PC is at __pm_runtime_idle+0x24/0x88
    [ 24.184745] LR is at ctx_mcasp_probe+0x9a0/0xbdc
    [ 24.189382] pc : [<c061d13c>] lr : [<c0643e98>] psr: 20000013
    [ 24.189382] sp : ee87bd28 ip : ee87bd48 fp : ee87bd44
    [ 24.200909] r10: c0e4883c r9 : 00000000 r8 : 00000000
    [ 24.206155] r7 : 00000000 r6 : eeac7c00 r5 : 00000005 r4 : eeac7c10
    [ 24.212710] r3 : eeac7ce8 r2 : 00000000 r1 : 00000005 r0 : eeac7c10
    [ 24.219266] Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
    [ 24.226430] Control: 30c5387d Table: 80003000 DAC: fffffffd
    [ 24.232199] Process swapper/0 (pid: 1, stack limit = 0xee87a210)
    [ 24.238231] Stack: (0xee87bd28 to 0xee87c000)
    [ 24.242608] bd20: ee37b410 eeac7c10 eeac7c00 00000000 ee87bdd4 ee87bd48
    [ 24.250822] bd40: c0643e98 c061d124 00000000 c0bdc794 00000002 00000000 ee87bd7c 00000002
    [ 24.259036] bd60: eed9d134 00000002 00000080 00000001 ee87bd9c 00000000 ee3bdb90 c0be2af8
    [ 24.267250] bd80: ee3bdc30 eeacacd0 00000001 c0e4883c ee87bdc4 ee87bda0 c0398c9c c039528c
    [ 24.275463] bda0: eeac7c10 00000000 eeac7c18 eeac7c10 ffffffed c1037168 fffffdfb 00000000
    [ 24.283677] bdc0: 00000000 c0e4883c ee87bdf4 ee87bdd8 c06151b4 c0643504 eeac7c10 c10906ec
    [ 24.291891] bde0: 00000000 c1037168 ee87be1c ee87bdf8 c06135dc c0615168 eeac7c10 c1037168
    [ 24.300104] be00: eeac7c44 00000000 c0e48834 c1052000 ee87be3c ee87be20 c0613774 c0613404
    [ 24.308318] be20: 00000000 c1037168 c06136c8 00000000 ee87be64 ee87be40 c06118e4 c06136d4
    [ 24.316531] be40: ee86465c eeac8534 ee864670 c1037168 ee30c780 c10357e8 ee87be74 ee87be68
    [ 24.324744] be60: c0612fc4 c061189c ee87be9c ee87be78 c0612b8c c0612fac c0bef40c ee87be88
    [ 24.332957] be80: c1037168 c0e2d160 ffffe000 c1052000 ee87beb4 ee87bea0 c06140d0 c0612a00
    [ 24.341171] bea0: 00000000 c0e2d160 ee87bec4 ee87beb8 c061510c c061405c ee87bed4 ee87bec8
    [ 24.349384] bec0: c0e2d17c c06150d0 ee87bf4c ee87bed8 c020179c c0e2d16c c0a21f04 effff69e
    [ 24.357598] bee0: 000000d2 c0c74928 ee87bf00 ee87bef8 c024a570 c0e00624 ee87bf34 c0ba87c4
    [ 24.365810] bf00: c0ba8810 00000000 00000006 00000006 c0ba80c8 c0c05bbc c0c738c0 00000000
    [ 24.374023] bf20: 000004dc c0c738c0 00000007 c0c738c0 c0e58fcc 00000007 c1052000 c0e48834
    [ 24.382237] bf40: ee87bf94 ee87bf50 c0e00f54 c020175c 00000006 00000006 00000000 c0e00618
    [ 24.390449] bf60: c0e00618 000000d2 00000000 00000000 c0919358 00000000 00000000 00000000
    [ 24.398662] bf80: 00000000 00000000 ee87bfac ee87bf98 c0919368 c0e00d7c 00000000 c0919358
    [ 24.406875] bfa0: 00000000 ee87bfb0 c0207dc8 c0919364 00000000 00000000 00000000 00000000
    [ 24.415088] bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
    [ 24.423301] bfe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
    [ 24.431509] Backtrace:
    [ 24.433977] [<c061d118>] (__pm_runtime_idle) from [<c0643e98>] (ctx_mcasp_probe+0x9a0/0xbdc)
    [ 24.442451] r7:00000000 r6:eeac7c00 r5:eeac7c10 r4:ee37b410
    [ 24.448139] [<c06434f8>] (ctx_mcasp_probe) from [<c06151b4>] (platform_drv_probe+0x58/0xb4)
    [ 24.456526] r10:c0e4883c r9:00000000 r8:00000000 r7:fffffdfb r6:c1037168 r5:ffffffed
    [ 24.464389] r4:eeac7c10
    [ 24.466937] [<c061515c>] (platform_drv_probe) from [<c06135dc>] (driver_probe_device+0x1e4/0x2d0)
    [ 24.475848] r7:c1037168 r6:00000000 r5:c10906ec r4:eeac7c10
    [ 24.481537] [<c06133f8>] (driver_probe_device) from [<c0613774>] (__driver_attach+0xac/0xb0)
    [ 24.490012] r9:c1052000 r8:c0e48834 r7:00000000 r6:eeac7c44 r5:c1037168 r4:eeac7c10
    [ 24.497792] [<c06136c8>] (__driver_attach) from [<c06118e4>] (bus_for_each_dev+0x54/0xa4)
    [ 24.506006] r7:00000000 r6:c06136c8 r5:c1037168 r4:00000000
    [ 24.511693] [<c0611890>] (bus_for_each_dev) from [<c0612fc4>] (driver_attach+0x24/0x28)
    [ 24.519731] r6:c10357e8 r5:ee30c780 r4:c1037168
    [ 24.524371] [<c0612fa0>] (driver_attach) from [<c0612b8c>] (bus_add_driver+0x198/0x21c)
    [ 24.532411] [<c06129f4>] (bus_add_driver) from [<c06140d0>] (driver_register+0x80/0xfc)
    [ 24.540450] r7:c1052000 r6:ffffe000 r5:c0e2d160 r4:c1037168
    [ 24.546138] [<c0614050>] (driver_register) from [<c061510c>] (__platform_driver_register+0x48/0x50)
    [ 24.555220] r5:c0e2d160 r4:00000000
    [ 24.558817] [<c06150c4>] (__platform_driver_register) from [<c0e2d17c>] (ctx_mcasp_driver_init+0x1c/0x20)
    [ 24.568428] [<c0e2d160>] (ctx_mcasp_driver_init) from [<c020179c>] (do_one_initcall+0x4c/0x178)
    [ 24.577168] [<c0201750>] (do_one_initcall) from [<c0e00f54>] (kernel_init_freeable+0x1e4/0x27c)
    [ 24.585905] r8:c0e48834 r7:c1052000 r6:00000007 r5:c0e58fcc r4:c0c738c0
    [ 24.592641] [<c0e00d70>] (kernel_init_freeable) from [<c0919368>] (kernel_init+0x10/0x118)
    [ 24.600941] r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c0919358
    [ 24.608803] r4:00000000
    [ 24.611352] [<c0919358>] (kernel_init) from [<c0207dc8>] (ret_from_fork+0x14/0x2c)
    [ 24.618952] r5:c0919358 r4:00000000
    [ 24.622545] Code: e1a04000 0a00000a f57ff05b e28030d8 (f593f000)
    [ 24.628671] ---[ end trace 33ad25b351a6eb4f ]---
    [ 24.633389] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
    [ 24.633389]
    [ 24.642571] CPU1: stopping
    [ 24.645292] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D W 4.9.41 #11
    [ 24.652544] Hardware name: Generic DRA74X (Flattened Device Tree)
    [ 24.658661] Backtrace:
    [ 24.661132] [<c020b3c4>] (dump_backtrace) from [<c020b6a0>] (show_stack+0x18/0x1c)
    [ 24.668737] r7:fa212000 r6:20000193 r5:00000000 r4:c1022a50
    [ 24.674425] [<c020b688>] (show_stack) from [<c04cd4fc>] (dump_stack+0x90/0xa4)
    [ 24.681683] [<c04cd46c>] (dump_stack) from [<c020e990>] (handle_IPI+0x1b8/0x1cc)
    [ 24.689111] r7:fa212000 r6:00000001 r5:00000000 r4:c0e5dddc
    [ 24.694797] [<c020e7d8>] (handle_IPI) from [<c02014a4>] (gic_handle_irq+0x78/0x7c)
    [ 24.702399] r6:ee8b7f40 r5:fa21200c r4:c1003424
    [ 24.707038] [<c020142c>] (gic_handle_irq) from [<c020c238>] (__irq_svc+0x58/0x8c)
    [ 24.714552] Exception stack(0xee8b7f40 to 0xee8b7f88)
    [ 24.719626] 7f40: 00000001 00000000 00000000 c0219640 ffffe000 c100303c 00000002 c10030a4
    [ 24.727840] 7f60: 00000000 00000000 c10030ac ee8b7f9c ee8b7fa0 ee8b7f90 c0208848 c020884c
    [ 24.736051] 7f80: 60000013 ffffffff
    [ 24.739556] r9:ee8b6000 r8:00000000 r7:ee8b7f74 r6:ffffffff r5:60000013 r4:c020884c
    [ 24.747337] [<c020880c>] (arch_cpu_idle) from [<c091e11c>] (default_idle_call+0x28/0x34)
    [ 24.755468] [<c091e0f4>] (default_idle_call) from [<c026ea88>] (cpu_startup_entry+0x19c/0x218)
    [ 24.764119] [<c026e8ec>] (cpu_startup_entry) from [<c020e550>] (secondary_start_kernel+0x170/0x17c)
    [ 24.773200] r7:c1052250
    [ 24.775748] [<c020e3e0>] (secondary_start_kernel) from [<8020154c>] (0x8020154c)
    [ 24.783175] r7:c1052250 r6:30c0387d r5:00000000 r4:ae85e340
    [ 24.788864] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
    [ 24.788864]

  • HI Brad,

       As a hardware engineer and not having a lot of knowledge about SW nuances, looking at the failure mode and reviewing the TRM, this is my opinion of what I think might be the root cause. Just looking at this from a hardware perspective. It might provoke a useful thought. If not, sorry for wasting your time.

       Since I2C accesses seem to work when the chip is warm; that indicates a HW race condition. Seems we managed to preempt that in u-boot by the delay. However, apparently not so easy for Linux.

       Looking at Figure 24-21. HS I2C Master Receiver Mode, Polling Method, in F/S and HS Modesin the TRM, the core loop is Read  I2Ci.I2C_IRQSTATUS_RAW register and save value. All decisions are based off of what is read. For something to be read wrong but fixes as the chip gets warmer smacks of either setup and hold issues or synchronizer issues. Either way, let us assume that 2C_IRQSTATUS_RAW reads wrong. The only real way to "fix" that is to read the register two or three times in succession. If the second read equals the first read then you done but if it's different, a third read is required and that should equal the second read. Without knowing how the synchronization is done, this is largely conjecture.

       A dumb question: are you sure that you really are reading 2C_IRQSTATUS_RAW and not a cached value? Register, of course should not be cached.

       In my mind there is no question the underlying issue is a hardware race condition. However, if we can "fix" it for u-boot, I'm sure we'll find a way in Linux

       Just a few thoughts! All your efforts are very much appreciated

           Thanks for you attention,

                 Ron.

  • Ronald Edgar said:
     Since I2C accesses seem to work when the chip is warm; that indicates a HW race condition.

    This is new info to me.  If I missed it in a previous response I apologize.  Are you saying this behavior is temperature dependent?  Was that true for the u-boot issue too?

    I also want to note that it is concerning to me that you're seeing issues even with the Linux driver.  For u-boot, it's not used quite as heavily as the Linux driver, so I was willing to believe that maybe there's a corner case where a timing error was lurking.  For Linux however, that seems very hard to imagine.  This I2C driver is heavily used across many SoC's and across many, many customer designs.  And furthermore the register outputs you've shared so far do not look like they suffered from the same issue as the u-boot driver.

    All this being said, I'm not sure this software path is the right way to go.  Even from the start it seemed highly unusual that the issue popped up when you moved the device to a different I2C controller.  Question: How many other I2C devices are on this same I2C bus?  Is it possible to remove the other devices?  While I know you performed a test where you showed conclusively that the ACK had a different level than the rest of the data, that really only showed that it was not your slave device sending the ACK.  It did not conclusively prove it was the AM57xx.  Removing the other devices would prove that.  Removing the other devices would also change the bus capacitance -- could that be related?

    In short, while some decent progress has been made, it is still not entirely clear to me whether this issue is hardware or software related, and it's also not clear if there is a single issue being observed or perhaps a combination of multiple issues impacting us at different times.

    Ronald Edgar said:
     A dumb question: are you sure that you really are reading 2C_IRQSTATUS_RAW and not a cached value? Register, of course should not be cached.

    That would be a major error in the driver.  I don't think that's the case.  We'd be seeing issues everywhere.

  • Hi Brad,

       Sorry for not emphasizing the temperature aspect more, Guess it got lost in the avalanche. The script in u-boot allows us to initialize the ad9545, not normally done there. When Linux starts, the AD9545 is initialized about 4 seconds in. Both scenarios fail the same way, with SDA stuck low. You were able to "fix" the issue in u-boot with the delay.

       We are seeing the issue across three boards: I am seeing it, the SW engineer is seeing it and I have another engineer who is seeing it. Normally this board is attached to another, which changes the loading on the I2C bus.

       For the old board, the AD9545 was on I2C4 together with 2 temp. sensors, a fiber link device and a DVI device. I2C4 was entirely on the processor board and we never saw any issues

       On the new board, the AD9545 is on I2C2, a bus which connects to the second board. On the processor board also live the same two temp. sensors. On the attached board also live a codec, an FPGA (I2C slave implemented) another temperature sensor and a USB power device. It seems to make little difference if the second board is there or not, the same failure occurs.

       On the old board, i2C2 had no loads on the processor board but had three loads on the attached board: a temp. sensor, a codec, and an FPGA, same as paragraph above

       I do not have the means at home to remove devices from the board. If you think the removal is really needed, I will take the board into work and get it done. There is no software way of disabling them. But, like you thought about my cache idea, I think someone else responding with the ACK is extremely unlikely. If the SI were bad, I could believe maybe, but the the SI is good!

       It's not like the I2C fails on the first I2C transaction. The most recent capture I have of the I2C bus shows major activity for nearly 200mS, that's a lot of transaction. And then it finally fails.

       Let me know if you have more questions.

           Thanks, Ron

  • HI Brad,

       This is interesting. Usually I see the failure on a read but here I see it on a write. And we're one clock short. Something not right!

    And now the board is warm, it runs fine. Here is what should have happened:

    What do you make of this?

    Ron

  • What voltage do you measure on your VDD rail?  Can you also read the 32-bit register at address 0x4A0025F4?  If you can get to a Linux prompt you can do "devmem2 0x4A0025F4" to read it.  Or in u-boot it would be the md command.

  • HI Brad,

       VDD - there are so many of them. DO you mean core? Or do you mean something else? Ron

  • I'm talking about "VDD" specifically, yes the core domain: