TPS6594-Q1: TPS6594 output abnormally in temperature cycle tests

Hi An,

The device expert is currently out of the office until 8/9. I will look into finding another, and try to look at finding an answer in the meantime. Please expect a delay in response accordingly.

Thanks,

Nicholas

Hello An,

What is VIO_IN when the rails turn off? If VIO_IN isn't provided, I2C communication will not work. 

What is happening before the PMIC powers down? Does the failure happen during boot or is the PMIC actively running and then suddenly shut down?

Hello Michael,

As below,

--What is VIO_IN when the rails turn off? If VIO_IN isn't provided, I2C communication will not work. 

After the abnormal has occurred, I tested the VIO_IN and it's normal.

--What is happening before the PMIC powers down? Does the failure happen during boot or is the PMIC actively running and then suddenly shut down?

To be honest, We have not yet observed the instantaneous changes that are happening with the anomaly. We just observed the results for several times. However, I can give you more information about tests. We power on ECU and set champer temperature from -40°~+85°. During the test, ECU will not be powered off. So it maybe a suddenly shut down. Meanwhile, I'm not sure if there are abnormal resets. I'll try to reproduce it and do more records.

Additionally, could you have some directions or speculations? For example, do you think PMIC has entered the safe recovery state？Does I2C work if PMIC is in recovery state. 

Hello An,

Do you know what temperature the system suddenly shuts down?

Do you lose I2C communication with both PMICs?

An said:

Additionally, could you have some directions or speculations? For example, do you think PMIC has entered the safe recovery state？Does I2C work if PMIC is in recovery state. 

I2C communication should work in recovery state. I suspect that something has caused the PMIC to execute a digital reset which does cause loss of communication. Digital resets are extremely rarely when PMIC layout guidelines are followed. Of particular importance is the placement and layout of capacitors for the LDO_VINT.

Hello Michael，

We read its currents and board temperature every 3 minutes. And we find the most abnormality occurs in low temperature, which is from -40℃~-20℃, but there are also some other temperatures, such as 40℃ and 90℃.

Some boards failure occurs in our PV test and I cannot debug it much. Then I try to reproduce it and actually this abnormality occurs twice. I cannot communication with PMIC_A for these 2 tests, but I can communicate with PMIC_B for the first test. I'm not sure there are some test deviation. And I'll reproduce it more to get more details and confirmation.

Then I have a question for your suspection. If there are a digital reset, will PMIC shut down immediately? Could it reset and power on again? How can we to test for it? Monitor the wave on LDO_VIN?

An said:

Then I have a question for your suspection. If there are a digital reset, will PMIC shut down immediately? Could it reset and power on again? How can we to test for it?

Yes the PMIC will shutdown all it's rails immediately. It can not recover without a power cycle of the VCCA supply. In the case where PMIC_B still communicates, please read the interrupt registers. Confirm SPMI_ERR_INT is set to 1. 

Hello Michael，

I have reproduced this abnormality again. I try to communicate with PMICs. PMIC_A can not be accessed, while PMIC_B is ok. I dump all the registers of PMIC_B and show it to you as below. Meanwhile, the register values in normal state are also attached.

Actually, I see SPMI_ERR_INT is set to 1. Then I have some new questions based on this test results.

Q1: When SPMI_ERR_INT is set to 1, can we be sure there must be a SPMI err and this err causes PMICs shutdown？I just would like to confirm if there is another possibility，for example PMIC's shutdown occurs firstly due to something and SPMI_ERR_INT is set secondly.

Q2: We see the I2C of PMIC_A can not work and the one of PMIC_B is ok. So can we think there is something wrong with PMIC_A, while PMIC_B is ok?

Q3: When SPMI_ERR_INT is set to 1, can we be sure there is a digital reset? Or We can only be highly skeptical, and the layout of capacitors for the LDO_VINT is a more likely direction if digital reset occurs?

Q4:Is it possible that this issue is related to PMIC power dissipation. For example, if the power dissipation of TDA4 is higher, this problem is more likely to occur.

To be honest, we have finished 2 rounds of the whole temperature cycle test last year. Then we changed some peripheral chips based on some functions requiremests and do a new temperature cycle test. However, there was no such issue before. But we found the issues occured just during this test. 3 of 6 samples has failed. There is no difference from HW design. It is only possible that the material batch and the build time are different. Another difference is that the power consumption of TDA4 has increased by about 3W.

Hello An,

You raised a few questions and I will try to answer. Overall, I believe the customer is experiencing some increased noise being injected into the PMIC caused by the higher load demands of the TDA4.  You stated the HW design was tested over a year ago with the lighter load demands. Can you confirm that the layout and placement of the LDO_VINT cap follows the guidelines outlined in the data sheet?

An said:

Q1: When SPMI_ERR_INT is set to 1, can we be sure there must be a SPMI err and this err causes PMICs shutdown？I just would like to confirm if there is another possibility，for example PMIC's shutdown occurs firstly due to something and SPMI_ERR_INT is set secondly.

I believe PMIC_A is experiencing a digital reset due to the before mentioned noise. During a digital reset, all the rails shut down and both I2C and SPMI communication is lost until a power cycle. PMIC-B reports an SPMI Error because it no longer "sees" PMIC-A. This causes PMIC-B to execute an orderly shutdown. 

An said:

Q2: We see the I2C of PMIC_A can not work and the one of PMIC_B is ok. So can we think there is something wrong with PMIC_A, while PMIC_B is ok?

Refer to my last answer.

An said:

Q3: When SPMI_ERR_INT is set to 1, can we be sure there is a digital reset? Or We can only be highly skeptical, and the layout of capacitors for the LDO_VINT is a more likely direction if digital reset occurs?

To be certain that a digital reset is happening, disable the clock monitor of PMIC-A after boot by setting CLKMON_EN=0 after boot. If the PMIC does not shut down during your test, then that would confirm that PMIC-A was experiencing a digital reset.

An said:

Q4:Is it possible that this issue is related to PMIC power dissipation. For example, if the power dissipation of TDA4 is higher, this problem is more likely to occur.

The increased load of TDA4 could be the reason the issue is seen now but not before. The increased loads means heavier load transients and more phase shedding and adding during operation. This increases the noise on the board. To reduce the noise, set BUCK1 on PMIC-A and PMIC-B to be in forced multi-phase mode, BUCK1_FPWM_MP=1 after boot. 

Hello Michael,

Thank you for your reply. We are checking all the points you mentioned.

Then, could you share you email address? We are communicating with Daviel and John, who are FAE in North American. I'm not sure you have got in touch. I would also like to loop you in the mails. By the way, what's your location? Maybe we can set up an online meeting if necessary. 

Hello Michael,

Your reply is really detailed one by one. Many thanks.

I check the layout and placement of the LDO_VINT cap. As the limitation of actual layout, the placement of the LDO_VINT cap can not completely meet the requirements of guideline. So we cannot rule out the influence of LDO_Vin cap until now. It maybe a direction.

Additional, we cannot reproduce this abnormality stably. It may occur once a day or once 2~3 days. It is not very convenient for investigating issue.

 Also, new questions from me.

Q1: You mentioned there may be a digital reset if the placement of LDO_VIN cap is wrong. And you also mention the increased load of TDA4 and increased noise could be the reason. Is there something related?

In other words, if the noise of Buck increases, can it directly lead to a digital reset? Or the buck noise increases → LDO noise increases → digital reset？

We would like to know if it is a combination of two factors. Or just a seperate factor that has no correlation and each can lead to this issue.

Q2: We did reproduce this abnormality most (more than 80%) at low temperature (-40℃), and once at 20 ℃, once at 60℃. Does it seem reasonable, combined with your suspection? What are the expected changes for PMIC or PMIC noise?

Q3: To confirm for your answer to the last Q3. I'm not very clear. You mean if we set CLKMON_EN=0, then PMIC_B(?) will not shutdown if a digital reset happen?

Q4: Setting BUCK1_FPWM_MP=1 may reduce the noise. We only need to set buck1, not care buck3 and others. Can all the buck follow buck1?

Q5: Is there any potential reason for digital reset? Just would like to learn more if you have more experience and ideas to share.

An said:

Q1: You mentioned there may be a digital reset if the placement of LDO_VIN cap is wrong. And you also mention the increased load of TDA4 and increased noise could be the reason. Is there something related?

In other words, if the noise of Buck increases, can it directly lead to a digital reset? Or the buck noise increases → LDO noise increases → digital reset？

We would like to know if it is a combination of two factors. Or just a seperate factor that has no correlation and each can lead to this issue.

Increased load on TDA4 means larger load transient events on CORE and CPU rails  --> Larger transient events means more phase shedding and adding --> more noise caused by bucks --> increased noise on LDO_VINT and internal clock --> increased chance of digital reset.

So it is a combination of the two factors.

An said:

Q2: We did reproduce this abnormality most (more than 80%) at low temperature (-40℃), and once at 20 ℃, once at 60℃. Does it seem reasonable, combined with your suspection? What are the expected changes for PMIC or PMIC noise?

Yes, this follows my suspicion because at -40C, the capacitors behave differently. So I would expect to see an increased rate of failure at cold temps.

An said:

Q3: To confirm for your answer to the last Q3. I'm not very clear. You mean if we set CLKMON_EN=0, then PMIC_B(?) will not shutdown if a digital reset happen?

PMIC-A is experiencing a digital reset not PMIC-B. PMIC-B powers down because it lost communication with PMIC-A. Execute the write on PMIC-A and see if it still experiences a digital reset.

An said:

Q4: Setting BUCK1_FPWM_MP=1 may reduce the noise. We only need to set buck1, not care buck3 and others. Can all the buck follow buck1?

On the primary phase on the multiphase rail needs to have this bit set. The other bucks in rail will follow.

An said:

Q5: Is there any potential reason for digital reset? Just would like to learn more if you have more experience and ideas to share.

Glitches on one of the monitored internal clocks is the primary reason for the digital reset. LDO_VINT powers these clocks, so noise on the LDO can negatively impact the internal clocks.

Hello Michael,

According to what we have aligned, we do such a test which is adding an electronic load to PMIC_A and PMIC_B buck1 to see what will occur. I would like to share the result with you, which make me confused.

All the tests are in -40℃ temperature.

1. Just add 3 ampere electronic load to each PMIC's buck1.

PMIC will reset every 1-5 minutes. Have to say, we see PMIC reset, not shutdown.

2. Add 3 ampere electronic load to each PMIC's buck1. And add an external I2C tool, which will provide a pull-up voltage to I2C SCL/SDA of PMICs.

Then we see PMIC can shut down after a few minutes and will not power up again until next power cycle. (There is no external I2C tools in our PV test)

3. Just add 1 ampere electronic load to each PMIC's buck1. Nothing happens. Anyway we plan let it run the whole night.

And quesions as follow.

Q1: Could you have some ideas for the test results? To be honest, maybe we don't have much resource to investigate the issues introduced due to the debugging methods. However, if you have any ideas, it's excellent.

Q2: Could you double confirm there will be a shutdown not reset if a digital reset happen? I believe you are right.

Q3: Refer to Q2. If yes, that means there is a digital reset in our PV test. But the increased load may not be the key reason. I remember something and would like to share with you. We have ever encountered an issue for TI9702. TI9702 will output a  wrong CSI CLK in low temperature. It's because of the PLL of TI9702 is not stable enough. We fix it by changing a register configuration to make 9702 PLL more stable. Tyler Matthews is the supporter. 

From my understanding, the issue occurs just because of CLK works not well. Maybe power noises and others. I'm not sure for this. So in other words, could we have some ways to make CLK more stable? 

Q4: You mentioned there will not a shutdown if we set CLKMON_EN=0. What influence would this lead to if we could solve the shutdown problem by it? Any other ideas you can share and let us try?

Really a thorny issue. I'm sorry to trouble you again and again.

Hello An,

An said:

1. Just add 3 ampere electronic load to each PMIC's buck1.

PMIC will reset every 1-5 minutes. Have to say, we see PMIC reset, not shutdown.

Do you have any scope captures of the PMIC outputs and VCCA input when this happens? The PMIC will make a max of 15 recovery attempts unless VCCA drops low enough to reset the recovery counter. 

After just one or two of these reset events, try turning off electronic load, raising the temperature to 25degC, then connect to the I2C without turning off your board? I want to confirm this is the same problem you described earlier. 

An said:

Q2: Could you double confirm there will be a shutdown not reset if a digital reset happen? I believe you are right.

A diigital reset will cause a shutdown and loss of communication. But if VCCA is pulled low as well, PMIC might experience a VCCA_UV event and make a recovery attempt. 

An said:

Q4: You mentioned there will not a shutdown if we set CLKMON_EN=0. What influence would this lead to if we could solve the shutdown problem by it? Any other ideas you can share and let us try?

With the clock monitor disabled, the PMIC will not react to glitches on the internal clocks/oscillators. If a shutdown does not occur, this means that any noise getting into the PMIC is likely very small. If a shutdown still occurs either there is a lot of noise entering the PMIC or there is a different cause for the problem.

Replied as follow:

1. I havn't tested the waveform of 3V3 under 3A electronic load, but tested it under 1A electronic load and it is stable.

Then recovery counter is not applicable for us. It will be cleared once TDA4 is power on because of our software. I dump the registers after PMICs reset and there are no abnoraml register values. This looks like the registers of PMIC have been cleared. I am worried about damaging the hardware due to too high load. So we didn't continue testing with 3A electronic load.

2.Let's return to the initial PV issue. I'm not sure if we can reproduce this problem by increasing the electronic load externally. But on the other hand, I have made register modifications to our initial issue ECU.

I made two modifications for both PMICs: one is to set BUCK1_FPWM_MP=1, and the other is to set CLKMON_EN=0. The result is also shared with you. We made this change last Friday and the abnormality has not recurred until now. According to the previous frequency, it should appear once a day or once every two days.

I have to confirm with you for more details. I think we can  set BUCK1_FPWM_MP=1 and treat it as a configuration optimization. However, can we set CLKMON_EN=0 for our ECU. What I mean is does it bring any other issues or potential risks？

Let's follow up with this via email.