PMICs not powering up (TPS65941213-Q1 and TPS65941111-Q1) DRA829

Hello,

Was the register readback from a problematic PMIC taken after the PMIC made power up attempts?

If you put an o-scope probe on GPIO9 of PMIC-A, do you see it toggle 15 times before staying low? 

Using an o-scope, can you tell how far into the TO_ACTIVE sequence the PMIC gets before powering down?

Hi Mr Gambrill,

Thank you for your prompt response.

Query 1:

The registers were read by keeping the poweron/EN pin of PMIC-A low. So the PMICs were powered with 3.3V, but were not enabled. So, the power up sequence might not have been initiated

Query 2:

I have observed only one pulse on the GPIO pin 9 of the PMIC-A and no retries were observed after that.

Query 3:

Pulses of corresponding voltage were observed from GPIO-9 to LDO-4 of PMIC-B in the TO_ACTIVE sequence. An exception being a small voltage pulse of 0.8V is observed on Buck 5 of PMIC-A. Following 1.8V on LDO-4 of PMIC-B, no further pulses were observed on Buck 1,2,3 of PMIC-A and the subsequent sequence

The corresponding snaps of the signals are enclosed to this post

Additional Info:

We have observed the following:

• 1.8V on VRTC and VINT LDOs were fine on both the PMICs
• There is no I2C communication between PMIC-A and B on the GPIO-5 and GPIO-6 (Communication was observed on the working cards - signal snaps attached)

Scope_180424.zip

Kartheek G said:

Query 1:

The registers were read by keeping the poweron/EN pin of PMIC-A low. So the PMICs were powered with 3.3V, but were not enabled. So, the power up sequence might not have been initiated

I need to read the interrupt registers AFTER a failed power up sequence so I can identify which rail had a fault.

Kartheek G said:

Query 2:

I have observed only one pulse on the GPIO pin 9 of the PMIC-A and no retries were observed after that.

This means only a single power up attempt was done by the PMIC. 

Feedback on scope captures:

1. Please try to capture multiple rails/signals on scope capture together with a horizontal zoom level that allows for more details. Being able to tell the relative timing between events is impossible with only single channel shots. 
2. GPIO-5 and GPIO-6 are SPMI communication between the PMICs. On a failing power up, do you see any communication traffic when 3.3V is first applied to VCCA?
3. I noticed that PMIC-A LDO4 does not turn all the way off, can you zoom in and measure the voltage level it ramps down to? This could be the reason the PMIC does not make multiple recovery attempts. 

Hi Mr Gambrill,

This time, we have observed 15 retries on GPIO-9 at power up

1) The images of the signals in scope taken 4 at a time are enclosed to the mail. The time line matches with the TO_ACTIVE status FSM time line. Please refer the attachment "TO_ACTIVE timeline.pdf"

2) There are multiple sets of clock and data signals on the GPIO 5 and GPIO 6 lines immediately after power ON. This continued till the 15 retries on GPIO9. After that, no clock and data signals were observed

3) The PMIC-A LDO4 ramps down to 0.2V immediately after the pulse goes down. And after a some time falls to 0.1V

I have enclosed the status of registers after power failure in comparison with the registers of successfully working PMICs.

Please refer the attachment "Register Comparison.xls"

Note: In the status registers, Buck 5 of PMIC-A has indicated over current tripping. In relation to it, we have checked and changed the caps and other components, but found no difference in the behavior

Thank you

TO_ACTIVE_timeline.pdf

Register Comparison.xls

Hello Kartheek,

I apologize for the delay...

Looking at your most recent register comparison, it appears that the nPWRON_LONG interrupt was triggered. The TPS65941213+TPS65941111 solution is configured by default for an ENABLE signal instead of nPWRON push button. This was not in previous register dumps, is this a correct read out for this register? Has there been any changes to the PMIC?

Are the below register reads from the TPS65941111 consistent on non-working boards? Also the I2C CRC is not enabled by default, so the CRC error is unexpected. 

Hello Mr Gambrill,

Sorry for the delayed feedback

1) There is a small correction. I have misread the register 0x6A of PMIC-1 in the previous comparison of the working and problematic PMICs' registers (refer to file Register Comparison.xls in the previous post)

I have interpreted its value as 0xA0 instead of 0x0A

The register says "Recovery count has reached the threshold and LBIST or ABIST has detected an error".

My apologies for this.

2) We have tested the two non working boards by power cycling each board approx 10 times. The register values are consistent in each card for all the 10 cycles.

But there is a difference between the registers read in card 1 (the registers of which are sent in the previous post) and card 2 (other problematic card)

There is a change in register 0x5B of PMIC-2. But the other registers highlighted i.e., 0x68 and 0x6A are the same.

But, the I2C1/SPI CRC error triggered in both the cards.

The register read out is enclosed to the post. Also, the comparison between registers of card 1 and card 2 is highlighted. The working card register list is provided only for reference.

Thank you

Comparison_2 not working cards.xlsx

Kartheek G said:

1) There is a small correction. I have misread the register 0x6A of PMIC-1 in the previous comparison of the working and problematic PMICs' registers (refer to file Register Comparison.xls in the previous post)

I have interpreted its value as 0xA0 instead of 0x0A

Did you mean register 0x67 for the comparison in the previous post? This would correspond to RECOV_INT and BIST_FAIL_INT and aligns with your recent testing.

Kartheek G said:

2) We have tested the two non working boards by power cycling each board approx 10 times. The register values are consistent in each card for all the 10 cycles.

During power cycling, is it just turning the VSYS power off and on? Does the processor attempt to execute any software during this cycling?

Kartheek G said:

There is a change in register 0x5B of PMIC-2.

The readings from 0x5B and 0x5D do not line up. Firstly, the reserved bits are reading back "1". Secondly, it says that BUCK3_4_INT is high, which means that 0x5D should NOT read back 0x00. 

Hello Mr Gambrill,

1) Even in the previous case and the present test case, the register 0x67 of PMIC-A was read as 0x0A, which corresponds to RECOV_INT and BIST_FAIL_INT. It is consistent with the power up retries of the PMIC, but not sure of BIST FAIL

2) Yes, the input supply is powered ON and OFF. In the two boards, the PMICs have not powered on yet. The boot files and OS are programmed through the UART and USB of the processor, after the processor is powered ON. Hence, the boot memory can be considered as empty.

3) Yes, as you have observed, the PMIC interrupt indication register ( INT_BUCK Register at 0x5B)) shows interrupt triggered in BUCK 3 or 4. But reading the INT_BUCK3_4 Register at 0x5D shows no interrupt triggered.

Also, at 0x60, LDO1_OV_INT-LDO1 over voltage interrupt triggered and at 0x61, LDO4_SC_INT – LDO4 output below threshold interrupts were triggered but the same is not indicated in INT_LDO_VMON Register at 0x5F

Thank you

Referring to "Comparison_2 not working cards.xlsx"..

Kartheek G said:

3) Yes, as you have observed, the PMIC interrupt indication register ( INT_BUCK Register at 0x5B)) shows interrupt triggered in BUCK 3 or 4. But reading the INT_BUCK3_4 Register at 0x5D shows no interrupt triggered.

Bad Card 1 PMIC-A has BUCK5_ILIM and BIST_FAIL. The BUCK5_ILIM would not cause a recovery attempt, but the BIST failure would. PMIC-B has bits reading back as  "1" that should read back as "0" since they are reserved bits. 

Bad Card 2 PMIC-A has LDO4_ILIM and BIST_FAIL. Again, the LDO4_ILIM would not cause a recovery attempt, but the BIST failure would. PMIC-B has LDO4_SC, LDO1_OV and multiple reserved bits reading back as "1"

On this design, is the I2C1 bus shared with any devices besides the two PMICs and the processor? Specifically is there anything else with a device address of 0x4C?

Hello Mr Gambrill,

Sorry for the delay.

The PMICs are connected on an I2C bus with provision of I2C switch on the bus

The I2C switch can be configured to connect to the processor or an external I2C entity

For our testing, we are configuring the switch to connect to a controller card (external I2C entity).

So, the PMICs and this controller card form an exclusive I2C network independent of other I2C devices.

Thank you

Hello Kartheek,

The multiple reserved bits reading back as "1" concerns me as they should read as "0". 

If the controller is interpreting the readback correctly, then they are a likely source of the BIST_FAIL. BIST includes checking of regmap CRC.

Considering you have some working boards and some non-working, have you tried swapping units from a non-working board onto a working board?