AM5718: IO Calibration issue

Hi,

Can you mention your SDK version, SR of the two ICs and OS.

Hi Sir

SDK  :  processor-sdk-linux-04.01.00

kernel  :    Linux version 4.9.41-gc6c5f1afae

BOOT LOADER:  2017.01-gf8adbe225f

New and old IC are the same AM5718AABCX, hardware version is 2.0.

What do you mean "SR"?

Could I skip " I/O calibration"?

Is there any risk if I skip " I/O calibration"?

Thanks

Daniel

Hi,

Daniel Cheng1 said:

What do you mean "SR"?

By SR I mean Silicon Revision. Can you mention the Silicon Revision of both IC and also, which silicon revision is  working and which one is not.

Hi Sir

New and old IC both are the same AM5718AABCX, silicon version is 2.0.

Thanks

Daniel

Hi,

Daniel Cheng1 said:

New and old IC both are the same AM5718AABCX, silicon version is 2.0.

If SOCs are same then its difficult to point out the issue.

• Can you check your hardware connection
  • while placing new SOC may be balls are not touching the pad properly.
  • Are you trying the two SOCs on two different boards.
    • can you explain about your setup environment.
• Also, please share the failure logs.

Hi Sir

My customer do many test to verify if they can skip "I/O calibration".

The test result is good.

Is there any risk or side effort if they skip "recalibrate iodelay"?

Thanks

Daniel

Hi Sir

Do you have any comments?

Thanks

Daniel

Can you confirm that sysboot[14] pulled low on the board, per TRM:

Hi Sirs

My customer skip "IO recalibration" on custom board.

They do many test to verify if it is ok to skip.

The test result is fine and custom board work well.

They want to know that is there any risk or side effort if skip "IO recalibration" ?

Thanks

Daniel

Hi Sirs

Is there any update?

Thanks

Daniel

The risk of skipping IO recalibration is that the IO timings listed the datasheet cannot be guaranteed. They could be shifted by several ns.

Hi B.C.

Sysboot[14] is pull low

Sysboot[15] / GPMC_D15 is CTRL_CORE_PAD_GPMC_AD15 on the TRM(spruhz7i)?

The Reg CTRL_CORE_PAD_GPMC_AD15 value is 0x 00 05 00 00 which is printed on the function __autoboot() from the uboot.

Thanks

Daniel

Thanks for confirming sysboot[14].

Can you please read the following IODELAYCONFIG registers before performing IO recalibration and report the values for both a passing and failing case:

CONFIG_REG_0 0x4844 A00C

CONFIG_REG_2 0x4844 A014

CONFIG_REG_3 0x4844 A018

CONFIG_REG_4 0x4844 A01C

HI B.C.

pass case:

scale_vcores(): CONFIG_REG_0=0x0
scale_vcores(): CONFIG_REG_2=0x21d2
scale_vcores(): CONFIG_REG_3=0xff00c23f
scale_vcores(): CONFIG_REG_4=0xff012ac1

scale_vcores(): CONFIG_REG_0=0x0
scale_vcores(): CONFIG_REG_2=0x21d2
scale_vcores(): CONFIG_REG_3=0xff00c165
scale_vcores(): CONFIG_REG_4=0xff052aa8

scale_vcores(): CONFIG_REG_0=0x0
scale_vcores(): CONFIG_REG_2=0x21d2
scale_vcores(): CONFIG_REG_3=0xff00c15b
scale_vcores(): CONFIG_REG_4=0xff042aa2

fail case:

scale_vcores(): CONFIG_REG_0=0x0
scale_vcores(): CONFIG_REG_2=0x21d2
scale_vcores(): CONFIG_REG_3=0xff00c164
scale_vcores(): CONFIG_REG_4=0xff022a3c

scale_vcores(): CONFIG_REG_0=0x0
scale_vcores(): CONFIG_REG_2=0x21d2
scale_vcores(): CONFIG_REG_3=0xff00c60d
scale_vcores(): CONFIG_REG_4=0xff032b67

scale_vcores(): CONFIG_REG_0=0x0
scale_vcores(): CONFIG_REG_2=0x21d2
scale_vcores(): CONFIG_REG_3=0xff00c535
scale_vcores(): CONFIG_REG_4=0xff022b44

Thanks

Daniel

Hi B.C.

What is the purpose of "IO recalibration" ?

Is there anything else to do besides "IODELAY config" optimization about ""IO recalibration"?

Thanks

Daniel

Hi B.C.

Could you help me to update this issue?

Thanks

Daniel

Thanks for providing the IO Delay register values. The values are similar in the passing and failing cases, which means that the IO Delay mechanisms themselves are OK in the failing case.

Can you determine where exactly in the IO Delay Recalibration sequence the device is failing? Is it stuck polling the PRM_IO_PMCTRL[1] ISOCLK_STATUS bit as part of the Isolation/De-isolation subsequences?

https://www.ti.com/lit/pdf/sprac44 is a good Appnote that gives some additional background on IO recalibration.

Hi B.C.

You are right.

The  ISOCLK_STATUS  bit will fail on PRM_IO_PMCTRL[1].

maybe isolate fail or de-isolate or both fail.

arch/arm/mach-omap2/omap5/dra7xx_iodelay.c:19
--------------------------------------------------------------------------------
static int isolate_io(u32 isolate)
{

if (isolate) {

clrsetbits_le32((*ctrl)->control_pbias, SDCARD_PWRDNZ,

SDCARD_PWRDNZ);

clrsetbits_le32((*ctrl)->control_pbias, SDCARD_BIAS_PWRDNZ,

SDCARD_BIAS_PWRDNZ);

}

/* Override control on ISOCLKIN signal to IO pad ring. */

clrsetbits_le32((*prcm)->prm_io_pmctrl, PMCTRL_ISOCLK_OVERRIDE_MASK,

PMCTRL_ISOCLK_OVERRIDE_CTRL);

if (!wait_on_value(PMCTRL_ISOCLK_STATUS_MASK, PMCTRL_ISOCLK_STATUS_MASK,

    (u32 *)(*prcm)->prm_io_pmctrl, LDELAY))

return ERR_DEISOLATE_IO << isolate;

/* Isolate/Deisolate IO */

clrsetbits_le32((*ctrl)->ctrl_core_sma_sw_0, CTRL_ISOLATE_MASK,

isolate << CTRL_ISOLATE_SHIFT);

/* Dummy read to add delay t > 10ns */

readl((*ctrl)->ctrl_core_sma_sw_0);

/* Return control on ISOCLKIN to hardware */

clrsetbits_le32((*prcm)->prm_io_pmctrl, PMCTRL_ISOCLK_OVERRIDE_MASK,

    PMCTRL_ISOCLK_NOT_OVERRIDE_CTRL);

if (!wait_on_value(PMCTRL_ISOCLK_STATUS_MASK,

0 << PMCTRL_ISOCLK_STATUS_SHIFT,

(u32 *)(*prcm)->prm_io_pmctrl, LDELAY))

return ERR_DEISOLATE_IO << isolate;

return 0;

}

Code will fail on 

!wait_on_value(PMCTRL_ISOCLK_STATUS_MASK,

0 << PMCTRL_ISOCLK_STATUS_SHIFT,

(u32 *)(*prcm)->prm_io_pmctrl, LDELAY)

Thanks

Daniel

Please check if you are encountering the issue described in errata advisory i890 (pasted below for reference). In addition, please verity the integrity of all IO power supply rails during the isolation sequence. The sequence will not complete if all IO supplies are not maintained within their operating limits.

-------------------------------

Errata Advisory i890

MMC1 IOs and PBIAS Must Be Powered-Up Before Isolation

DESCRIPTION

IO Isolation, as described in TRM section “Isolation Requirements”, may fail if the MMC1 IOs and PBIAS are not powered-up.

WORKAROUND

Power-up the MMC1 IOs and PBIAS before starting the Isolation Sequence. This can be done by setting the CTRL_CORE_CONTROL_PBIAS[27] SDCARD_BIAS_PWRDNZ and CTRL_CORE_CONTROL_PBIAS[26] SDCARD_IO_PWRDNZ bits to 1.

Hi B.C.

It's different from the description of Errata Advisory i890. I've printed the value of CTRL_CORE_CONTROL_PBIAS (0x4A00 2E00) before and after isolation, the [26] and [27] bits are both 1 (the register reads 0x0d20 0000 before and after isolate_io).

I inserted some printf()s inside the isolate_io() function like this:

static int isolate_io(u32 isolate)
{
    printf("isolate_io(%d):0 (*ctrl)->control_pbias=0x%x\n", isolate, (*ctrl)->control_pbias);
       
    printf("isolate_io(%d):1 *((*ctrl)->control_pbias)=0x%x\n", isolate, *(int*)((*ctrl)->control_pbias));

    // existing code: write CTRL_CORE_CONTROL_PBIAS [26] and [27] bits
    if (isolate) {
        clrsetbits_le32((*ctrl)->control_pbias, SDCARD_PWRDNZ,
                SDCARD_PWRDNZ);
        clrsetbits_le32((*ctrl)->control_pbias, SDCARD_BIAS_PWRDNZ,
                SDCARD_BIAS_PWRDNZ);
    }

    printf("isolate_io(%d):2 *((*ctrl)->control_pbias)=0x%x\n", isolate, *(int*)((*ctrl)->control_pbias));

    /* Override control on ISOCLKIN signal to IO pad ring. */
    clrsetbits_le32((*prcm)->prm_io_pmctrl, PMCTRL_ISOCLK_OVERRIDE_MASK,
            PMCTRL_ISOCLK_OVERRIDE_CTRL);
           
    printf("isolate_io(%d):3 *((*ctrl)->control_pbias)=0x%x\n", isolate, *(int*)((*ctrl)->control_pbias));

    // BAD DEVICES FAILED HERE

    if (!wait_on_value(PMCTRL_ISOCLK_STATUS_MASK, PMCTRL_ISOCLK_STATUS_MASK,
               (u32 *)(*prcm)->prm_io_pmctrl, LDELAY))
        return ERR_DEISOLATE_IO << isolate;
       
    printf("isolate_io(%d):4 *((*ctrl)->control_pbias)=0x%x\n", isolate, *(int*)((*ctrl)->control_pbias));

    /* Isolate/Deisolate IO */
    clrsetbits_le32((*ctrl)->ctrl_core_sma_sw_0, CTRL_ISOLATE_MASK,
            isolate << CTRL_ISOLATE_SHIFT);
    /* Dummy read to add delay t > 10ns */
    readl((*ctrl)->ctrl_core_sma_sw_0);

    printf("isolate_io(%d):5 *((*ctrl)->control_pbias)=0x%x\n", isolate, *(int*)((*ctrl)->control_pbias));

    /* Return control on ISOCLKIN to hardware */
    clrsetbits_le32((*prcm)->prm_io_pmctrl, PMCTRL_ISOCLK_OVERRIDE_MASK,
            PMCTRL_ISOCLK_NOT_OVERRIDE_CTRL);
           
    printf("isolate_io(%d):6 *((*ctrl)->control_pbias)=0x%x\n", isolate, *(int*)((*ctrl)->control_pbias));

    if (!wait_on_value(PMCTRL_ISOCLK_STATUS_MASK,
               0 << PMCTRL_ISOCLK_STATUS_SHIFT,
               (u32 *)(*prcm)->prm_io_pmctrl, LDELAY))
        return ERR_DEISOLATE_IO << isolate;

    printf("isolate_io(%d):7 *((*ctrl)->control_pbias)=0x%x\n", isolate, *(int*)((*ctrl)->control_pbias));

    return 0;
}

On good device, the console output is:

U-Boot SPL 2017.01-dirty (Aug 18 2021 - 19:54:25)
DRA722-GP ES2.0
isolate_io(1):0 (*ctrl)->control_pbias=0x4a002e00
isolate_io(1):1 *((*ctrl)->control_pbias)=0xd200000
isolate_io(1):2 *((*ctrl)->control_pbias)=0xd200000
isolate_io(1):3 *((*ctrl)->control_pbias)=0xd200000
isolate_io(1):4 *((*ctrl)->control_pbias)=0xd200000�k����ѕ}���0):5 *((*ctrl)->control_pbias)=0xd200000
isolate_io(0):6 *((*ctrl)->control_pbias)=0xd200000
isolate_io(0):7 *((*ctrl)->control_pbias)=0xd200000
Trying to boot from MMC1

...

I assume that the garbled output is normal due to the IO isolation. The device boots normally and the output is consistent and every single time.

On bad devices, the console output looked like this:

U-Boot SPL 2017.01-dirty (Aug 18 2021 - 19:54:25)
DRA722-GP ES2.0
isolate_io(1):0 (*ctrl)->control_pbias=0x4a002e00
isolate_io(1):1 *((*ctrl)->control_pbias)=0xd200000
isolate_io(1):2 *((*ctrl)->control_pbias)=0xd200000
isolate_io(1):3 *((*ctrl)->control_pbias)=0xd200000

IODELAY: Isolation of Device IOs failed

...

All of the 3 bad devices failed polling PMCTRL_ISOCLK_STATUS of prm_io_pmctrl:

wait_on_value(PMCTRL_ISOCLK_STATUS_MASK, PMCTRL_ISOCLK_STATUS_MASK, (u32 *)(*prcm)->prm_io_pmctrl, LDELAY))

Is there any other possibility that could lead this failure?

Thanks for your kindly help

Daniel

Thanks for checking the MMC errata. A power integrity issue on any of the IO power supply rails could also cause the issue. Can you probe the IO power supplies to the SoC to confirm there are no dips below the minimum allowed voltage?

Also, what is the timeout value of LDELAY? Can you try increasing this value to give more time for it to complete?

Hi B.C.

I'll have to discuss with our EE engineer to see what we could do about the power rails.

The LDELAY is: clock.h:20:#define LDELAY 1000000

The wait_on_value function is looked like this:

/*********************************************************************
 * wait_on_value() - common routine to allow waiting for changes in
 *   volatile regs.
 *********************************************************************/
u32 wait_on_value(u32 read_bit_mask, u32 match_value, void *read_addr, u32 bound)
{
    u32 i = 0, val;
    do {
        ++i;
        val = readl((u32)read_addr) & read_bit_mask;
        if (val == match_value)
            return 1;
        if (i == bound)
            return 0;
    } while (1);
}

I multiply the LDELAY by 1000 and that doesn't work. The polling runs for around 8 seconds before the system powers itself off automatically.

The console output is:

U-Boot SPL 2017.01-dirty (Aug 20 2021 - 14:27:02)
DRA722-GP ES2.0
isolate_io(1):0 (*ctrl)->control_pbias=0x4a002e00
isolate_io(1):1 *((*ctrl)->control_pbias)=0xd200000
isolate_io(1):2 *((*ctrl)->control_pbias)=0xd200000
isolate_io(1):3 *((*ctrl)->control_pbias)=0xd200000

And then it goes off. It seems that the bit is never gonna be set on a bad device.

Thanks

Daniel

Hi B.C.

voltage is normal.

Could you give me some suggestions?

Thanks

Daniel

Hello Daniel,

Can you do an AB BA swap?  I.e., take one of the "failing" systems and desolder, resolder that part onto a "good" PCB.  And take the unit from the "good" PCB and resolder onto the original "failing" PCB.

Thanks,

Kyle

Hi Kyle

The issue is follow IC. Could you give me some suggestions?

Thanks

Daniel

Daniel,

Our main suspicion is that one or more of the power rails to the SoC are operating out of spec.  Initial focus should be on vddshv*.

The isolation ring will fail to isolate/de-isolate if the IO power supplies are out of spec.

Can you collect waveforms of the power supply sequencing during initial power on as well as the DC voltage when you execute the isolation software?

Thanks,

Kyle

Hi Kyle

Thanks

Daniel

Hi Kyle

Is there any update?

Thanks

Daniel

Hello Daniel,

Can you return 2 of the failing SoCs to TI for evaluation?  

https://www.ti.com/support-quality/additional-information/customer-returns.html

Thanks,

Kyle

Hi Kyle

I will return 2 of the failing SoCs to TI for evaluation.

Thanks

Daniel