This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

66AK2H06: DDR3 data corruption

Part Number: 66AK2H06

Hi,

This is additional question to the original post.
https://e2e.ti.com/support/processors/f/791/p/877712/3333133#3333133
Customer had DDR3 data corruption issue on the 1st proto board.
We had some discussion and concluded there was leveling issue due to board layout.

Based on the discussion, customer modified the layout and now 2nd proto board is ready.
But the issue still persists and data corruption occurs at DDR3-1600 operation.

Please see attached excel sheet.

DDR3 Leveling waveforms.xlsx
- “Data corruption” sheet shows the symptom. DDR3 memory contents changed each time memory window is refreshed.
The issue is depending of DDR3 leveling results.

-“Waveform” sheet shows PHY registers in OK cases(no data corruption) and Failing cases.
When leveling is done properly, DX1LCDLR2 register value is around 0x3A and no data corruption is seen.
On the other hand, when the issue happens the register value is around 0x0D.
The sheet also shows waveforms under leveling in each cases. Leveling is done by GEL file (ddr3A_64bit_DDR1600_setup2()).
Customer found leveling takes longer time in failing cases compared to OK cases (see waveforms attached around AB38 cell)

ddr3A_64bit_DDR1600_setup2()_gel(snippet).txt
/*----------------------------------------------------- DDR3A : DDR1600, 64bit,TEST--------------------------------------------------------------------------*/
ddr3A_64bit_DDR1600_setup2()
{
        unsigned int multiplier = 39;
        unsigned int divider = 0;
        int temp,i, delay=1000;
        unsigned int OD_val = 10;
    KICK0 = 0x83E70B13;
    KICK1 = 0x95A4F1E0;

//1.    Poll for IDONE=1 in the PHY General Status Register 0 (address offset 0x010).
         do {
    read_val = DDR3A_PGSR0;
    } while ((read_val&0x00000001) != 0x00000001);

//      Clocks are enabled and frequency is stable---------------------------------------
        // DDR3A PLL setup
        GEL_TextOut ( "DDR3 PLL Setup ... \n");
    // DDR3APLLCTL0 = DDR3APLLCTL0 & 0xFF7FFFFF;
        // Set ENSAT = 1
        DDR3APLLCTL1 |= 0x00000040;
        // Put the PLL in PLL Mode
        DDR3APLLCTL0 |= 0x00800000;

        // Program the necessary multipliers/dividers and BW adjustments
    // Set the divider values
        DDR3APLLCTL0 &= ~(0x0000003F);
        DDR3APLLCTL0 |= (divider & 0x0000003F);
    // Program OD[3:0] in the SECCTL register
    DDR3APLLCTL0 &= OUTPUT_DIVIDE_MASK;  // clear the OD bit field
    DDR3APLLCTL0 |= ~OUTPUT_DIVIDE_MASK & (OD_val - 1) << OUTPUT_DIVIDE_OFFSET;  // set the OD[3:0] bit field of PLLD to OD_val
        // Set the Multipler values
        DDR3APLLCTL0 &= ~(0x0007FFC0);
        DDR3APLLCTL0 |= ((multiplier << 6) & 0x0007FFC0 );
        temp = ((multiplier + 1) >> 1) - 1;
        DDR3APLLCTL0 &= ~(0xFF000000);
        DDR3APLLCTL0 |= ((temp << 24) & 0xFF000000);
        DDR3APLLCTL1 &= ~(0x0000000F);
        DDR3APLLCTL1 |= ((temp >> 8) & 0x0000000F);

        // In PLL Controller, reset the PLL (bit 13 in DDR3APLLCTL1 register)
        DDR3APLLCTL1 |= 0x00004000;
        for(i=0;i<delay;i++);
        // In DDR3PLLCTL1, write PLLRST = 0 to bring PLL out of reset
    DDR3APLLCTL1 &= ~(0x00004000);
        for(i=0;i<delay;i++);

        // Put the PLL in PLL Mode
        DDR3APLLCTL0 &= ~(0x00800000); // ReSet the Bit 23
                GEL_TextOut( "DDR3 PLL Setup complete, DDR3A clock now running at 800 MHz.\n" );
//DDR3A PLL setup complete ---------------------------------------


    /*------------------------------- ECO FIX -----------------------------------------*/
    /*----------------------------- LEVELING FIX --------------------------------------*/
    // DDR3 write leveling ECO - Assert & release DDR PHY RESET after DDR PLL setup...
    DDR3APLLCTL1 = DDR3APLLCTL1 | 0x80000000;   //Assert DDR PHY reset after PLL enabled
    for(i=0;i<delay;i++);
    DDR3APLLCTL1 = DDR3APLLCTL1 & 0x7FFFFFFF;   //Release DDR PHY reset

    do {   // Poll IDONE after resetting PHY
        read_val = DDR3A_PGSR0;
    } while ((read_val&0x00000001) != 0x00000001);


/*------------------------- Start PHY Configuration -------------------------------*/

        //DDR3A_PGCR1 = 0x0280C487;

        //5.a   Program FRQSEL in the PLL Control Register (address offset 0x018).
        DDR3A_PLLCR = 0x1C000; //Set FRQSEL=11

        //5.b.  Program WLSTEP=1, IODDRM=1, and ZCKSEL in the PHY General Configuration Register 1 (address offset 0x00C).
        DDR3A_PGCR1 |= (1 << 2); //WLSTEP = 1

        DDR3A_PGCR1 &= ~(IODDRM_MASK);
        DDR3A_PGCR1 |= (( 1 << 7) & IODDRM_MASK);

        DDR3A_PGCR1 &= ~(ZCKSEL_MASK);
        DDR3A_PGCR1 |= (( 1 << 23) & ZCKSEL_MASK);

        //5.c.  Program PHY Timing Parameters Register 0-4 (address offset 0x01C - 0x02C).
        DDR3A_PTR0 = 0x42C21590;
        DDR3A_PTR1 = 0xD05612C0;

        // Maintaining default values of Phy Timing Parameters Register 2 in PUB
        DDR3A_PTR3 = 0x0D861A80;//0x072515C2; //0x0B4515C2;//0x18061A80;
        DDR3A_PTR4 = 0x0C827100;//0x0AAE7100;

        //5.d.  Program PDQ, MPRDQ, and BYTEMASK in the DRAM Configuration Register (address offset 0x044).
        //              All other fields must be left at their default values.
        DDR3A_DCR &= ~(PDQ_MASK); //PDQ = 0
        DDR3A_DCR &= ~(MPRDQ_MASK); //MPRDQ = 0

        DDR3A_DCR &= ~(BYTEMASK_MASK);
        DDR3A_DCR |= (( 1 << 10) & BYTEMASK_MASK);

        //Uncomment for Dual Rank, Address Mirrored DIMMS
        //DDR3A_DCR &= ~(NOSRA_MASK);
        //DDR3A_DCR |= (( 1 << 27) & NOSRA_MASK);
        //DDR3A_DCR &= ~(UDIMM_MASK);
        //DDR3A_DCR |= (( 1 << 29) & UDIMM_MASK);


        //5.e.  Program DRAM Timing Parameters Register 0-2 (address offset 0x048 - 0x050).
        DDR3A_DTPR0 = 0x9D9CBB66;
        DDR3A_DTPR1 = 0x32868400;
        DDR3A_DTPR2 = 0x5002D200;

        //5.f.  Program BL=0, CL, WR, and PD=1 in the Mode Register 0 (address offset 0x054).
        //All other fields must be left at their default values.
        DDR3A_MR0 = 0x00001C70; //50

        //5.g.  Program DIC, RTT, and TDQS in the Mode Register 1 (address offset 0x058).
        //All other fields must be left at their default values.
        DDR3A_MR1 = 0x00000006;


//---------------------------------------------------------------------------------------------------------

        //5.h.  Program Mode Register 2 (address offset 0x05C).
        // Maintaining default values of Program Mode Register 2
        DDR3A_MR2 = 0x00000018;

        //5.i.  Program DTMPR=1, DTEXD, DTEXG, RANKEN=1 or 3, and RFSHDT=7 in the Data Training Configuration Register (address offset 0x068).
        //All other fields must be left at their default values.
        DDR3A_DTCR = 0x710035C7; //Single-Rank
        //DDR3A_DTCR = 0x730035C7; //Dual-Rank

        //5.j.  Program tREFPRD=(5*tREFI/ddr_clk_period) in the PHY General Configuration Register 2 (address offset 0x08C).
        //All other fields must be left at their default values.
        DDR3A_PGCR2 = 0x00F079E0;

        //Set Impedence Register
        DDR3A_ZQ0CR1 = 0x0001005D;
        DDR3A_ZQ1CR1 = 0x0001005B;
        DDR3A_ZQ2CR1 = 0x0001005B;
        //DDR3A_ZQ3CR1 = 0x0000005D;

        //6.    Re-trigger PHY initialization in DDR PHY through the VBUSP interface.
        //6.a.  Program 0x00000033 to the PHY Initialization Register (address offset 0x004) to re-trigger PLL, ZCAL, and DCAL initialization.
        DDR3A_PIR = 0x00000033;

        //6.b.  Poll for IDONE=1 in the PHY General Status Register 0 (address offset 0x010).
    do {
                read_val = DDR3A_PGSR0;
    } while ((read_val&0x00000001) != 0x00000001);

//---------------------------------------------------------------------------------------------------------

        // 7.   Trigger DDR3 initialization and leveling/training in DDR PHY through the VBUSP interface.
        // a.   If using a 16-bit wide DDR interface, program DXEN=0 in the DATX8 2-7 General Configuration Registers (address offsets 0x240, 0x280, 0x2C0, 0x300, 0x340, and 0x380) to disable the leveling/training for the upper byte lanes.
        // b.   If using a 32-bit wide DDR interface, program DXEN=0 in the DATX8 4-7 General Configuration Registers (address offsets 0x2C0, 0x300, 0x340, and 0x380) to disable the leveling/training for the upper byte lanes.
        // c.   If ECC is not required, program DXEN=0 in the DATX8 8 General Configuration Register (address offset 0x3C0) to disable the leveling/training for the ECC byte lane.
        // NOTE: Setup supports 64-bit by default,  ECC enable by default.

        //7.d.  Program 0x0000XF81 to the PHY Initialization Register (address offset 0x004) to trigger DDR3 initialization and leveling/training sequences

        DDR3A_PIR = 0x0000FF81; //WLADJ - ON
        //DDR3A_PIR = 0x00000781;  //WLADJ - OFF


//---------------------------------------------------------------------------------------------------------

//7.e.  Poll for IDONE=1 in the PHY General Status Register 0 (address offset 0x010).
        do {
                read_val = DDR3A_PGSR0;
           } while ((read_val&0x00000001) != 0x00000001);


        /* End PHY Configuration */




//---------------------------------------------------------------------------------------------------------
/* START EMIF INITIALIZATION
  ++++++++++++++++++SDCFG Register Calculation+++++++++++++++++++
  | 31 - 29  | 28 |27 - 25 | 24   | 23 - 22| 21 - 17 |
  |SDRAM_TYPE|Rsvd|DDR_TERM| DDQS | DYN_ODT|  Rsvd   |
  |  0x011   |  0 | 0x001  | 0x0  |   0x00 |   0x0   |

  | 16-14 |13 - 12 |  11 - 8 |  7   |6 - 5 |  4  |  3  |  2  |  1 - 0  |
  |   CWL | NM     |   CL    | Rsvd |IBANK | Rsvd|EBANK| Rsvd|PAGE_SIZE|
  |  0x11 | 0x00   |  0x1110 |  0x0 | 0x110| 0x0 |  0  |  0  |   0x10  |
  SDCFG = 0x0110 0011 0010 0010 0011 0011 1011 0010
  SDCFG = 0x6200CE62;

  SDRAM_TYPE = 3
  DDR_TERM = 3 (RZQ/4 = 1; RZQ/6=3)
  DDQS = 1
  DYN_ODT = 0

  CWL = 3 (CWL5=0; CWL6=1; CWL7=2; CWL8=3)
  NM = 0 (64-bit=0, 32-bit=1, 16-bit=2)
  CL = 14 (CL5=2; CL6=4; CL7=6; CL8=8; CL9=10; CL10=12; CL11=14)
  IBANK = 3 (8bank)
  EBANK = 0 (0 - pad_cs_o_n[0] , 1 - pad_cs_o_n[1:0])
  PAGE_SIZE = 2 (1024page-size=2; 2048page-size=3)
*/
/* Start DDR3A EMIF Configuration */
        //8.    Configure the EMIF through the VBUSM interface.
        //8.a.  Program all EMIF MMR<92>s.
        DDR3A_SDCFG   = 0x6200CE62; // Single-Rank
        //DDR3A_SDCFG = 0x6200CE6A;     // Dual-Rank

    DDR3A_SDTIM1   = 0x166C9875;
    DDR3A_SDTIM2   = 0x00001D4A;
    DDR3A_SDTIM3   = 0x435DFF53;
        DDR3A_SDTIM4   = 0x543F0CFF;

        DDR3A_ZQCFG    = 0x70073200; // Single-Rank
        //DDR3A_ZQCFG  = 0xF0073200; // Dual-Rank

        //8.b.  Program reg_initref_dis=0 in the SDRAM Refresh Control Register (address offset 0x10).
    DDR3A_SDRFC = 0x00001860;

        GEL_TextOut("DDR3A initialization complete \n");
           /* End  DDR3A EMIF Configuration */
}

Customer checked waveforms in details (see column AO or later).
Sequential reads(0,1,0,1,0…) in leveling steps take longer time in failing cases.
As a result, read timings are different. (see waveforms around BL87 cell)

- Customer compared waveforms with EVMK2H. See “Comparing with EVMK2H” sheet.
On EVMK2H, the sequential reads takes less time.
And DQS line release timing is earlier in read access. This is similar in OK cases on customer board.
In failing case, DQS line is released just before read access.
Customer checked DQS-DQ timings on both boards, but no clear differences are seen.(Column AL and later)

Customer gets stuck and need TI help for further investigation to solve this issue.
Could you check the leveling waveform and provide your feedback?
Raw data (*.csv format) are available to send in separate e-mail. (see “Raw data” sheet for waveform information)
Customer can provide *.wfm format, if you have an environment to see keysight oscilloscope data.

Thanks and regards,
Koichiro Tashiro

  • Tashiro-san,

    It is known that leveling can fail or be intermittent due to signal integrity problems.  Proper board layout and stack-up are required for robust signal integrity.  We provide an extensive set of documentation to enable customers to successfully design boards that operate at the rated speeds.  These documents and tools are all linked from the KeyStone II DDR3 Interface Bring-Up Application Report (SPRACM0).

    Proper length matching and trace spacing are needed to meet the critical timing margins.  Impedance control through proper use of reference planes and layer transitions is also mandatory.  On top of that, you need to properly isolate the DDR signals from noise from other circuits like power supplies.  Can you provide a length matching report showing that the routing rules have been met?  Are all routes sandwiched between ground planes?  If not, have you analyzed whether the routed impedance is correct?

    How many boards of this new revision were manufactured?  Do they all fail similarly?  Do you see robust operation if you reduce the DDR data rate to1333MT/s?

    Tom

    '

  • Hi Tom,

    >Do you see robust operation if you reduce the DDR data rate to1333MT/s?

    Yes, the board works fine with 1333MT/s setting.

    I asked customer to provide remaining information you wanted.

    Meanwhile, could you check below updated information from customer?
    Customer found if DTMPR bit in DTCR register was changed to 0x0, the issue was gone.
    Please see attached excel sheet (“DTMPR=0x0” sheet)
    DDR3 Leveling waveforms-DTMPR=0.xlsx
    According to user’s guide, DTMPR bit just selects SDRAM MPR(Multi-Purpose Register) is used or not.
    Why changing the bit results in different leveling results? Any ideas?

    Customer checked calibration waveforms and found DQ8 test pattern is changed (0,1,0,1… to 1,0,1,0…).
    Does this affect to something?
    Please see “Waveforms” sheet.

    TI register calculation sheet only shows an option using DTMPR=0x1. Using 0x0 is not supported?

    Thanks and regards,
    Koichiro Tashiro

  • Hi Tom,


    >Can you provide a length matching report showing that the routing rules have been met?

    Please find below a length matching report.
    The format is a different from the application note, but the contents should be the same.
    DDR3_length matching.xlsx

    >Are all routes sandwiched between ground planes? 

    PCB layers are like this. DDR signals are marked “<==”.

     TOP(DSP, DDR3)
     GND
       Signal <==
       1.5V
       Signal <==
       GND
     GND
     Signal <==
     GND
     Signal <==
     GND
     BOTTOM

    >How many boards of this new revision were manufactured?  Do they all fail similarly? 

    Customer checked four boards so far and all boards showed similar behaviors.
    So all other boards seems to be the same.

    Thanks and regards,
    Koichiro Tashiro

  • Tashiro-san,

    The length matching report looks correct.  You state that the boards work correctly at 1333MT/s.  This points to a marginality that is making the leveling unreliable.  We need to examine the routing in more detail.  The spreadsheet indicates that the ACCC routing is fly-by.  Is this fully understood and implemented correctly?  Also, there must be a separate track branch to each SDRAM for each ACCC signal.  Are all of these branches of similar length?  They may need to examine the min and max of the ACCC branches at each SDRAM.

    Are the processor and all DDR memories on the same side of the PCB?  If so, which side?

    The stack-up provided shows DDR routes sandwiched between GND and 1.5V planes.  Are all of these planes solid within the DDR keep-out region?  Are there plenty of ground stitching vias so that there is one within 100mils of all signal vias?  Similarly, where the signals are referenced to the 1.5V plane, are there decoupling caps and associated vias placed to provide a high-frequency return path to ground where these signals switch reference planes?  Do the signals cross voids in the planes such as due to groups of vias?

    You indicate that the DDR traces are routed on layers 3, 5, 8 and 10.  Where are the data group nets routed?  Are all routes within a single data group routed on a single layer?  Do they all have exactly 2 vias?  What about the fly-by routes?  Do they each have a single via per route for each connected chip?

    What are the widths of the DDR traces?  What is the edge to edge spacing between these traces.  What are their target impedances?  Are there other non-DDR traces or vias in the DDR area that may provide crosstalk?

    They also need to explore possible sources of electrical and magnetic interference.  Are there any POL supplies close to the DDR region.  Spacing is the only solution to reduce the possibility of magnetic interference from inductors.  Additional ground stitching vias can be added along the perimeter of the DDR region to reduce noise currents in the GND planes.  Another solution around power supplies is to use localized ground planes on surface layers to reduce the amount of AC ripple current in the inner ground planes.

    I know that this is a lot of questions.  All of these considerations must be carefully examined to obtain optimum rated performance.

    Tom

  • Tashiro-san,

    You asked about the DTMPR register functionality.  As observed, this bit changes the behavior of the data training sequence.  This option is outside of our supported and validated leveling solutions.  As you have shown, under some conditions it can improve leveling.  However, this is not true for all designs.  That is why it is not part of our standard support model.  If the board is properly implemented as we have described, this optional mode is not required.

    Tom

  • Hi Tom,

    Please find in attachment customer PCB layers.
    I hope you can get most of information you asked from these figures.
    DDR3_pattern2.xlsx
    Do you see any suspicious points here?

    Regarding DTMPR bit configuration, the only difference using 0x0 or 0x1 is using MPR(0,1,0,1,0…)
    or other data located different memory location?

    One more question:
    Customer wants to know the meaning of DX1LCDLR2 register value as they saw clear differences in this value
    between failing cases and passing cases.
    The user’s guide only tells these values are “Rank n Read DQS Gating Delay: Rank n delay select for the read DQS gating(DQSG) LCDL“.
    What does it exactly mean?


    Thanks and regards,
    Koichiro Tashiro

  • Tashiro-san,

    We cannot perform a layout review.  This is too time intensive.  Also, the screen captures in the spreadsheet are not sufficient for this type of analysis.  The point of my many questions is to make it clear that there are a lot of details that must be carefully checked one at a time.  The board designer must work through these questions and verify the answers to each.  We can support a dialogue on these individual topics.

    The DTMPR bit changes the data training pattern.  I believe that the options are not limited to alternate 1010 patterns.  The patterns may also be programmable with the involvement of other registers.  This is outside our standard support and we cannot provide any guarantee of performance or behavior.

    The DXnLCDLRx register values are delay values that are the result of the leveling process.  When the leveling process fails, the delay line values will be different from when it succeeds.

    Tom