C6654 DDR3 initialization problem

Mark,

The results listed above are from ROM configuration of the DDR3.  Do you see similar results when you configure the DDR3 using a GEL file containing the proper initialization values?

Tom

Tom,

If I used a gel file to configure the DDR3 for the boards that did not work after a ROM configuration, the DDR3 worked as expected and did not show any problems.  I did some testing by changing some things in the gel file to try to reproduce the behavior  seen after a ROM configuration, which is how I stumble onto the INITREF_DIS bit.  At this point I'm just looking for a place to start, because I don't understand how this only affects some boards and why the workaround of setting SDCFG again causes the DDR3 to configure correctly.

Thanks,

Mark

Mark,

If I understand you correctly, you are able to reproduce the ROM behavior using the GEL file if you set INITREF_DIS to 0 at the beginning of the ddr3 configuration?

What is the value of the STATUS register for the passing and failing cases after both the ROM configuration and the GEL file? When you say you cannot read or write to one of the DDRs, do you mean you read back only 0s or 0xFs? Is the failure signature exactly the same after both the ROM configuration and GEL configuration?

Aditya,

I double checked and I am able to get similar behavior with a gel file and the INITREF_DIS bit set to 0, but it is not exactly the same.  It is the same in that only one of the DDR chips is working, but it behaves differently when you try to write to an address.  After a ROM configuration, the value read from the DDR chip that is not working is stuck at a value (it is different every power up) and writing has no effect.  For the gel file, the value from the DDR that is not working starts at a random value but sets to 0 after a write.

In all cases the STATUS register is 0x40000004.

Mark,

The STATUS register says that leveling took place correctly.

A few more questions to get an idea of the behavior pattern:

1) Is the issue on a 'bad' board repeatable with the same failure signature?

2) Do all 'bad' boards exhibit the same failure signature after ROM configuration?

3) Do all 'bad' boards exhibit the same failure signature after GEL configuration?

4) On any of the 'bad' boards, are you able to ever get correct DDR3 operation to both DDR devices?

Aditya,

1) We have some boards that will always show this behavior at power up, and we also have some that will only do it occasionally.

2)  All the boards I have tested show the same behavior after a ROM configuration.

3) Same for GEL

4) As long as INITREF_DIS is set to one before the other configuration registers are written, the DDR3 works fine on all boards.

Thanks,

Mark

Mark,

Please provide completed PHY_CALC and REG_CALC worksheets along with your GEL file reflecting the calculated values.  Also provide a track length report showing that the routing rules have been met.

Tom

Tom,

Sorry for the delay, the hardware engineer that did the routing was busy so it took a while to get the track length report.  Attached is a zip file that contains everything.  Sheet 3 of the DDR3 length report shows the adjusted lengths for all the signals.  The GEL file has a function that I used for testing called ddr3_setup_test() that sets up the DDR3 registers similar to how the ROM code does which I used to try to reproduce the problem.  Let me know if you need anything else or have any questions.

Thanks,

Mark0246.DDR3 worksheets.zip

Mark,

This forum post is for C6654.  We have recently found that the C665x GEL does not provide all of the registers needed to properly enable Partial Automatic Leveling.  The GEL provided successfully works at DDR3-800 but is not robust at higher speeds.  Your GEL appears to configure the DDR3 interface for DDR3-1066.

We are currently updating the DDR3 Init App note and the DDR3 UG to reflect these changes.  These documents will be released by the end of January.  You will need to add the following to your GEL file:

#define DDR3_CONFIG_REG_52   (*(unsigned int*)(CHIP_LEVEL_REG + 0x04D4))
#define DDR3_CONFIG_REG_53   (*(unsigned int*)(CHIP_LEVEL_REG + 0x04D8))
#define DDR3_CONFIG_REG_54   (*(unsigned int*)(CHIP_LEVEL_REG + 0x04DC))
#define DDR3_CONFIG_REG_55   (*(unsigned int*)(CHIP_LEVEL_REG + 0x04E0))
#define DDR3_CONFIG_REG_60   (*(unsigned int*)(CHIP_LEVEL_REG + 0x04F4))

 DDR3_CONFIG_REG_52 |= 0x00000200;
 DDR3_CONFIG_REG_53 |= 0x00000200;
 DDR3_CONFIG_REG_54 |= 0x00000200;
 DDR3_CONFIG_REG_55 |= 0x00000200;
 DDR3_CONFIG_REG_60 |= 0x00000200;

The above registers are written where the DDR3_CONFIG_REG_23 register is written to enable Partial Automatic Leveling in the initialization sequence for C667x devices.

I also noticed that your GEL file initialized the wrong INIT_RATIO registers.  We need to clarify this in the DDR# UG as well for C665x.  Please see the correct registers below:

 DATA0_WRLVL_INIT_RATIO = 0x2C;
 DATA1_WRLVL_INIT_RATIO = 0x2C;
 DATA2_WRLVL_INIT_RATIO = 0x3A;
 DATA3_WRLVL_INIT_RATIO = 0x33;
 DATA8_WRLVL_INIT_RATIO = 0x1C;
 
 DATA0_GTLVL_INIT_RATIO = 0xA4;
 DATA1_GTLVL_INIT_RATIO = 0xA4;
 DATA2_GTLVL_INIT_RATIO = 0xB1;
 DATA3_GTLVL_INIT_RATIO = 0xB7;
 DATA8_GTLVL_INIT_RATIO = 0x98;

We apologize for the confusion.  We are diligently working to get the documentation corrected.

Tom

Tom,

Thanks for the update that is definitely some useful information.  I have a couple more questions though so I can better understand it.

 1) You mentioned CONFIG_REG52-55 and 60 were needed for Partial Automatic leveling, are they necessary for Full Automatic leveling and is there any reason I couldn't use Full Automatic leveling (followed by an incremental leveling to address the Errata)?

 2) The DDR3 UG mentions that DATA0_WRLVL_INIT_RATIO and DATA0_GTLVL_INIT_RATIO are for Byte lane 7, DATA1_WRLVL_INIT_RATIO and DATA1_GTLVL_INIT_RATIO are for Byte lane 6, etc., is the order different for the C665x compared to the C667x?

I also have some questions related to the original problem I was trying to solve, which is why some boards DDR3 does not work after a ROM configuration.

 3) Looking at the source for the RBL it appears that it does not execute leveling for the DDR after it is configured (which I really hope I'm just missing it), so do your above statements apply for that case?

 4) The GEL file I sent was just used to try to reproduce this problem, and the one thing I noticed the RBL does is set INITREF_DIS to 1 before writing the the other configuration registers, and then set it to 0 before writing SDCFG.  I have noticed that on the boards that have problems, if INITREF_DIS is set to 1 when SDTIM1/2/3, DDR_PHY_CTRL1, ZQCFG and PMCTL are set and then set to 0 before SDCFG is set the DDR will not work.  If INITREF_DIS is set to 0 for the entire configuration process the DDR works fine.  Why does disabling initialization and refreshes when SDTIM1/2/3, DDR_PHY_CTRL1, ZQCFG and PMCTL are written change the DDR3 behavior? (Note: on other boards what INITREF_DIS is set to during the configuration process does not matter and they will always work)

The end goal is to use the RBL to configure the DDR3 and load the program using one file.  I want to avoid using a two stage boot loader, but if the RBL does not do any leveling that might be the best option.

Thanks,

Mark

Mark,

Responses to your questions:

#1:  The C665x devices equally support the Partial Automatic Leveling and full automatic leveling as defined in the DDR3 Initialization Application Report.  I did review the GEL that you provided and realized that you do not trigger any type of leveling in this GEL.  This needs to be corrected.

#2:  We are double-checking the register mappings for the C665x devices.  It is known to be the registers I listed above for the C665x devices.  We are verifying the order.

#3:  I am not an expert on the RBL and its capabilities.  I will work with you to get repeatable DDR3 configuration on your board using a GEL script or a 2nd level boot loader.  After that we can get the right person to help you with the RBL.

#4:  INITREF_DIS enables and disables DDR3 mode register writes during register programming.  When enabled, every update to the timing or config registers will invoke a complete DDR3 SDRAM initialization sequence.  We normally leave it disabled during the register programming and then enable it before the SDCFG write.  (Be sure the SDRFC has the slow refresh rate programmed whenever an initialization cycle is executed.)

I am not sure why changing this helps.  It must somehow invoke a leveling cycle because your GEL file does not do this explicitly as required but the status register indicates leveling success.

Note that the leveling sequence documented has mandatory wait periods.  The leveling process may fail if these are not implemented correctly.  The MCSDK IBL and PlatformLib code shows validated sequences for this.

Tom

Tom,

Thanks for the continued assistance but I think we are not on the same wavelength anymore.  I already have a repeatable DDR3 configuration for these boards, and have been using it for at least 6 months with out any problems. I am trying to figure out why a small number of boards don't work after the RBL.  Also, the GEL does contain leveling in a funciton called DDR3Leveling() that executes Full Automatic leveling, I split it apart for debugging purposes.

To go back to question 4, According to the DDR3 UG a complete DDR3 SDRAM initialization sequence should only happen after a hard or soft reset, or a write to SDCFG (section 2.11).  So it should not matter what INITREF_DIS is set to when the timing registers are written (unless this is also not accurate in the UG).

I am trying to rule out any hardware issues with the boards, because if there are any we need to do another revision.  If is just a problem with the RBL, I can solve it in software.

Mark 

Mark,

Your last statement contradicts your previous assertion.  My primary concern is to help you have a robust configuration sequence that operates on all of your boards boards.  This desire is specifically to make sure your board design is robust and does not need to change.

You state in your first paragraph that you have a configuration using Full Leveling that repeatably works.  Does it work repeatably on ALL boards at ALL times?  What testing have you performed after DDR3 configuraiton to prove that the DDR3 memory interface is configured robustly?  Do you have this configuration in GEL form and can you provide it?

Tom

Tom,

Sorry for the confusion, here is the gel file that always works on all boards: 1072.apc6654_testDDR3.gel . I have it updated with the values you mentioned in your earlier post for the leveling init registers, but it also worked with my original values.  

To test it, I have a program that will loop through 32K random DDR memory addresses and write a random value, it then loops through the same addresses and compares the values it reads to what was written.  That entire process endlessly loops and will go through the entire DDR memory range.  While this test isn't perfect it is good enough to make sure reads and writes are working correctly and I have run it several times on different boards over night and did not get any errors.  We also do not see any strange behavior when running our application code (which is located in DDR). 

Mark

Mark,

OK, that means that your board design is robust and we are not debugging possible board layout issues.  Your concern is truly limited to getting the DDR3 configuration functional through the ROM Bootloader.  I will need to transfer this forum thread to an RBL expert.

Does your board contain an SEEPROM attached to the I2C (or a FLASH device on the SPI port) that could contain a second level bootloader?  Alternately, are you using a boot method such as over PCIe where an intermediate bootloader could be loaded into memory and executed prior to loading the full application?

Tom

Tom,

Thanks for sticking with this and helping with the debug process.  We have a way we could do a two-stage boot process if necessary, we were just trying to avoid it if possible and use the RBL with one boot file.

Mark

Hi Rahul,

Here is the DDR3 configuraiton table values we use in text format: 

00 00 01 20 00 8F FD 20 
01 01 44 7B 00 7F D7 FD 00 00 00 00 00 00 00 03 00 00 00 40 00 00 00 02
62 45 5A 32 00 00 00 00 00 00 10 46 0C CF 36 B3 30 3F 7F DD 55 FF 83 AF
00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00
00 00 00 00 70 07 4C 1F 00 00 00 00 00 10 01 0A 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 05
00 20 00 0F 00 00 00 00 00 00 00 2C 00 00 00 2C 00 00 00 3A 00 00 00 33
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1C 00 00 00 00
0A 00 00 00 00 00 00 00 00 00 00 A4 00 00 00 A4 00 00 00 B1 00 00 00 B7
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 98 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Thanks for including the DDR3 configuration table structure, I actually already had that from when we started trying this on the EVM (see thread: http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/223644/809091.aspx#809091 ).

Regards,

Mark

Hi Mark,

Thanks for providing the DDR3 configuration table.I will examine the parameters and let you know if there is something that jumps at me. 

I spoke to the RBL author regarding the DDR3 initialization seuqence in the RBL and his response regarding the DDR3 configuration in the RBL was that the only thing frozen in the ROM is the actual DDR3 initialization sequence that the DDR3 and phy registers are programmed. This sequence may have been changed since and so the only way to correctly initialize the DDR and load code would be to have a 2 stage boot where the first stage sets up the DDR  and the then re-enters the ROM to load the second stage.

Regards,

Rahul

An update on the mapping between DATAn ratios and byte lanes for C665x devices:

DATA0 => byte lane 3

DATA1 => byte lane 2

DATA2 => byte lane 1

DATA3 => byte lane0

DATA_ECC => ECC byte lane

 The DDR3 users guide on ti.com relfects this.