Changing from AM335 720MHz to 1GHz

Hi Jarrod,

Have you checked your power rails, especially VDD_MPU? What PMIC are you using?

Biser,

Thank you very much for the reply!  

Voltage was the first thing that came to mind since the 1GHz processor needs a 1.325 MPU voltage.  I added prints to U-boot to make sure it was finding the correct cpu and setting the correct voltage and it was.  And i also read all the voltages using  a multimeter and everything looked good : (.

I am using the TPS65910A3 PMIC.

I metered voltages between a working 720MHz board and the non-working 1GHz board and here is a table of comparisons (all readings taken while sitting idle at the U-boot prompt):

I did think it was a little weird that the Vaux2 for both boards is over 4 volts, but it is the same on working boards and goes down to 3.3 once the kernel boots and sets it.

Now i did dig a little deeper into this issue by starting from the line in the kernel code that prints "Uncompressing Linux..." since i know it gets at least that far.  I added prints in to figure out where i was getting to and here is the basic progression of events:

1. After printing the uncompressing message it calls gunzip in lib/decompress_inflate.c

2. the gunzip function gets down to its while loop and performs the first call of zlib_inflate (which is in lib/zlib_inflate/inflate.c).

3.Within zlib_inflate we get to case LEN

4. We never escape case LEN, the first if statement (have >=6 && left >= 258) is true and it calls inflate_fast and eventually just locks up it seems.

I added some prints to case LEN and from what i can tell it does call inflate_fast multiple times, but at some point it just locks up because my prints stop and nothing happens.

Could it be that i have a RAM issue?  I wouldn't think it would be necessary to change DDR3 parameters if all i did was swap the processor.  The board i am working with was a working board with a 720MHz processor, so as long as they didn't mess anything else up when they swapped it out for me everything else should be functional.

I think my next step is going to be to have them do the swap to another working board, so i can confirm consistency in the the issue and rule out a hardware problem.  This is a custom board and our production has only built 14 prototypes so far, so it could very well be that there is an issue with the hardware itself.  Once i do another swap i can at least hopefully pinpoint the cause a little further.

Let me know if you have any other suggestions or information!!! : )

Thanks!

Alright, so i had them swap a second board and the same issue occurs on this board as well.  Both fail during the "Uncompressing Linux..." step.  Sometimes they just lock up and sometimes they will actually return an error and say "decompressor returned an error"  I had it print out the message set by the inflate_fast function and when it did return an error it was always "invalid distance too far back".

So the problem is consistent across 2 boards making it less likely that our production goobered something up (although still possible).  So any ideas?  The only thing i am leaning towards is some issue with RAM.  I may try to play with that some more and see what i come up with i guess : /.

Very strange... I doubt that DDR causes the issue, as u-boot also runs from it. I suggest you check the AM335X power supplies for excessive noise/spikes - you could compare a 720MHz vs. a 1000MHz board.

Also this VAUX2 4V level is very dangerous. You should investigate this. This is usually caused by voltage leaks through the processor before the power supplies have ramped up and stabilized. Double check and make sure there are no voltages applied to processor pins before the power supplies have ramped up and stabilized. The AM335X I/O pins are not fail-safe, with the exception of USBx_VBUS pins. See the AM335X Datasheet, Rev. G, section 5.1, parameter "Steady State Max. Voltage at all IO pins" and Note 8 below the table.

Thanks for the suggestions! : )

The 4V VAUX2 was just a stupid blunder on my part.  I was using a little USB-UART board to talk to the console and i have 2 of those: one jumpered to 5V and one jumpered to 3.3v.  I was accidentally using the 5v one (doh!) : /.  Either disconnecting that board or using the board properly jumpered for 3.3 caused the VAUX2 to be at 3.3 like it should be.

The other issue remains unchanged however.  I also didn't think RAM would be the issue since i am running u-boot out of it, but couldn't think of anything else it could be : (.  I will check the supplies for noise and try a couple different supplies to see if that is the issue.

I'll let you know what i find : ).  Thanks again!!!

Well, i checked the supplies and compared between the 720MHz and 1GHz boards, but there didn't seem to be much difference.  It also didn't seem like there was much noise or spikes on either : /.

I am not really sure where to progress in my investigation from here.  I will have to think about it some more i guess and see if i can think of anything else that may be a problem.  Very frustrating : /.

Any other suggestions of what i should try?

Here are schematics of my processor and power sections if that is helpful at all.  It follows closely to the starterkit:

Thanks.

Biser,

Thanks i forgot about those!  Those shouldn't cause the problem I am getting though should they?  I replaced them with 0 Ohm on the boards and still the same issue : (.

I put a JTAG header back on the board and hooked up the debugger and went through the DDR3 calibration again following the TI instructions:

http://processors.wiki.ti.com/index.php/AM335x_EMIF_Configuration_tips

http://processors.wiki.ti.com/index.php/AM335x_DDR_PHY_register_configuration_for_DDR3

And i ran the DDR tests in the Gel file and everything was successful, but linux still failed during decompresssion : (.

However, a bit of good news.  I played around some more and was finally able to get it to boot.  What i did was:

1. Power up the board.
2. Hit a key to stop U-boot from starting the kernel.
3. Connected to the board via a JTAG debugger.
4. Loaded in the GEL File from the above links (slightly modified to support my board)
5. Ran the system initialization script.
6. Allowed execution to continue and typed boot on the U-Boot prompt

So whatever that init script from the GEL file is doing it allows my board to boot.  I will investigate this matter further tomorrow to see if i can find the exact setting that is correcting the issue.  I will post back tomorrow and let you know what i find : ).

Just a quick update.  Running the Gel file before booting fixes it because it sets the MPU PLL to a lower speed : /.

So this allows the board to boot up fine:

MPU_PLL_Config( CLKIN, 23, 800, 1);
CORE_PLL_Config( CLKIN, 23, 1000, 10, 8, 4);
DDR_PLL_Config( CLKIN, 23, 400, 1);
PER_PLL_Config( CLKIN, 23, 960, 5);
DISP_PLL_Config( CLKIN, 23, 48, 1);

And this fails:

MPU_PLL_Config( CLKIN, 23, 1000, 1);
CORE_PLL_Config( CLKIN, 23, 1000, 10, 8, 4);
DDR_PLL_Config( CLKIN, 23, 400, 1);
PER_PLL_Config( CLKIN, 23, 960, 5);
DISP_PLL_Config( CLKIN, 23, 48, 1);

So it is directly tied with the 1GHz speed in the kernel.  If i start up at 800MHz and then try to change to 1GHz via:

echo 1000000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed

Then everything locks up.  Is there something i could be doing somewhere in the kernel config or board init file that is changing things?

I am going to keep playing around to see if i can figure it out.  I guess i would have to have messed something up in the kernel setup, not sure what though : /.

Hi Jarod,

I don't know if you have already done this, but take a look at the following U-Boot file, where the MPU PLL and VDD are initialized: <u-boot_dir>/board/ti/am335x/board.c

The function you would be interested in is am33xx_spl_board_init().

Try to initialize everything there.

Best regards,
Miroslav

Miroslav,

Thanks for the reply : ).  I actually added print outs to this function earlier on to make sure it was setting the correct MPU PLL and MPU VDD and it was.  It correctly sets PLL to MPUPLL_M_1000 and VDD to PMIC_OP_REG_SEL_1_3_2_5. And reading the voltage of the MPU Supply line shows that it is in fact set to 1.325.

What do you mean by try to initialize everything here? Do you mean i should move the code from s_init that sets everything up into this function?

Thanks,

Jarrod

No, sorry for not being clear. I meant exactly what you have already done - set the MPU PLL and VDD in the s_init function rather than doing it later via the kernel sysfs interface.

Best regards,
Miroslav

Oh okay, no problem : ).  Yeah U-Boot doesn't seem to have any issues, it sets the 1GHz speed and voltage and runs without a hiccup. When i try to boot linux though it just locks up during uncompression or occasionally will throw an uncompression error caused by: "invalid distance too far back".  Reading the MPU voltage at this point in time it is still 1.325, so no issues there.

The only way i can get the kernel to boot is by turning the speed back down to 800MHz or lower before callling boot in U-Boot.  I am currently playing with my kernel init stuff (arch/arm/mach-omap2/board-am335xevm.c) assuming the issue is there somewhere and hoping i can pinpoint it : /.