This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM3354 Standby Resume issues with DDR3 Using Two 8-bit DDR3 devices

Other Parts Discussed in Thread: AM3354

I am trying to get standby/resume working on our custom AM3354 based module.  The board has two MT41K256M8DA-125 Micrel modules.  

I have been able to reproduce the standby functionality using a BBB running the debian wheezy filesystem and the 3.14 ti kernel.  

Booting the same kernel and filesystem on our board does not work.  The system is able to go into standby but when the spacebar is pressed the system doesn't resume.  I am able to see the power usage increase after trying to wake up the device and the reset button starts functioning.

My question is has anyone tested sleep mode when using 2 DDR3 modules?  Is it possible that the 2 DDR3 setup could be the cause of this failure? What steps can I take to narrow down the cause of this problem.

Thanks

Jonathan Cormier

  • Jonathan Cormier said:
    My question is has anyone tested sleep mode when using 2 DDR3 modules?  Is it possible that the 2 DDR3 setup could be the cause of this failure? What steps can I take to narrow down the cause of this problem.

    Rev 1.5 of the AM335x EVM uses two 8-bit DDR3 devices.  I don't think that's related.

  • Brad,

    Thanks I wasn't aware there was a newer version of the EVM. I have one of the older DDR2 EVMs.  Looking on the EVM page: http://www.ti.com/tool/TMDXEVM3358, I can't find any of the schematics for the Rev 1.5 EVM.  Would it be possible to get these updated schematics?

    Also do you have any tips on diagnosis the resume issue?

    [Edit] Readding text the editor decided to delete...

  • Hi Jonathan,

    The GP EVM design files can be found here: http://processors.wiki.ti.com/index.php/AM335x_General_Purpose_EVM_Board_Design_Files I'm also attaching a power management tutorial, that has some debugging tips in it:

    3554.sitara_boot_camp_am335x_pm_sw_training.pdf

  • Thanks. I will look over it and get back to you.
  • Alright it looks like our problem is most likely our VTT. Its VTT_EN is coming from the TPS VDIG1 instead of being controlled by DDR_CKE. Can you confirm that this would be a problem?
  • Yes, this is a possible reason.
  • Also is there a revision history somewhere for the EVM changes?
  • So what is the implication on VTT_EN, i.e. does it stay on all the time? If not, at what point does it get turned off and at what point does it get turned on? We include VTT on our starter kit and in that scenario we were using a GPIO to manage the enable.
  • VTT stays on all the time
  • In that case, I would just expect wasted power but not a failure. I would look elsewhere. Have you tried creating a minimal code set, e.g. basically strip everything out of your dtb? It's likely a driver causing an issue. You may need to add drivers back in one at a time if the minimal test is successful.
  • The thing is that I'm using the same exact filesystem/kernel/dtb for the beaglebone and our board. The only software that I believe to be different is u-boot.

    Another difference is that our TPS is on I2C1 and I2C2 instead of both lines being on I2C0. Would this cause the CM3 code to fail to resume?
  • Jonathan Cormier said:

    Another difference is that our TPS is on I2C1 and I2C2 instead of both lines being on I2C0. Would this cause the CM3 code to fail to resume?

    That's a pretty major difference!  You must update your dtb to account for this!

  • Sorry to clarify I was using our my dtb when booting my device and when booting the BBB.  Even though the TPS fails to load on the BBB its sleep mode still works.

    Boot log: 

    3.14_bootlog_devkit_3354.txt
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    U-Boot SPL 2013.10-00050-ged374e1-dirty (Aug 25 2015 - 17:25:28)
    MitySOM335x profile 1 - Model No: 3354-HX-X38-RC Serial No: 137410
    Configuring for 512 MB DDR3 @ 400MHz
    Critical Link AM335X Dev Kit -- NAND Page size = 4096k booting from dev 5
    Using 4k bch16 layout
    ECC Mode = 2 lo = 208/0
    nand_spl_load_image: loading 4096 bytes from 100000 to 80800000
    nand_spl_load_image: loading 458116 bytes from 100000 to 807fffc0
    U-Boot 2013.10-00050-ged374e1-dirty (Aug 25 2015 - 17:25:28)
    AM335X-GP rev 2
    I2C: ready
    DRAM: 512 MiB
    WARNING: Caches not enabled
    NAND: Using 4k bch16 layout
    ECC Mode = 2 lo = 208/0
    512 MiB
    MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1
    Net: cpsw:1 is connected to cpsw. Reconnecting to cpsw
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    Our device

    [    2.693952] omap_i2c 44e0b000.i2c: bus 0 rev0.11 at 400 kHz
    [    2.707633] omap_i2c 4802a000.i2c: bus 1 rev0.11 at 400 kHz
    [    2.732584] tps65910 2-002d: No interrupt support, no core IRQ
    [    2.752051] vrtc: 1800 mV 
    [    2.759519] vrtc: supplied by vbat
    [    2.770771] vio: at 1500 mV 
    [    2.777972] vio: supplied by vbat
    [    2.788774] vdd_mpu: 600 <--> 1500 mV at 1262 mV 
    [    2.797864] vdd_mpu: supplied by vbat
    [    2.808991] vdd_core: 600 <--> 1500 mV at 1137 mV 
    [    2.818113] vdd_core: supplied by vbat
    [    2.828572] vdd3: 5000 mV 
    [    2.837878] vdig1: at 1800 mV 
    [    2.844957] vdig1: supplied by vbat
    [    2.855130] vdig2: at 1800 mV 
    [    2.861997] vdig2: supplied by vbat
    [    2.871827] vpll: at 1800 mV 
    [    2.878281] vpll: supplied by vbat
    [    2.887809] vdac: at 1800 mV 
    [    2.894120] vdac: supplied by vbat
    [    2.903403] vaux1: at 1800 mV 
    [    2.909516] vaux1: supplied by vbat
    [    2.918658] vaux2: at 3300 mV 
    [    2.924599] vaux2: supplied by vbat
    [    2.933539] vaux33: at 3300 mV 
    [    2.939330] vaux33: supplied by vbat
    [    2.948166] vmmc_33: 3300 mV 
    [    2.953531] vmmc_33: supplied by vbat
    [    2.961762] omap_i2c 4819c000.i2c: bus 2 rev0.11 at 400 kHz
    

    BBB

    [    2.906205] tps65910 2-002d: No interrupt support, no core IRQ
    [    2.922506] tps65910 2-002d: Error in configuring external control EN1
    [    2.934503] tps65910 2-002d: Failed to initialise ext control config
    [    2.948267] vrtc: failed to enable
    [    2.959659] tps65910 2-002d: failed to register tps65910-pmic regulator
    [    2.971885] tps65910-pmic: probe of tps65910-pmic failed with error -121
    

    My device never prints anything past this.

    root@beaglebone:~# echo standby > /sys/power/state 
    [  127.341491] PM: Syncing filesystems ... done.
    [  127.870697] Freezing user space processes ... (elapsed 0.015 seconds) done.
    [  127.894278] Freezing remaining freezable tasks ... (elapsed 0.007 seconds) done.
    [  128.078751] PM: suspend of devices complete after 159.736 msecs
    [  128.096191] PM: late suspend of devices complete after 10.963 msecs
    [  128.116656] PM: noirq suspend of devices complete after 13.685 msecs
    

  • Also is there a guide for setting up for debugging the CM3 over JTAG using CCS6 or CCS5.5?
  • I think I see the issue.  I just cloned the Cortex M3 git tree and for instance if I look inside the pm_services/i2c.c file I see the following:

    static void i2c_reg_write(unsigned short val, int reg)

    {

           __raw_writew(val, I2C0_BASE + reg);

    }

    static unsigned short i2c_reg_read(int reg)

    {

           return __raw_readw(I2C0_BASE + reg);

    }

    In other words, it looks like it hard codes I2C0 as the port for adjusting the PMIC.  You'll either need to spin your hardware or modify the firmware.

  • Brad Griffis said:

    In other words, it looks like it hard codes I2C0 as the port for adjusting the PMIC.  You'll either need to spin your hardware or modify the firmware.

    Thanks Brad.  I take it that if the CM3 code fails to contact the PMIC it's possible it might hang up?  From what I understand of the code the I2C communications shouldn't be crucial to the system resume, would it be enough to comment out the I2C comms to see if this fixes the resume issue.
    Building and debugging CM3 code
    • git clone git://git.ti.com/ti-cm3-pm-firmware/amx3-cm3.git

    CCS 6

    • New -> project..
    • Makefile Project with Existing Code
    • etc
    • New -> Target config file
      • Connection: XDS510USB
      • Board: AM3354

    • Project Properties -> C/C++ Build -> Environment
      • Append to path: /opt/ti-sdk-am335x-evm-05.03.02.00/linux-devkit/bin:
    • Setup debug configuration to only connect to M3_wakeupSS_0


  • Jonathan Cormier said:
    I take it that if the CM3 code fails to contact the PMIC it's possible it might hang up?

    That's my suspicion anyway...  Can you just tie your I2C buses together temporarily with some wires (i.e. SCL-to-SCL and SDA-to-SDA)?  Perhaps that would be an easier test, provided that I2C0 is accessible on your board.  Another thought that comes to mind is to connect to the M3 with JTAG while you're in suspend and see where it's at.  Perhaps it's stuck in a loop or something.  I had a brief look at the I2C code, and I see some timeout loops, etc. in there so it's not obvious to me exactly where/how it fails.  It may not be related to these I2C changes at all, but as of now, that seems the mostly culprit.

  • Our DDR2 module uses the same I2C bus to talk to the PMIC and it is able to suspend and resume, so it seems unlikely that the I2C would suddenly become an issue for the DDR3 variant.  I'll have to check with the hardware guys to see if we can tie the I2C busses together.

  • I can't get the debugger to connect to the CM3 anymore.  When I try it goes through the connecting screens and then I see the M3_wakeupSS_0 target but it says Disconnected: Unknown.  No error messages are displayed.  I'm not sure what to do try to fix it.

  • 	/* For some reason, the compiler does not like the WFI wrapper here :\ */
    	while(1)
    		__asm("wfi");

    Tried debugging using CCS5.5 and was able to connect to the CM3.  Connected before going to sleep, after going to sleep, and after trying to wake up device.  Each time the PC was set to the wfi instruction above.

  • Jonathan Cormier said:
    /* For some reason, the compiler does not like the WFI wrapper here :\ */

    while(1) __asm("wfi");

    Could it be related to the fact that you don't have a space before wfi?  The assembler treats anything lines that don't start with a space or tab as a label.  I would have expected something like __asm(" wfi");

  • Brad Griffis said:

    Could it be related to the fact that you don't have a space before wfi?  The assembler treats anything lines that don't start with a space or tab as a label.  I would have expected something like __asm(" wfi");

    According the the git repo it's been that way since its initial commit in 2011. 

    git://git.ti.com/ti-cm3-pm-firmware/amx3-cm3.git

  • Sorry, I think I mistook the comment from the code as being something that you were trying to debug.  I don't see any compiler errors when building that file so I'm not sure why that statement is even there!

    The way I've been stepping through some of this code was by doing something like this:

    --- a/src/pm_services/pm_handlers.c
    +++ b/src/pm_services/pm_handlers.c
    @@ -83,6 +83,10 @@ void a8_lp_ds0_handler(struct cmd_data *data)

            unsigned int per_st;
            unsigned int mpu_st;
            int temp;
    +       volatile int stop;
    +
    +       stop=1;
    +       while (stop) {};  // need to connect and set stop=0 to move forward

            if (cmd_handlers[cmd_global_data.cmd_id].do_ddr)
                    ds_save();

    So when I would run "echo mem > /sys/power/state" the M3 would stick in the loop above.  I would then attach to the M3 and load symbols to debug.  I should also note that the very last line of that file (clkdm_sleep(CLKDM_WKUP)) will cause you to lose JTAG connectivity.  I recommend that you actually DISCONNECT from the M3 before executing that instruction so that you get a clean disconnect.  When the JTAG disconnects from the M3 it will automatically start running again.

    You could try a similar technique to step through a8_wake_ds0_handler.  You would need to place your stop loop after the clkdm_wake(CLKDM_WKUP) line so that you can connect with JTAG.  I haven't actually tried that one out, so you may want to first try it with the BeagleBone Black where suspend resume is working to double check.

  • Thanks. The troubleshooting pdf posted earlier mentioned being able to access the ARMs registers such as CM_PER_<MODULE>_CLKCTRL. Are these simply located at the same memory offsets are on the ARM?
  • Yes, exact same addresses as you would view them from the A8. On a related note, you might find this script useful for grabbing a register dump:

    processors.wiki.ti.com/.../AM335x_Clock_Tree_Tool

    Note, however, that one small adjustment needs to be made to the dss script itself. Normally that script uses the DAP to read the registers. You need to change that to use the M3 instead. So line 232 should look like this:

    debugSessionDAP = ds.openSession("*","M3_wakeupSS_0");

    I've seen some slight differences in the naming of this core across CCS versions. Your earlier screenshot of CCS looks like it uses the same name as mine (M3_wakeupSS_0). I had another customer mention that they did not have the trailing _0 in their version.

    I generally use this for comparing a known good system against a bad system. It might be useful to compare your older working DDR2 system against your newer DDR3 system to see if there are any obvious differences from the PRCM perspective.
  • Using the above register dump script with the M3 modification you mentioned.  I dumped the registers before sleep, during sleep, and after wakeup for the working DDR2 module and for the nonworking DDR3 module.  I then compared each state between DDR2 and DDR3 to see when things deviate.  Both before sleep and during sleep the registers mostly matched except for some expected changes due to DDR clock speed.  

    Comparing the after sleep registers shows many differences, most of which are clock domains which are disabled or inactive.  I'm not sure what to conclude from this other than that the resume failed.

    Manual comparison of the register dumps: 

    ddr2_ddr3_clocks.txt
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    DDR2 -> DDR3
    h3. Comparing Before
    0x44e00040 0x00030000 -> 0x00000002
    CM_PER_ELM_CLKCTRL
    MODULEMODE Disabled to Enable
    IDLEST Disable to Functional
    0x44e0042c 0x00011317 -> 0x00001901
    CM_CLKSEL_DPLL_MPU
    DPLL_DIV divider from 22 to 1
    DPLL_MULT multiplier from 275 to 25
    0x44e00440 0x00010a17 -> 0x00019017
    CM_CLKSEL_DPLL_DDR
    DPLL_MULT multiplier from 266 to 400
    h3. Comparing standby
    Same as Before
    h3. Comparing after resume
    0x44e00000 0x18024d02 -> 0x00000102
    CM_PER_L4LS_CLKSTCTRL
    CLKACTIVITY_UART_GFCLK Act -> Inact
    CLKACTIVITY_CAN_CLK Act -> Inact
    CLKACTIVITY_TIMER2_CLK Act -> Inact
    CLKACTIVITY_LCDC_GCLK Act -> Inact
    CLKACTIVITY_TIMER5_GCLK Act -> Inact
    CLKACTIVITY_TIMER6_GCLK Act -> Inact
    0x44e0000c 0x000000d6 -> 0x00000016
    CM_PER_L3_CLKSTCTRL
    CLKACTIVITY_CPTS_RFT_GCLK Act -> Inact
    CLKACTIVITY_MCASP_GCLK Act -> Inact
    0x44e00014 0x00000002 -> 0x00070000
    CM_PER_CPGMAC0_CLKCTRL
    0x44e00018 0x00000002 -> 0x00070000
    CM_PER_LCDC_CLKCTRL
    0x44e00024 0x00000002 -> 0x00070000
    CM_PER_TPTC0_CLKCTRL
    0x44e000fc 0x00000002 -> 0x00030000
    CM_PER_TPTC1_CLKCTRL
    0x44e00100 0x00000002 -> 0x00030000
    CM_PER_TPTC2_CLKCTRL
    MODULEMODE Enable -> Disabled
    IDLEST Func -> Disable
    STBYST Func -> Standby
    0x44e00038 0x00000002 -> 0x00030000
    CM_PER_UART5_CLKCTRL
    0x44e00068 0x00000002 -> 0x00030000
    CM_PER_MCASP1_CLKCTRL
    0x44e0006c 0x00000002 -> 0x00030000
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    Archive of register dumps:

    am335x-ctt.tar.gz

  • I can't seem to connect the debugger to the CortexA8 cpu. When I try to connect, CCS and 335x lockup and eventually CCS prints Lost USB connection error and then locks up again. I have to remove power from the module to get it to work again. CCS becomes responsive once modules power has been removed. I am using the 3.2 kernel currently and CCS 6.

    Tried connecting while in u-boot with similar results.

    I don't have any problem connecting to the M3 core or the DAP.
  • Correction CCS recovers when USB is removed from debugger not when power is removed. This is when the USB errors are generated. The question is why is it all locking up?
  • It seems like the Cortex A8 is crashing upon the resume. One reason for this would be if the DDR contents is being trashed when you enter DS0. Do you see DDR_CKE go low when you enter suspend, and then come back high on resume?

    I think we need to try and test if the DDR3 contents are staying intact even after we put the memory into self-refresh.
  • Yes, I was trying to connect to the DAP using the JTAG script and dump the first 1k of ddr registers to see if it is corrupt but after resume the DAP doesn't respond.  Not sure who to verify memory contents otherwise.  Will check for DDR_CKE and DDR_reset.

    CS_DAP_DebugSS: Trouble Reading Memory Block at 0x4c000008 on Page 0 of Length 0x4: Error 0x80000002/-1144 Fatal Error during: Memory,  The memory at 0x4C000008 continually indicated it was 'not ready' The emulator was unable to regain control of the processor. The processor must be reset.  
    

    am335x-dump-mem.dss.txt
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    /*
    * Copyright (c) 2006-2014, Texas Instruments Incorporated
    * All rights reserved.
    *
    * Redistribution and use in source and binary forms, with or without
    * modification, are permitted provided that the following conditions
    * are met:
    *
    * * Redistributions of source code must retain the above copyright
    * notice, this list of conditions and the following disclaimer.
    *
    * * Redistributions in binary form must reproduce the above copyright
    * notice, this list of conditions and the following disclaimer in the
    * documentation and/or other materials provided with the distribution.
    *
    * * Neither the name of Texas Instruments Incorporated nor the names of
    * its contributors may be used to endorse or promote products derived
    * from this software without specific prior written permission.
    *
    * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
    * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
    * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
    * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
    * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
    * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
    * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
    * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
    * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
    * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
    * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
    *
    */
    var EMIF0_SDRAM_BASE_ADDR = 0x80000000;
    var SDRAM_LEN = 10*1024;
    var SDRAM_END_ADDR = EMIF0_SDRAM_BASE_ADDR + SDRAM_LEN;
    debugSessionDAP = ds.openSession("*","CS_DAP_DebugSS");
    debugSessionDAP.target.connect();
    //Build a filename that includes date/time
    var today = new Date();
    var year4digit = today.getFullYear();
    var month2digit = ("0" + (today.getMonth()+1)).slice(-2);
    var day2digit = ("0" + today.getDate()).slice(-2);
    var hour2digit = ("0" + today.getHours()).slice(-2);
    var minutes2digit = ("0" + today.getMinutes()).slice(-2);
    var seconds2digit = ("0" + today.getSeconds()).slice(-2);
    var filename_date = '_' + year4digit + '-' + month2digit + '-' + day2digit + '_' + hour2digit + minutes2digit + seconds2digit;
    var userHomeFolder = System.getProperty("user.home");
    var filename = userHomeFolder + '/Desktop/' + 'am335x-dump-mem' + filename_date + '.rd1';
    file = new java.io.FileWriter(filename);
    //file.write("DeviceName AM335x1.0\n");
    // helper function to create 8-digit hex numbers in ascii format
    function d2h(d) {return ("00000000" + (+d).toString(16)).slice(-8);}
    print("BEG:" + EMIF0_SDRAM_BASE_ADDR);
    print("END:" + SDRAM_END_ADDR);
    // read CTT data from physical addresses
    for(i=EMIF0_SDRAM_BASE_ADDR; i<SDRAM_END_ADDR; i+=4)
    {
    value = debugSessionDAP.memory.readWord(0,i,false);
    value_string = d2h(value);
    file.write('0x' + ("0000" + i.toString(16)).slice(-8) + " 0x" + value_string + "\n");
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

  • Are you able to read any registers with the DAP successfully at that point in time? You may want to use something like the PRCM registers just to verify that the DAP is operational before trying to read the DDR registers or the DDR memory.
  • Retested today to see if I could access the CLK registers through the DAP.  And I was able to.  Then retested if I could dump the first 1k of DDR3 and it worked without a debugger error.  No idea why its working today unless the computer restart fixed something...

    The DDR3 dump when the device was in standby returned all 0x0BAD values which seems reasonable since its in self refresh.  After trying to resume the DDR3 contents almost all the memory is 0x0.  I'm assuming this means the contents are not being properly refreshed during standby?

    am335x-dump-mem_2015-10-19_111124_ddr3_before.txt
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    0x80000000 0x0000003a
    0x80000004 0x0000003b
    0x80000008 0x0000003c
    0x8000000c 0x15df0801
    0x80000010 0x000400ea
    0x80000014 0x00000000
    0x80000018 0x00000000
    0x8000001c 0x00000000
    0x80000020 0x0000803a
    0x80000024 0x0000803b
    0x80000028 0x0000803c
    0x8000002c 0x199e1ba5
    0x80000030 0x0004006d
    0x80000034 0x00000000
    0x80000038 0x00000000
    0x8000003c 0x00000000
    0x80000040 0x00010000
    0x80000044 0x00010001
    0x80000048 0x00010002
    0x8000004c 0x14980042
    0x80000050 0x00040262
    0x80000054 0x00000000
    0x80000058 0x00000000
    0x8000005c 0x00000000
    0x80000060 0x0001803a
    0x80000064 0x0001803b
    0x80000068 0x0001803c
    0x8000006c 0x11981e3c
    0x80000070 0x0004006a
    0x80000074 0x00000000
    0x80000078 0x00000000
    0x8000007c 0x00000000
    0x80000080 0x00020000
    0x80000084 0x00020001
    0x80000088 0x00020002
    0x8000008c 0x13d00220
    0x80000090 0x00040045
    0x80000094 0x00000000
    0x80000098 0x00000000
    0x8000009c 0x00000000
    0x800000a0 0x0002803a
    0x800000a4 0x0002803b
    0x800000a8 0x0002803c
    0x800000ac 0x120803ca
    0x800000b0 0x0004003a
    0x800000b4 0x00000000
    0x800000b8 0x00000000
    0x800000bc 0x00000000
    0x800000c0 0x00030000
    0x800000c4 0x00030001
    0x800000c8 0x00030002
    0x800000cc 0x0cc018fa
    0x800000d0 0x00040209
    0x800000d4 0x00000000
    0x800000d8 0x00000000
    0x800000dc 0x00000000
    0x800000e0 0x0003803a
    0x800000e4 0x0003803b
    0x800000e8 0x0003803c
    0x800000ec 0x195c0051
    0x800000f0 0x0004008d
    0x800000f4 0x00000000
    0x800000f8 0x00000000
    0x800000fc 0x00000000
    0x80000100 0x00000000
    0x80000104 0x00000000
    0x80000108 0x00000000
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    am335x-dump-mem_2015-10-19_111142_ddr3_during.txt
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    0x80000000 0x0bad0bad
    0x80000004 0x0bad0bad
    0x80000008 0x0bad0bad
    0x8000000c 0x0bad0bad
    0x80000010 0x0bad0bad
    0x80000014 0x0bad0bad
    0x80000018 0x0bad0bad
    0x8000001c 0x0bad0bad
    0x80000020 0x0bad0bad
    0x80000024 0x0bad0bad
    0x80000028 0x0bad0bad
    0x8000002c 0x0bad0bad
    0x80000030 0x0bad0bad
    0x80000034 0x0bad0bad
    0x80000038 0x0bad0bad
    0x8000003c 0x0bad0bad
    0x80000040 0x0bad0bad
    0x80000044 0x0bad0bad
    0x80000048 0x0bad0bad
    0x8000004c 0x0bad0bad
    0x80000050 0x0bad0bad
    0x80000054 0x0bad0bad
    0x80000058 0x0bad0bad
    0x8000005c 0x0bad0bad
    0x80000060 0x0bad0bad
    0x80000064 0x0bad0bad
    0x80000068 0x0bad0bad
    0x8000006c 0x0bad0bad
    0x80000070 0x0bad0bad
    0x80000074 0x0bad0bad
    0x80000078 0x0bad0bad
    0x8000007c 0x0bad0bad
    0x80000080 0x0bad0bad
    0x80000084 0x0bad0bad
    0x80000088 0x0bad0bad
    0x8000008c 0x0bad0bad
    0x80000090 0x0bad0bad
    0x80000094 0x0bad0bad
    0x80000098 0x0bad0bad
    0x8000009c 0x0bad0bad
    0x800000a0 0x0bad0bad
    0x800000a4 0x0bad0bad
    0x800000a8 0x0bad0bad
    0x800000ac 0x0bad0bad
    0x800000b0 0x0bad0bad
    0x800000b4 0x0bad0bad
    0x800000b8 0x0bad0bad
    0x800000bc 0x0bad0bad
    0x800000c0 0x0bad0bad
    0x800000c4 0x0bad0bad
    0x800000c8 0x0bad0bad
    0x800000cc 0x0bad0bad
    0x800000d0 0x0bad0bad
    0x800000d4 0x0bad0bad
    0x800000d8 0x0bad0bad
    0x800000dc 0x0bad0bad
    0x800000e0 0x0bad0bad
    0x800000e4 0x0bad0bad
    0x800000e8 0x0bad0bad
    0x800000ec 0x0bad0bad
    0x800000f0 0x0bad0bad
    0x800000f4 0x0bad0bad
    0x800000f8 0x0bad0bad
    0x800000fc 0x0bad0bad
    0x80000100 0x0bad0bad
    0x80000104 0x0bad0bad
    0x80000108 0x0bad0bad
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    am335x-dump-mem_2015-10-19_111155_ddr3_after.txt
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    0x80000000 0x00000000
    0x80000004 0x00000000
    0x80000008 0x00000000
    0x8000000c 0x00000000
    0x80000010 0x00000000
    0x80000014 0x00000000
    0x80000018 0x00000000
    0x8000001c 0x00000000
    0x80000020 0x00000000
    0x80000024 0x00000000
    0x80000028 0x00000000
    0x8000002c 0x00000000
    0x80000030 0x00000000
    0x80000034 0x00000000
    0x80000038 0x00000000
    0x8000003c 0x00000000
    0x80000040 0x00000000
    0x80000044 0x00000000
    0x80000048 0x00000000
    0x8000004c 0x00000000
    0x80000050 0x00000000
    0x80000054 0x00000000
    0x80000058 0x00000000
    0x8000005c 0x00000000
    0x80000060 0x00000000
    0x80000064 0x00000000
    0x80000068 0x00000000
    0x8000006c 0x00000000
    0x80000070 0x00000200
    0x80000074 0x00000000
    0x80000078 0x00000000
    0x8000007c 0x00000000
    0x80000080 0x00000000
    0x80000084 0x00000000
    0x80000088 0x00000000
    0x8000008c 0x00000000
    0x80000090 0x00000000
    0x80000094 0x00000000
    0x80000098 0x00000000
    0x8000009c 0x00000000
    0x800000a0 0x00000000
    0x800000a4 0x00000000
    0x800000a8 0x00000000
    0x800000ac 0x00000000
    0x800000b0 0x00000000
    0x800000b4 0x00000000
    0x800000b8 0x00000000
    0x800000bc 0x00000000
    0x800000c0 0x00000000
    0x800000c4 0x00000000
    0x800000c8 0x00000000
    0x800000cc 0x00000000
    0x800000d0 0x00000000
    0x800000d4 0x00000000
    0x800000d8 0x00000000
    0x800000dc 0x00000000
    0x800000e0 0x00000000
    0x800000e4 0x00000000
    0x800000e8 0x00000000
    0x800000ec 0x00000000
    0x800000f0 0x00000000
    0x800000f4 0x00000000
    0x800000f8 0x00000000
    0x800000fc 0x00000000
    0x80000100 0x00000000
    0x80000104 0x00000000
    0x80000108 0x00000000
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

  • Jonathan Cormier said:
    After trying to resume the DDR3 contents almost all the memory is 0x0.

    Somehow you're losing all your DDR3 contents.  I would the DDR3 power rails as well as critical signals like DDR_RST and CKE.

  • Heres a capture of the CKE signal (in blue) and power usage (in red) upon resume.  CKE goes high and stays high after resume.

  • It never comes back low when you resume?! Well, that's definitely an issue. I don't understand why that would be the case. Were you able to see the EMIF registers? The PWR_MGMT_CTRL register in particular relates to this.
  • Sorry ignore my last reply. I was experiencing polarity inversion. Let me think about this some more. :-)
  • Scoping the DDR ResetN signal shows it dropping from 1.12V to .75V when going into standby. After resume it returns to 1.12V.  The .75V comes from the VTT pullup since VTT sits at .75V.  According to the datasheet VTT is correct since it should be half of VDDR (1.5V).


    Should DDR ResetN be pulled up to VTT?  The DDR3 EVM pulls the reset line up to VDDR though that could be because VTT is disabled during sleep on that hardware.

  • Jonathan Cormier said:
    Should DDR ResetN be pulled up to VTT?

    No, it should be pulled to VDDS_DDR.  This is the root of your issues.

  • Alright we will be modding our board and testing this. Please note that figure 7-47 and 7-49 in the AM335x datasheet as of may 2015 shows DDR_CKE being pulled up to DDR_VTT instead of a pulldown. And doesn't show any pullup/down for DDR_RESETn.

    Currently the mods we've had to do to our existing design:
    * Replace DDR_CKE pullup with pulldown
    * Replace DDR_RESETn pullup to DDR_VTT to a pullup to VDDS_DDR

    We also expect to have to use DDR_CKE to enable VDDR_VTT to improve power savings (as noted in EVM schematic rev1.6). I don't think we will be able to test this before spinning hardware. To be clear was the DDR_CKE used to replace the need of a controlled GPIO like that used in the EVM-SK?

    From sleep33xx.S:
    * EVM-SK Rev 1.2, has an added external pull down on CKE and the ability to disable
    * VTT regulator via gpio to address the above issues.
  • That worked!! After changing the reset pullup, I was able to resume from standby.

    root@mitysom-335x ~ $ echo standby > /sys/power/state
    [ 77.358367] PM: Syncing filesystems ... done.
    [ 77.386047] Freezing user space processes ... (elapsed 0.01 seconds) done.
    [ 77.413482] Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.
    [ 77.433502] Suspending console(s) (use no_console_suspend to debug)
    [ 77.456726] PM: suspend of devices complete after 15.472 msecs
    [ 77.461273] omap_hwmod: adc_tsc: _wait_target_disable failed
    [ 77.465515] PM: late suspend of devices complete after 8.758 msecs
    [ 81.639404] GFX domain entered low power state
    [ 81.639495] Successfully transitioned all domains to low power state
    [ 81.642303] PM: early resume of devices complete after 2.410 msecs
    [ 81.647216] PHY 0:00 not found
    [ 81.647308] am335x_vsc8601_phy_fixup 70421 here addr = 1
    [ 81.648437] net eth0: CPSW phy found : id is : 0x70421
    [ 81.759429] PM: resume of devices complete after 116.447 msecs
    [ 81.816802] Restarting tasks ... done.
    root@mitysom-335x ~ $ [ 84.646209] PHY: 0:01 - Link is Up - 1000/Full
  • Now when I test mem sleep I find that it isn't able to resume. Does mem work with the DDR3 memory? How is it different?

    root@mitysom-335x ~ $ echo mem > /sys/power/state
    [ 225.012603] PM: Syncing filesystems ... done.
    [ 225.859252] Freezing user space processes ... (elapsed 0.01 seconds) done.
    [ 225.878082] Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.
    [ 225.898162] Suspending console(s) (use no_console_suspend to debug)
  • Jonathan Cormier said:
    Currently the mods we've had to do to our existing design:
    * Replace DDR_CKE pullup with pulldown
    * Replace DDR_RESETn pullup to DDR_VTT to a pullup to VDDS_DDR

    Looks good.

    EDIT 10/21/2015:  It would be better to have no connection at all on DDR_RESETn.  The JEDEC recommendation on that pin is that it should remain low during power-up.

    Jonathan Cormier said:
    We also expect to have to use DDR_CKE to enable VDDR_VTT to improve power savings (as noted in EVM schematic rev1.6). I don't think we will be able to test this before spinning hardware. To be clear was the DDR_CKE used to replace the need of a controlled GPIO like that used in the EVM-SK?

    Yes, looks like Rev 1.6 EVM controls VTT with the DDR_CKE signal.  Nice!  No extra software configuration!


    EDIT Nov 5, 2015: The factory team does not recommend controlling the VTT regulator with CKE.  A gpio pin is recommended.

  • Jonathan Cormier said:
    Now when I test mem sleep I find that it isn't able to resume. Does mem work with the DDR3 memory? How is it different?

    root@mitysom-335x ~ $ echo mem > /sys/power/state
    [ 225.012603] PM: Syncing filesystems ... done.
    [ 225.859252] Freezing user space processes ... (elapsed 0.01 seconds) done.
    [ 225.878082] Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.
    [ 225.898162] Suspending console(s) (use no_console_suspend to debug)

    Yes, "mem" should work.  It's a lower power mode (DeepSleep0) than standby.  Did this work on your older boards with DDR2?  We're debugging a "new" issue now so you may need to go back and repeat some of the previous debug with respect to connecting to the M3 and/or DAP after you attempt to resume and interrogating the state of the device.  Does CKE come back high in this scenario or is it "stuck" low?  Did the M3 start waking stuff up, e.g. is the Cortex A8 running?

  • Note for future reference, the pullup on the reset pin broke off and I retested standby and it still works.  Reset stays high the whole time.  So at least for standby the pullup may not be necessary.

  • Brad Griffis said:
    Yes, "mem" should work.  It's a lower power mode (DeepSleep0) than standby.  Did this work on your older boards with DDR2?  We're debugging a "new" issue now so you may need to go back and repeat some of the previous debug with respect to connecting to the M3 and/or DAP after you attempt to resume and interrogating the state of the device.  Does CKE come back high in this scenario or is it "stuck" low?  Did the M3 start waking stuff up, e.g. is the Cortex A8 running?

    After the resume from 'mem', the power usage goes back up but CKE stays low. I am able to reset system using reset button which I think indicates the processor has been awoken.

    I am unable to connect to the DAP when the system is in 'mem' state. I get the following error:
    CS_DAP_DebugSS: Error connecting to the target: Error 0xC0002240/-5008 Error during: Initialization, OCS, Control,  Unknown Error

    I was able to dump memory after resume, and found that the memory was all 0's again.

  • Can you connect to the M3 when you attempt to resume?
  • I was able to run the am335x-ctt script connecting to the M3 before and after but not during.

    am335x-ctt_2015-10-19_171022_ddr3_mem_before.rd1.txt
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    DeviceName AM335x1.0
    0x44e00000 0x18024d02
    0x44e00004 0x0000000a
    0x44e00008 0x00000102
    0x44e0000c 0x000000d6
    0x44e00010 0x00070000
    0x44e00014 0x00000002
    0x44e00018 0x00000002
    0x44e0001c 0x00000002
    0x44e00020 0x00070000
    0x44e00024 0x00000002
    0x44e00028 0x00000002
    0x44e0002c 0x00000002
    0x44e00030 0x00000002
    0x44e00034 0x00030000
    0x44e00038 0x00000002
    0x44e0003c 0x00030000
    0x44e00040 0x00000002
    0x44e00044 0x00030000
    0x44e00048 0x00030000
    0x44e0004c 0x00030000
    0x44e00050 0x00030000
    0x44e00054 0x00030000
    0x44e00058 0x00030000
    0x44e00060 0x00000002
    0x44e00064 0x00000002
    0x44e00068 0x00000002
    0x44e0006c 0x00000002
    0x44e00070 0x00000002
    0x44e00074 0x00000002
    0x44e00078 0x00000002
    0x44e0007c 0x00030000
    0x44e00080 0x00000002
    0x44e00084 0x00030000
    0x44e00088 0x00030000
    0x44e0008c 0x00030000
    0x44e00090 0x00030000
    0x44e00094 0x00030000
    0x44e00098 0x00030000
    0x44e0009c 0x00030000
    0x44e000a0 0x00030000
    0x44e000a4 0x00030000
    0x44e000a8 0x00030000
    0x44e000ac 0x00000002
    0x44e000b0 0x00000002
    0x44e000b4 0x00000002
    0x44e000b8 0x00030000
    0x44e000bc 0x00000002
    0x44e000c0 0x00000002
    0x44e000c4 0x00000002
    0x44e000c8 0x00030000
    0x44e000cc 0x00030000
    0x44e000d0 0x00000002
    0x44e000d4 0x00030000
    0x44e000d8 0x00030000
    0x44e000dc 0x00000002
    0x44e000e0 0x00000002
    0x44e000e4 0x00070000
    0x44e000e8 0x00070000
    0x44e000ec 0x00000002
    0x44e000f0 0x00000002
    0x44e000f4 0x00030000
    0x44e000f8 0x00030000
    0x44e000fc 0x00000002
    0x44e00100 0x00000002
    0x44e00104 0x00030000
    0x44e0010c 0x00030000
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    am335x-ctt_2015-10-19_171110_ddr3_mem_after.rd1.txt
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    DeviceName AM335x1.0
    0x44e00000 0x00000102
    0x44e00004 0x0000000a
    0x44e00008 0x00000102
    0x44e0000c 0x00000016
    0x44e00010 0x00070000
    0x44e00014 0x00070000
    0x44e00018 0x00070000
    0x44e0001c 0x00030000
    0x44e00020 0x00070000
    0x44e00024 0x00030000
    0x44e00028 0x00000002
    0x44e0002c 0x00000002
    0x44e00030 0x00030000
    0x44e00034 0x00030000
    0x44e00038 0x00030000
    0x44e0003c 0x00030000
    0x44e00040 0x00030000
    0x44e00044 0x00030000
    0x44e00048 0x00030000
    0x44e0004c 0x00030000
    0x44e00050 0x00030000
    0x44e00054 0x00030000
    0x44e00058 0x00030000
    0x44e00060 0x00000002
    0x44e00064 0x00000002
    0x44e00068 0x00030000
    0x44e0006c 0x00030000
    0x44e00070 0x00030000
    0x44e00074 0x00030000
    0x44e00078 0x00030000
    0x44e0007c 0x00030000
    0x44e00080 0x00030000
    0x44e00084 0x00030000
    0x44e00088 0x00030000
    0x44e0008c 0x00030000
    0x44e00090 0x00030000
    0x44e00094 0x00030000
    0x44e00098 0x00030000
    0x44e0009c 0x00030000
    0x44e000a0 0x00030000
    0x44e000a4 0x00030000
    0x44e000a8 0x00030000
    0x44e000ac 0x00000002
    0x44e000b0 0x00000002
    0x44e000b4 0x00000002
    0x44e000b8 0x00030000
    0x44e000bc 0x00030000
    0x44e000c0 0x00030000
    0x44e000c4 0x00030000
    0x44e000c8 0x00030000
    0x44e000cc 0x00030000
    0x44e000d0 0x00000002
    0x44e000d4 0x00030000
    0x44e000d8 0x00030000
    0x44e000dc 0x00000002
    0x44e000e0 0x00000002
    0x44e000e4 0x00040002
    0x44e000e8 0x00070000
    0x44e000ec 0x00030000
    0x44e000f0 0x00030000
    0x44e000f4 0x00030000
    0x44e000f8 0x00030000
    0x44e000fc 0x00030000
    0x44e00100 0x00030000
    0x44e00104 0x00030000
    0x44e0010c 0x00030000
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

  • 1. Did this work on your older DDR2 hardware?
    2. How are you attempting to wake the device? Console/UART?
  • Yes this works on the DDR2 hardware.


    I am using the Console and no_console_suspend isn't enabled. I have found with the DDR2 hardware that you can't wake up from mem using console if no_console_suspend is set which makes sense.

  • Retested with no_console_suspend set in case any error messages are printed.  Using RTC to wake up device.  No error messages where printed.

    root@mitysom-335x ~ $ rtcwake -s 2 -m standby -d /dev/rtc0
    wakeup from "standby" at Fri Dec 30 19:44:59 2011
    [   76.855163] PM: Syncing filesystems ... done.
    [   76.888610] Freezing user space processes ... (elapsed 0.01 seconds) done.
    [   76.908081] Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.
    [   76.939270] PM: suspend of devices complete after 10.467 msecs
    [   76.950103] omap_hwmod: adc_tsc: _wait_target_disable failed
    [   76.960327] PM: late suspend of devices complete after 14.495 msecs
    [   78.826354] GFX domain entered low power state
    [   78.831146] Successfully transitioned all domains to low power state
    [   78.840606] PM: early resume of devices complete after 2.410 msecs
    [   78.851898] PHY 0:00 not found
    [   78.855590] am335x_vsc8601_phy_fixup 70421 here addr = 1
    [   78.862304] net eth0: CPSW phy found : id is : 0x70421
    [   78.977355] PM: resume of devices complete after 129.464 msecs
    [   78.984313] Restarting tasks ... done.
    root@mitysom-335x ~ $ [   81.856994] PHY: 0:01 - Link is Up - 1000/Full
    root@mitysom-335x ~ $ rtcwake -s 2 -m mem -d /dev/rtc0    
    wakeup from "mem" at Fri Dec 30 19:45:39 2011
    [  118.235382] PM: Syncing filesystems ... done.
    [  118.263397] Freezing user space processes ... (elapsed 0.01 seconds) done.
    [  118.289245] Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.
    [  118.327789] PM: suspend of devices complete after 17.761 msecs
    [  118.338592] omap_hwmod: adc_tsc: _wait_target_disable failed
    [  118.348846] PM: late suspend of devices complete after 14.526 msecs

1 2