OMAP3530 MUSB Driver and Power Management

A few additional points:

I do believe it's reasonable that adding the MUSB driver might increase the power required by the system--when the MUSB is not in the OFF state.

When the MUSB driver is in the kernel or after removing it the core powerdomain won't even go to inactive, even when try to suspend to RAM:

usbhost_pwrdm (RET),OFF:2,RET:3,INA:0,ON:4
sgx_pwrdm (OFF),OFF:1,RET:0,INA:0,ON:1
per_pwrdm (ON),OFF:165,RET:1221,INA:0,ON:1387
dss_pwrdm (RET),OFF:1,RET:2,INA:0,ON:1
cam_pwrdm (RET),OFF:1,RET:4,INA:0,ON:3
core_pwrdm (ON),OFF:0,RET:0,INA:0,ON:1
neon_pwrdm (ON),OFF:166,RET:1199,INA:21,ON:1387
mpu_pwrdm (ON),OFF:166,RET:1199,INA:21,ON:1387
iva2_pwrdm (RET),OFF:1,RET:2,INA:0,ON:1
per_clkdm->per_pwrdm (9)
usbhost_clkdm->usbhost_pwrdm (0)
cam_clkdm->cam_pwrdm (0)
dss_clkdm->dss_pwrdm (0)
core_l4_clkdm->core_pwrdm (7)
core_l3_clkdm->core_pwrdm (5)
d2d_clkdm->core_pwrdm (0)
sgx_clkdm->sgx_pwrdm (0)
iva2_clkdm->iva2_pwrdm (0)
neon_clkdm->neon_pwrdm (0)
mpu_clkdm->mpu_pwrdm (0)
prm_clkdm->wkup_pwrdm (0)
cm_clkdm->core_pwrdm (0)

Chris,

There are few more patches on top of 03.00.01.06 at http://arago-project.org/git/people/?p=sriram/ti-psp-omap.git;a=summary

Please apply all the remaining patches.

>>Occasionally with MUSB installed it will allow the core powerdomain to retention and off, but that's rare

I think here you are referring to idle core off. This depends on all the other peripherals also. If all of them allow to enter off mode then only this would happen. Also this requires us to enable "sleep_while_idle" and "enable_off_mode" flags. You can find details on this in 03.00.01.06 UserGuide.

>>The frustrating thing is that if I insmod the MUSB driver into the kernel and then remove it, it can't get to the low power states that it could before the MSUB driver was put in.

This is known and is due to removal of usb suspend function call from idle path if USB is built as module. this is to fix the compilation error as idle path code would not be able to see usb suspend function until module is inserted. Moreover when usb module is not inserted and idle path get executed then usb suspend function has to be stub.

>>Are there any subtle configuration issues for the MUSB driver?

No.

>>Do you have any suggestions for diagnosing the problem?

Just see if any other peripheral is active and preventing core off. May be try disabling other modules and let USb alone to be in system.

Regards,
Ajay

Ajay Wrote:

>> There are a few more patches on top of 03.00.01.06 at http://arago-project.org/git/people/?p=sriram/ti-psp-omap.git;a=summary

>> Please apply all the remaining patches.

Thanks.  I have now applied all those MUSB patches.

>>  >> Occasionally with MUSB installed it will allow the core powerdomain to retention and off, but that's rare

>> I think here you are referring to idle core off. This depends on all the other peripherals also. If all of them allow to enter off mode then only this would happen. Also this requires us to enable "sleep_while_idle" and "enable_off_mode" flags. You can find details on this in 03.00.01.06 UserGuide.

I have already been using the sleep_while_idle and voltage_off_while_idle flags, and sometimes the enable_off_mode flags.  In order to see how well those flags are working I do

cat /debug/pm_debug/count

which shows

usbhost_pwrdm (OFF),OFF:1,RET:2,INA:0,ON:2
sgx_pwrdm (OFF),OFF:1,RET:0,INA:0,ON:1
per_pwrdm (ON),OFF:2730,RET:1519,INA:0,ON:4250
dss_pwrdm (OFF),OFF:1,RET:1,INA:0,ON:1
cam_pwrdm (OFF),OFF:1,RET:3,INA:0,ON:3
core_pwrdm (ON),OFF:0,RET:0,INA:0,ON:1
neon_pwrdm (ON),OFF:2720,RET:1516,INA:13,ON:4250
mpu_pwrdm (ON),OFF:2720,RET:1516,INA:13,ON:4250
iva2_pwrdm (OFF),OFF:1,RET:1,INA:0,ON:1
per_clkdm->per_pwrdm (9)
usbhost_clkdm->usbhost_pwrdm (0)
cam_clkdm->cam_pwrdm (0)
dss_clkdm->dss_pwrdm (0)
core_l4_clkdm->core_pwrdm (7)
core_l3_clkdm->core_pwrdm (5)
d2d_clkdm->core_pwrdm (0)
sgx_clkdm->sgx_pwrdm (0)
iva2_clkdm->iva2_pwrdm (0)
neon_clkdm->neon_pwrdm (0)
mpu_clkdm->mpu_pwrdm (0)
prm_clkdm->wkup_pwrdm (0)
cm_clkdm->core_pwrdm (0)

you can see that the core powerdomain is never anything but ON.  It never reaches off, retention, or even inactive since I have built the MUSB driver into the kernel.  If I don't build MUSB into the kernel then with sleep_while_idle, voltage_off_while_idle, and enable_off_mode flags all set to one, it will go to retention frequently, and when I suspend to RAM

echo -n "mem" > /sys/power/state

the core powerdomain will go to off mode, and my system's current draw drops significantly.

When I have MUSB in the kernel and I try to suspend to RAM, it seems to succeed, but it says upon awakening either

Powerdomain (core_pwrdm) didn't enter target state 1

Could not enter target state in pm_suspend

or

Powerdomain (core_pwrdm) didn't enter target state 0

Could not enter target state in pm_suspend

depending on whether enable_off_mode is set.  Without MUSB in the kernel it will go to the target state (either 0 or 1) just fine.  In addition, in order to confirm that the we really are reaching the full off mode, I check various registers in the /debug/pm_debug/registers/2 pseudo-file after attempting to suspend to RAM.  When MUSB is in the kernel I will see:

  e8 => 00000000        PM_PREPWSTST_IVA2
  e8 => 00000000        PM_PREPWSTST_MPU
  e8 => 000000f7        PM_PREPWSTST_CORE
  e8 => 00000000        PM_PREPWSTST_SGX
  e8 => 00000000        PM_PREPWSTST_DSS
  e8 => 00000000        PM_PREPWSTST_CAM
  e8 => 00000000        PM_PREPWSTST_PER
  e8 => 00000000        PM_PREPWSTST_NEON
  e8 => 00000000        PM_PREPWSTST_USBHOST

indicating that we didn't get to off mode.  When MUSB is not in the kernel I see:

  e8 => 00000000        PM_PREPWSTST_IVA2
  e8 => 00000000        PM_PREPWSTST_MPU
  e8 => 00000000        PM_PREPWSTST_CORE
  e8 => 00000000        PM_PREPWSTST_SGX
  e8 => 00000000        PM_PREPWSTST_DSS
  e8 => 00000000        PM_PREPWSTST_CAM
  e8 => 00000000        PM_PREPWSTST_PER
  e8 => 00000000        PM_PREPWSTST_NEON
  e8 => 00000000        PM_PREPWSTST_USBHOST

Which seems to indicate that I actually got the system to go to off-mode in the suspend state.

>>  >>The frustrating thing is that if I insmod the MUSB driver into the kernel and then remove it, it can't get to the low power states that it could before the MSUB driver was put in.

>>  This is known and is due to removal of usb suspend function call from idle path if USB is built as module. this is to fix the compilation error as idle path code would not be able to see usb suspend function until module is inserted. Moreover when usb module is not inserted and idle path get executed then usb suspend function has to be stub.

OK.  I guess we need to keep MUSB built into the kernel.

Can we remove twl4030-usb.ko and reinsert it without problem?  (That seems to help with power by turning off VUSB_1v5, VUSB_1v8, and VUSB_3v1 when we remove it.  Is there a better way to turn those off?)

>>  >>Do you have any suggestions for diagnosing the problem?

>>  Just see if any other peripheral is active and preventing core off. May be try disabling other modules and let USB alone to be in system.

I have been unsuccessful trying to get CORE to go to any state but ON since I have built MUSB into the kernel today.

When I have done MUSB testing in the past the CM_IDLEST1_CORE register indicates that bits 2, 4, 5, and 6 are not 1, which correspond to the SDRC, SCM, and HS OTG USB are holding it up.  Today I also get bits 13 & 14 (the UART1 and UART2 idle status) holding it up.

Chris,

which platform are you using? As you have mentioned twl4030 so it seems you are not using OMAP3EVM. PSP-03.00.01.06 release has been tested against OMAP3EVM which uses ISP1507 PHY and not twl4030 thus twl4030 is disabled in omap3_evm_defconfig.

Another thing to check is if you have any devices connected to MUSB port ? if so then disconnect all the devices to exercise idle off.

Regards,
Ajay

Thanks very much for your help, Ajay.

We are using our own platform that has the MUSB port using the TPS65950's PHY.  Are there any known problems with this configuration, particularly as far as power management is concerned?

Ours is also an embedded system that has something constantly connected to the USB port, though we can (and do) turn its power off.  Is that sufficient for idle-off?

If you turn off the attached devices then only musb will allow idle core off so please turn off your device. I think this should be sufficient.

Regards,
Ajay

Hi Ajay,

I'm still struggling with getting the OTG port (host mode, not using gadget) into idle.  It's always hard on still...

Can the PHY keep the MUSB hardware from going into idle?

Do you know if there are any serious issues with the TPS65950 PHY Linux driver or HW?

(I guess I'm a little concerned because the EVM decided not to enable the TPS's PHY by default.  This implies to me that there may be some problem with the PHY or its driver.)

Since the MUSB driver is being built into the kernel, is it OK for the twl4030-usb driver to be a module, or must it be built into the kernel?  Does it have to be built in for proper idle mode of MUSB to work?

Thanks,

Chris

Chris,

I have not tested TPS65950 phy based boards for power management so not sure on the observation you have. I think there is TPS65950 PHY suspend/resume feature which is getting done in drivers/usb/otg/twl4030-usb.c file.

As such I am not aware of any known issue on TPD65950 device. I would advice you to have twl4030 also built in.

Regards,
ajay

Thanks, Ajay.

I am still unsuccessful, so I have additional questions, comments...

1.  I looked at the OTG_SYSCONFIG register and it is 0x00002010.  This indicates smart-idle.  This page, http://processors.wiki.ti.com/index.php/OMAP35x_Debug_Steps_for_Idle_Entry, indicates that it should be force-idle.  Is that correct?

2.  Will the MUSB hardware go into idle (inactive, retention, and/or off) automatically if nothing is connected, or do you have to enter suspend mode?

     a.  Is there any specific configuration to obtain this?

3.  What registers should I check to make sure MUSB should go inactive?

     a.  What registers would indicate the thing that is keeping MUSB active instead of allowing it to go inactive?

Thanks for your assistance,

Chris

>>1.  I looked at the OTG_SYSCONFIG register and it is 0x00002010.  This indicates smart-idle.  This page, http://processors.wiki.ti.com/index.php   OMAP35x_Debug_Steps_for_Idle_Entry, indicates that it should be force-idle.  Is that correct?

force-idle in not always needed. Smart idle should be enough to make musb enter into idle.

>>2.  Will the MUSB hardware go into idle (inactive, retention, and/or off) automatically if nothing is connected, or do you have to enter suspend mode?

>>     a.  Is there any specific configuration to obtain this?

If nothing is connected then it should enter into idle mode.

>>3.  What registers should I check to make sure MUSB should go inactive?
I
>>     a.  What registers would indicate the thing that is keeping MUSB active instead of allowing it to go inactive?

I am not aware of any such register. Can you try below patch which may shoud help in cutting the clocks during idle path.

http://arago-project.org/git/people/?p=sriram/ti-psp-omap.git;a=commitdiff;h=7719b06aaac0b5bdd02177308e5fdf82951b7e67

-Ajay

That patch looks to be in the kernel development line for PSP 3.00.01.06, but doesn't seem to have made it into the release.

Can you help me understand why it was omitted?

Are there any other patches that might relate to idle mode that I should be aware of?

Thanks again,

Chris

Chris,

this patch was added to cut the musb clocks during idle but it was dropped as it had issues due to current usb driver architecture and idle path handling. We were facing issues to integrate it along with USB power management from usb stack.

Regards,
Ajay

Thanks, Ajay, for your quick reply...

Have you had a chance to test it lately?  Do you still expect it to have issues?

What issues should I expect to encounter?

Is it still planned for some future integration?

I also started an e2e thread (http://e2e.ti.com/support/power_management/pmu/f/43/p/73232/266070.aspx#266070) on the PMU side for twl4030-usb driver questions, one of which you might be able to answer:

Can I remove the NOP PHY driver if I'm using the twl4030-usb driver?

Thanks,

Chris

Hello Chris,

Could you try "# cat /debug/pm_debug/registers/1" to dump PRCM registers snapshot taken just before suspend (just before jump into SRAM idle code) and post the snapshots for both w/ or w/o the MUSB driver ? All check items listed in the wiki have to be checked just before suspend. Attached is the PRCM register snapshot just before a succesful suspend entry for your reference.

Here are the two files:

5672.registers1+MUSB.txt

6013.registers1-MUSB.txt

  The only difference in the Linux kernel is saying "y" to the line in  make menuconfig:

<*>   Inventra Highspeed Dual Role Controller (TI, ADI, ...)

As far as the checklist items from the wiki, I have been able to confirm #1-4.

I don't have an easy ways to check 5, as the IRQ status registers aren't saved by the /debug/pm_debug/registers/ mechanism. 

For #6-8, I know there are problems with CM_IDLEST and CM_ICLKEN, and those bits point to MUSB (and sometimes to UART1 and UART2).  It is really strange to me that the CM_IDLEST1_CORE shows UART1 & UART2 not idle, and CM_ICLKEN1_CORE shows their clocks enabled when the only thing I have done is add MUSB support.  FCLKEN1_CORE also shows the UART functional clocks as being active.

For 10-16, the SYSCONFIG registers aren't saved with the /debug/pm_debug/registers/ mechanism, but I tried to cause that to be done.  It seems that the clock domains for those registers are turned off by the time the /debug/pm_debug/registers/1 saving of registers occurs, so I haven't gotten it to work yet.

#17 is confirmed:

~ # fgrep AUTOIDLE /debug/pm_debug/registers/1
  34 => 00000001        CM_AUTOIDLE_PLL_IVA2
  34 => 00000001        CM_AUTOIDLE_PLL_MPU
  30 => fffffed9        CM_AUTOIDLE1_CORE
  34 => 0000001f        CM_AUTOIDLE2_CORE
  38 => 0000000c        CM_AUTOIDLE3_CORE
  30 => 0000003f        CM_AUTOIDLE_WKUP
  30 => 00000009        CM_AUTOIDLE_PLL
  34 => 00000001        CM_AUTOIDLE2_PLL
  30 => 00000001        CM_AUTOIDLE_DSS
  30 => 00000001        CM_AUTOIDLE_CAM
  30 => 0003ffff        CM_AUTOIDLE_PER
  30 => 00000001        CM_AUTOIDLE_USBHOST

I'm not sure how to check 18 & 19.  I expect the Linux kernel to do those things.  Do you know of cases where it fails to do this?

For #20, I checked SDRC_POWER_REG and the self-refresh bit wasn't set, but the code in sleep34xx.S seems to set it.

For #21, I note that CM_CLKSTCTRL_CORE is 0 when MUSB is installed, but should be 0x0f.  There are functions to control this in clockdomain.c, but I've not yet traced down when/where it would be called for CORE.  I also note, however, that the CM_CLKSTCTRL_MPU register is 0, but it does go into idle, so this isn't conclusive.

For #22, PM_PWSTCTRL_CORE, the POWERSTATE bit looks like it's not getting set to OFF in the case with the MUSB driver.  Otherwise it looks OK as far as I know, but I do note that it sets the SAVEANDRESTORE bit for the USB TLL module.  I don't know how this should be set.

I assume Linux handle #23 correctly.

Chris,

I haven;t worked on that patch there after and also not planned for future release. Anyways suspend to memory is supported and that works fine on EVM. I think this also works for you.

If you are using twl4030 PHy then you should remove NOP.

Regards,
Ajay

I am now building the twl4030-usb driver into the kernel (and have removed the NOP XCEIV driver).

I used to use the insmod/rmmod of that driver to turn on and off the VUSB1V5, VUSB1V8, and VUSB3V1.  Since the driver is now built in to the kernel, I need a new mechanism to turn those supplies off when I don't need the USB, since they consume a number of mA of current.

Is it OK to power down those supplies by some other means?  Or will that mess up the PHY?

Can those be automatically turned off when the MUSB enters idle?

Thanks,

Chris

Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4

Questions that remain:

1.   1.      Are there any other MUSB or PHY hardware issues that could be keeping MUSB active?

2.  2.       Is it safe to power off the vusb1v5 and vusb1v8 via direct I2C commands to change their device group to 0, and then back on when I need them?

a.       Is there a way to do that using the twl4030-usb driver instead?

b.      I tried turning them off via a method I developed, letting them dissipate for a few minutes and then turning them back on.  This seemed to work, but I’d like some official word that would indicate that it’s OK.

.  .  3.      The patch that I was sent sets some flags for when we want the MUSB to go idle, but there wan’t any patch for coming out of idle.  Should there be?

4.   4.      What is the connection between MUSB and UAR1/UART2 and the 48MHz clock?

a.       I do see a 48MHz clock active in CM_IDLEST_CKGEN (= 0x0000020b).  That register also shows DPLL4’s 96MHz clock active, as well as DPLL4 (ST_PERIPH_CLK) and DPLL3 (ST_CORE_CLK).

b.      Does the 48MHz clock active give a idea as to what is holding MUSB active?

4.   5.       I see the Per, MPU, and Neon domains transitioning regularly between ON and OFF state in cat /debug/pm_debug/count’s output.  However the PER clock seems to be active according to  cat /debug/pm_debug/registers/1’s output.  Which one is correct?

4.   6.      What Pinmux/Padconf settings for the MUSB pins are recommended for off-mode?

7.   7.      Does anyone have any ideas of which padconf values could cause either UART1/2 or MUSB to not be willing to go idle?

 8   8.      Are there any places in the Linux kernel code you can recommend to diagnose the UARTs not sleeping?

Chris,

Please find my comments inline.

Normal 0 false false false MicrosoftInternetExplorer4

>> Are there any other MUSB or PHY hardware issues that could be keeping MUSB active?

[Ajay] I am not aware of any issue with PM.

>>Is it safe to power off the vusb1v5 and vusb1v8 via direct I2C commands to change their device group to 0, and then back on when I need them?

[Ajay] I think it’s safe.

>>Is there a way to do that using the twl4030-usb driver instead?

[Ajay] Twl4030 driver is doing it in suspend/resume function. You can refer that part of code to do this.

>>I tried turning them off via a method I developed, letting them dissipate for a few minutes and then turning them back on.  This seemed to

>>work, but I’d like some official word >>that would indicate that it’s OK.  That might be a Paul Eaves or an Ajay question.

[Ajay] I think it should be safe.

>>The patch that I was sent sets some flags for when we want the MUSB to go idle, but there wan’t any patch for coming out of idle.  Should there be?

[Ajay] Resume function already does the necessary stuff.

Regards,

Ajay

Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4

Ajay,

Thanks for your answers.  Does anyone else have answers for my remaining questions?

4.      What is the connection between MUSB and UAR1/UART2 and the 48MHz clock?

a.       I do see a 48MHz clock active in CM_IDLEST_CKGEN (= 0x0000020b).  That register also shows DPLL4’s 96MHz clock active, as well as DPLL4 (ST_PERIPH_CLK) and DPLL3 (ST_CORE_CLK).

b.      Does the 48MHz clock active give a idea as to what is holding MUSB active?

5.       I see the Per, MPU, and Neon domains transitioning regularly between ON and OFF state in cat /debug/pm_debug/count’s output.  However the PER clock seems to be active according to  cat /debug/pm_debug/registers/1’s output.  Which one is correct?

6.      What Pinmux/Padconf settings for the MUSB pins are recommended for off-mode?

7.      Does anyone have any ideas of which padconf values could cause either UART1/2 or MUSB to not be willing to go idle?

8.      Are there any places in the Linux kernel code you can recommend to diagnose the UARTs not sleeping?

Chris,

Please see my replies

4     What is the connection between MUSB and UAR1/UART2 and the 48MHz clock?

[Ranjith] 48MHz clock acts as the functional clock for UART1/2, McSPI etc.. Unless these module level functional clocks are gated, the 48M_FCLK wont be idled. You can refer TRM section 4.7..7 

a.       I do see a 48MHz clock active in CM_IDLEST_CKGEN (= 0x0000020b).  That register also shows DPLL4’s 96MHz clock active, as well as DPLL4 (ST_PERIPH_CLK) and DPLL3 (ST_CORE_CLK).

[Ranjith] See my above comment

b.      Does the 48MHz clock active give a idea as to what is holding MUSB active?

[Ranjith} Unfortunately no

5.       I see the Per, MPU, and Neon domains transitioning regularly between ON and OFF state in cat /debug/pm_debug/count’s output.  However the PER clock seems to be active according to  cat /debug/pm_debug/registers/1’s output.  Which one is correct?

[Ranjith] Both of them used to give correct information at that point in time when it was captured. There are certain clocks which can be gated at the very end and the register snapshot doesnt capture that. But these are not the ones that you are seeing as active. ex. UART1/2 all should be idled at the point where snapshot is taken

6.      What Pinmux/Padconf settings for the MUSB pins are recommended for off-mode?

[Ranjith] Can you provide what additional padconf changes you have done (taking the EVM configuration as base)? We can then check with the hardware team.

7.      Does anyone have any ideas of which padconf values could cause either UART1/2 or MUSB to not be willing to go idle?

[Ranjith] Unfortunately no

8.      Are there any places in the Linux kernel code you can recommend to diagnose the UARTs not sleeping?

{Ranjith} I would recommend the following

1. Put a debug print at the beginning of function omap_sram_idle in pm34xx.c

2. enable off mode by writing 1 to /debug/pm_debug/enable_off_mode

3. issue a suspend operation

4. Check whether the system is sitting in the suspended state waiting for some user activity or whether you are seeing it automatically resuming

5.Resume the system

6 Take register snapshot

7. Provide us the entire console log 

Thanks,

Ranjith

My Responses:

6.  My padconf/pinux settings changes in U-Boot are here: 4375.u-boot-mux-diffs.txt, though I don't think any of them change UART or MUSB pins.

     The padconf/pinux settings in my board file for Linux are here: 5100.linux_mux_changes.txt, though I don't think any of them change UART or MUSB pins.

     A couple more from a different file:

        OMAP3_MUX(MCSPI1_CS2, OMAP_MUX_MODE4 | OMAP_PIN_INPUT_PULLUP),
        OMAP3_MUX(MCBSP1_FSR, OMAP_MUX_MODE4 | OMAP_PIN_INPUT_PULLUP |   \
                OMAP_PIN_OFF_INPUT_PULLUP),

8.  Following the outlined 7 steps, here is the console log:  3157.log-for-usb-sleep.txt

The system sits in the suspended state, waiting for a wakeup event, which I provided with a keystroke.  I printed out both the before and after sleep states (registers/1 and registers/2).

Thanks so much for your attention and assistance,

Chris

I am seeing a number of cases where I get kernel output that says:

prcm: WARNING: PRCM interrupt received, but no code to handle it (00200000)

and it also happens where the hex number is 00018400, 00008000, 00210400, 00000400, 00008400, etc. where each bit set is in the reserved bit section of the PRM_IRQSTATUS_MPU register.  Can I simply mask and ignore this, or is there something there that might be causing the CORE to be awake because of this?

Chris,

Currently I am having the same problem: MUSB prevents core to go into idle mode. Did you finally resolved that issue ?

I didn't get the issue resolved completely.  I did attempt to use the driver in the Android 2.6.32 kernel for the omap, and that seemed to help some.  My hardware was a bit different, and also, I didn't need to get it to go idle automatically, so I hacked together an internal command that would allow it to go idle, since the hardware wasn't triggering it.  However, we had some other issues, and didn't end up using the code.

Sorry...