This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Suspend/resume issue in EHCIPDD, BSP 6.14.01

Other Parts Discussed in Thread: SYSCONFIG

Hi all,

Just updated to version 6.14.01 of the CE BSP, and am seeing an issue with suspend/resume in EHCIPDD.

Symptom is that resume from suspend does not work reliably.

Resume from suspend either
  - just works, which is of course good, or
  - seems to work, but EVM wakes up with backlight dimmed, or
  - does not work at all, board drawing more current after resume, but display and other devices are not switched back on

I have implemented a workaround (see below), but would appreciate if someone with a better understanding of the code could comment on my observations and perhaps work out a real fix.

We are using EVM2 rev G boards with CPU board rev C and Power Module rev D.
No expansion boards connected.

Software is a modified version of the Texas BSP, CE6R3 and all updates up until and including March 2010.

This problem was not seen with BSP version 6.14.00.

EHCIPDD

The problem area seems to be in the handling of IOCTL_POWER_SET in HcdPdd_IOControl() in the file
    C:\WINCE600\PLATFORM\COMMON\SRC\SOC\OMAP35XX_TPS659XX_TI_V1\omap35xx\USB\EHCIPDD\system.c

1) The Sleep(200) call in line 1068 sometimes causes the system to hang when resuming.
I am wondering if HcdPdd_IOControl is called in a power handler context where blocking calls such as Sleep are not allowed.
Replacing the Sleep call with DelayMilliSeconds(200, g_fOmapEhciSuspended) seems to fix this.

2) Data Abort when reading pUHHRegs->SYSCONFIG during IOCTL_POWER_SET (D0) handling immediately after resume.

Exception 'Data Abort' (4): Thread-Id=027f0002(pth=8ad1c468), Proc-Id=00400002(pprc=8c011448) 'NK.EXE',
VM-active=06a50002(pprc=88fd0900) 'servicesd.exe' PC=d6322e2c(ehcihcd.dll+0x00002e2c)
RA=d6322e2c(ehcihcd.dll+0x00002e2c) SP=d5ddf000, BVA=00000000
EHCI: Exception in IOCTL_POWER_SET

Interestingly, the Data Abort only occurs if SetDevicePowerState is called before the pUHHRegs access.

The pUHHRegs pointer looks valid, it has the value 0xd0544000.

IOCTL_POWER_SET (D0) is called twice during suspend resume (why?).
The second time around the pUHHRegs pointer gets mapped to the same address (0xd0544000), but this time the access succeeds, no Data Abort.

Workaround

This workaround makes suspend/resume work reliably again, but probably also negates some of the power management improvements that 6.14.01 was supposed to bring.

1) The SetDevicePowerState and Sleep calls are only executed depending on the BusSuspendResume parameter in registry, which is set to FALSE by default.
2) Moved SetDevicePowerState and DeviceIoControl calls from HcdPdd_PowerDown to HcdPdd_IOControl to avoid system calls in the power handler.

This is the modified part of HcdPdd_IOControl:

    switch (ReqDx)
        {
        case D0:
            DeviceIoControl(pPddObject->hRootBus, IOCTL_BUS_REQUEST_CLOCK, &dwClock, sizeof(dwClock), NULL, 0, NULL, NULL);

            if (pPddObject->BusSuspendResume)
            {
                SetDevicePowerState(pPddObject->hParentBusHandle, D0, NULL);
                Sleep(200);
            }                           
           
            RegVal = pUHHRegs->SYSCONFIG & ~(UHH_SYSCONFIG_MIDLEMODE(0x03) | UHH_SYSCONFIG_SIDLEMODE(0x03));
            pUHHRegs->SYSCONFIG = RegVal | UHH_SYSCONFIG_MIDLEMODE(MIDLE_SMART) | UHH_SYSCONFIG_SIDLEMODE(SIDLE_SMART);
            HcdMdd_PowerUp(pPddObject->lpvEHCDMddObject);
            g_fOmapEhciSuspended = FALSE;
            pPddObject->dwActualPowerState = ReqDx;                    
            #if DUMP_REGS
                dumpRegs();
            #endif
            break;

        case D4:
            HcdMdd_PowerDown(pPddObject->lpvEHCDMddObject);
            g_fOmapEhciSuspended = TRUE;
            pUHHRegs->SYSCONFIG &= ~(UHH_SYSCONFIG_MIDLEMODE(0x03) | UHH_SYSCONFIG_SIDLEMODE(0x03));
            pUHHRegs->SYSCONFIG |= (UHH_SYSCONFIG_MIDLEMODE(MIDLE_FORCE) | UHH_SYSCONFIG_SIDLEMODE(SIDLE_FORCE));        

            if (pPddObject->BusSuspendResume)
            {
                SetDevicePowerState(pPddObject->hParentBusHandle, D4, NULL);
            }

            DeviceIoControl(pPddObject->hRootBus, IOCTL_BUS_RELEASE_CLOCK, &dwClock, sizeof(dwClock), NULL, 0, NULL, NULL);

            pPddObject->dwActualPowerState = ReqDx;              
            break;
        }



Incompatible transceiver

There is a comment in platform.reg to the effect that the USB transceiver used on the EVM is incompatible with the OMAP.

    ; The EVM2 ECHI port uses a transceiver that has problems with the EHCI
    ; controller PHY interface, it will fail after USB bus suspend/resume.


Please could anyone shed some light on this?
Does that mean that suspend/resume cannot be implemented on a design that uses this transceiver?

Best regards,
Carsten

 

  • Carsten,

    Thanks for the detailed analysis/first level fix suggestion.  Could you confirm a few things on your setup before your fix.

    1. When you see the problem (under the various conditions you mentioned) can you wait for a little while and see if the board recovers (after about 30 seconds). I understand that this would not be acceptable in a real use case but I just want to make sure that the issue you are seeing is same as one of the documented known issues.

    2. Do you have anything connected to the USB OTG port?

    3. I assume that the unreliable operation you are observing is rare (< 10% of the time).

    In BSP 6.14.00, EHCI was not being taken into proper suspend state and that maybe the reason it seemed to have been working reliably (and  was contributing to higher power consumption in suspend).

    Thanks.

    Jatin

  • Regarding your second question about xrcvr - I believe it corresponds to the following OMAP Errata note:

     

    Advisory 3.1.1.193 HSUSB Interoperability Issue with SMSC USB3320 PHY
    Revision(s) Affected: 3.1 and earlier
    Details: After suspend sequence, USB3320 USB PHY goes correctly in low-power mode:
    • DP Line goes High and DM line remains Low (J state)
    • Rbias Voltage = 0 V
    Whereas OMAP HOST controller exit from suspend mode (while it is expected to keep in
    low power mode).
    OMAP Host state (exited from low power mode) is inconsistent with PHY state (low
    power mode) resulting in a lockup situation.
    Resuming the port has no effect as HOST controller has already exited from low-power
    mode.
    Root cause: Delay in assertion of DIR causes USBHOST ULPI interface to exit ULPI
    Low Power mode. USB3320 USB PHY assert DIR signal 3 clock cycle after STP signal
    is de-asserted.
    Workaround(s): There is no workaround.

  • Jatin,

    Sorry for the late response, I was offline all Monday.

    1. I have tried waiting 10 minutes and the board still doesn't recover.

    I should mention that in the cases where it fails to resume I do see a debug message on RS232 immediately after I press the power button:
        -OTG SetPowerState

    Still, the system doesn't get any further than that - display does not switch on, LAN is dead, etc.

    2. There is nothing connected to the OTG- or EHCI port.

    3. Unfortunately it happens quite a lot.

    Just did 20 suspend/resume cycles, and it completely failed to resume about 50% of the time.
    Resume with backlight dimmed happened about 25% of the time, and in the remaining 25% it worked.

    Best regards,
    Carsten

     

  • Atul,

    Thanks very much - exactly the kind of information I was looking for, although I can't believe how we'd missed that.

    Best regards,
    Carsten

  • Hi Carsten,

    1) You mention in your initial post that

    "Software is a modified version of the Texas BSP, CE6R3 and all updates up until and including March 2010."
    Are you seeing the same behaviour with the unmodified version of 6.14.01 also?

    2) Since you are seeing messages on the RS232 port, I assume you are using a build which has kitl disabled and the issue is being seen with a retail build. Also, I assume that you are using the latest bootloader.

    The board not progressing beyond "-OTG SetPowerState" was seen by us when we had a USB hub connected to the OTG port and this is a known issue for this release. But we did not see issues when no devices were connected to EHCI and OTG ports. Hence, the questions above might help us understand the scenario better.

    Thanks,

    Aparna

  • Hi Aparna,

    I will test with an unmodified BSP and get back to you.

    Yes, the issue was seen in a retail build with KITL disabled, using the latest XLDR, EBoot and NK.
    I did not try a debug build and/or with KITL enabled.

    Best regards,
    Carsten

     

  • Hi again,

    Good news.
    I can't reproduce the suspend/resume problem with an unmodified version of the 6.14.01 BSP.

    I can think of two reasons why we are seeing problems with our modified version.

    1) We've made a mistake when merging a BSP update with our own code (very likely).
    If I am not mistaken we started out on 6.12.04, and have since merged 6.13.00, 6.14.00, and 6.14.01 into our codebase.

    2) One of our changes to ti_evm_3530.bat is causing the error.

    Looks like I have a bit of detective work ahead of me.
    Will write again when I've figured out what the problem is.

    Best regards,
    Carsten

     

  • Hi,

    I am seeing very similar problem as Carsten explained in the beginning of this thread. Any time I resume from suspend, EHCI fails at IO_POWER_SET. Exact output on serial is 

    Exception 'Data Abort' (4): Thread-Id=05b00012(pth=83560a0c), Proc-Id=00400002(pprc=820d2308) 'NK.EXE', VM-active=04e00006(pprc=8331cd10) 'explorer.exe'
    PC=d31a85d0(omap_ehcihcd.dll+0x000085d0) RA=d31a85d0(omap_ehcihcd.dll+0x000085d0) SP=d51debe8, BVA=00000000
    EHCI: Exception in IOCTL_POWER_SET.

    The above messages gets put out sometimes; it hangs sometimes in which case above message doesn't show up; it looks like it's resumed but backlight is off and current draws more than mode on. There's nothing connected to EHCI OTG port.

    I've tried Carsten's workaround code and that seems to make resume stable but I always gets the error message.

    Is there way to fix the EHCI so error message doesn't get put out?

    I am using SBC8530 from Embest which uses very similar BSP to EVM2. Also I'm still quite a beginner to WinCE so any help or just pointing me to some direction would be appreciated.

    Thanks,

    Aelex

  • Hello Aparna.

    You said that "this is a known problem for this release".  Could you please say was it fixed since then, if so  - in what release.

    We get this problem occasionally upon soft reset -  SetSystemPowerState(NULL, POWER_STATE_RESET, 0);

    is there any workaround? we have a hub connected, and I could try to shut it off before reset and see if that helps.

    But anyway, it's all workarounds and we want a solution.

    Thanks,

    Elad