CCS v5: High CPU usage in Debug Session

Ralf,

Ralf Goebel said:

I'm observing high host CPU usage on my system (CCS v5.4, Win7 64 Bit). After loading a program file, CCS starts using 100% on one CPU. If I restart the Debug Session and load a program again, CCS starts using another 100% on a second CPU, and so on.

Hi Rafael,

after restart of a debug session, an additional CPU is using 100%. If I relaunch again, a third CPU also uses 100%, and so on. The process which is using the CPU is always ccstudio.exe. Only a complete restart solves the problem.

This problem doesn't always apear immediately after loading a program, sometimes it happens during the debug session.

I also observed, that the CIO console output sometimes gets slower and slower over time with this emulator. At the moment, I cannot recommend this emulator.

Thanks,
Ralf

Just a small update:

I switched now to the SD XDS560v2 STM using the ethernet connection and the same problem happens after working in a debug session for some time. The problem seems to be more general. I will try to work with a new workspace now.

Ralf

Ralf,

We are a bit lost on this particular issue; if you have the time, a few logs could help get additional insights (as suggested by some developers).

A Debug Server log might help (menu Help --> CCS Support --> enable and define a filename for the field Debug Server Log). It sounds like there’s a thread that is created on every Debug Server launch and that thread is stuck in a loop which never terminates.  This is unlikely to be directly debugger related as the debugger usually only uses one thread. The debugger log might capture suspicious activity that could point a finger, but it isn’t likely.

I think a more useful thing would be a Java thread dump. I’ve never done this before, but check the instructions below:

Java thread dumps 

1. Open DOS Command window and navigate to CCSInstallRoot\ccsv#\eclipse
2. Start CCS by typing “eclipsec --launcher.ini ccstudio.ini > javadumps.log” in command-window.
3. Then, when the deadlock happens or java thread dump should be captured, place focus on the command-window, and hit Ctrl+Break.  The thread dump is now in eclipse/ javadumps.log.

One third detail I was thinking that may help is increase the heap memory used by CCS (edit the file <CCS_INSTALL_DIR>/ccsv5/eclipse/ccstudio.ini and change the option -Xmx384m to something higher such as 512m or 768m). This would be more a workaround than a real fix, but if nothing else helps it may be worth trying.

Hope this helps,

Rafael

Hi Rafael,

I enabled the Debug Server Log, but this file gets very big after a few minutes (several 100MB). Is there something I can search for in this file?

I also tried to create a java thread dump, but the created file is empty.

The heap size was already increased to 1024m. I had problems using multiple cores with the default heap configuration.

Thanks,
Ralf

I'm experiencing the same issue, I've had to give my virtual machine (I'm running ccstudio in an XP guest on an OSX host) both CPUs to make ccstudio usable one it enters its high CPU usage.

If I shut down ccstudio and restart it, it goes back to normal. I haven't been able to exactly determine what causes the high CPU as yet.

CCS v5, same problem.  Using SysInternals Process Explorer, I have captured the stack of the one of the offending threads.  Sorry for the poor formatting - the forum doesn't seem to like cut and paste from Notepad++.

ntoskrnl.exe!KeWaitForMultipleObjects+0xc0antoskrnl.exe!ExfReleasePushLock+0x8ecntoskrnl.exe!PoStartNextPowerIrp+0x331ntoskrnl.exe!PoStartNextPowerIrp+0x17e7ntoskrnl.exe!PoStartNextPowerIrp+0x1a97jvm.dll!AsyncGetCallTrace+0x1059fjvm.dll!_JVM_EnqueueOperation@20+0x5de21jvm.dll!_JVM_EnqueueOperation@20+0x3df44jvm.dll+0x844fjvm.dll!_JVM_EnqueueOperation@20+0x3cb6fkernel32.dll!WaitForSingleObjectEx+0x43jvm.dll!_JVM_EnqueueOperation@20+0x3e1a3jvm.dll+0x10dfjvm.dll!_JVM_EnqueueOperation@20+0x5de21jvm.dll!_JVM_EnqueueOperation@20+0x5de30jvm.dll!_JVM_FindSignal@4+0x71b38jvm.dll!_JVM_EnqueueOperation@20+0x5de21jvm.dll!_JVM_EnqueueOperation@20+0x5de30jvm.dll!_JVM_FindSignal@4+0x71b38jvm.dll!_JVM_EnqueueOperation@20+0x5de21ntdll.dll!RtlQueryEnvironmentVariable+0x241jvm.dll!_JVM_FindSignal@4+0x2f80djvm.dll!AsyncGetCallTrace+0x27c8cjvm.dll!_JVM_FindSignal@4+0x50b61jvm.dll!AsyncGetCallTrace+0x28224

Greg,

Are you are using Process Explorer already, could you create a few minidumps for the ccstudio.exe process before and after you experience the problem and post them here? This is done by right clicking on the process and selecting the "create dump" menu.  By creating a few of them I can potentially compare the thread states.

I'm not sure if it will help point a finger, but it would a good place to start.

Is there a more secure way of getting this info to you guys rather than posting it in a public forum.  I'd like to help, but you never know what a memory dump is going to contain.

Sure, and thanks for your help!

There are two options to choose between depending on the file content and the sizes. If the content isn't too large you can email it directly to me using the address in my profile. A second option is for me to create a password protected FTP site. In that case, please also send me an email and I'll reply with the specifics.

I am seeing a similar problem, but in this case the TraceServer.exe is using nearly 100% of a CPU in system mode.  I know that I used to edit code while debugging in previous versions, but this cpu utilization issue means that I have to disconnect before editing any code, because the editor is extremely unresponsive.

I am using CCS 5.5.0.201308270800

Here is the stack dump of the offending thread:

ntoskrnl.exe!KeWaitForMultipleObjects+0xc0a
ntoskrnl.exe!KeAcquireSpinLockAtDpcLevel+0x732
ntoskrnl.exe!KeWaitForMutexObject+0x19f
ntoskrnl.exe!__misaligned_access+0xba4
ntoskrnl.exe!__misaligned_access+0x1821
ntoskrnl.exe!KiCheckForKernelApcDelivery+0x25
ntoskrnl.exe!PsDereferenceKernelStack+0x808e
ntoskrnl.exe!LpcRequestPort+0x8ed
ntoskrnl.exe!TmCurrentTransaction+0x1046
ntoskrnl.exe!NtQuerySystemInformation+0x4d
ntoskrnl.exe!KeSynchronizeExecution+0x3a23
ntdll.dll!NtQuerySystemInformation+0xa
wow64.dll!Wow64EmulateAtlThunk+0xa447
wow64.dll!Wow64EmulateAtlThunk+0xacd6
wow64.dll!Wow64EmulateAtlThunk+0xc789
wow64.dll!Wow64SystemServiceEx+0xd7
wow64cpu.dll!TurboDispatchJumpAddressEnd+0x2d
wow64.dll!Wow64SystemServiceEx+0x1ce
wow64.dll!Wow64LdrpInitialize+0x429
ntdll.dll!RtlIsDosDeviceName_U+0x24c87
ntdll.dll!LdrInitializeThunk+0xe
ntdll.dll!ZwQuerySystemInformation+0x12
kernel32.dll!CreateToolhelp32Snapshot+0x5b
TraceServer.exe!?Delete@cToolsError@Ctools@Sds@Ti@@UAEXXZ+0x2ab1c
TraceServer.exe!?Delete@cToolsError@Ctools@Sds@Ti@@UAEXXZ+0xc27
TraceServer.exe!?Delete@cToolsError@Ctools@Sds@Ti@@UAEXXZ+0x296a5
MSVCR90.dll!_endthreadex+0x44
MSVCR90.dll!_endthreadex+0xd8
kernel32.dll!BaseThreadInitThunk+0x12
ntdll.dll!RtlInitializeExceptionChain+0x63
ntdll.dll!RtlInitializeExceptionChain+0x36

Has there been any movement on this issue? I am currently reproducing this problem on a Win7 x86 VPC running CCS 5.5.0.00077. The host is a Lenovo T530 running Win7 x64 Enterprise. The VPC has been assigned 2GB of memory, though this appears to be a problem with CPU usage only.

My symptoms are very similar, except that since I'm in a VM the process will only ever consume 1 core (VPC can/will only assign 1 core per VM). This behavior is triggered consistently by entering a debug session and isn't fixed until I exit CCS and re-open the workspace. It's making work very, very painful.

I'm using CCSv6 for some time now and I didn't notice this problem with the new version.

By the way, my PC is also a Lenovo T530 running Windows 7 x64.

Ralf

Hi Ralf,

Thanks for the info. My end customer is still using CCS v5.5, but I'll look into whether it makes sense for us to upgrade to CCS v6 at this point in the project. Since posting, I have found a couple ways to mitigate this problem (but I can still reproduce it readily).

1) Make sure that the workspace has only the necessary number of projects open. This is a general performance enhancement recommended by TI; the workspace I'm using has ~10 medium sized projects and I'm only actively using 1-2, so closing the balance of the other projects notably improved responsiveness. I wish CCS was a bit better at ignoring projects which aren't dependencies or actively being worked on, but this is satisfactory workaround for my use case.

2) When running a debug session, make sure to use the Stop button to end the debug session, especially if you're about to launch another debug session. My work flow was to debug for a while, make code changes from the debug view, and then re-launch debug immediately (generating the "you'll have to stop your current debug session to do this" prompt). This seems to have an influence on how often the problem reproduces, but it could just be a placebo.

It may just be that I'm debugging less often as the code base has matured, but these two steps seem to have helped. After closing all the extra projects, I've found that the exit/restart cycle time of CCS is much more bearable, so it's not as much of an inconvenience to restart the IDE when the problem repros.

Chris Beuschel said:

My end customer is still using CCS v5.5, but I'll look into whether it makes sense for us to upgrade to CCS v6 at this point in the project.

Even if you have a different cause of a high CPU load, that other thread might have some ideas on how to find what component of CCS is causing the high CPU load.

Chester Gillon said:

I don't know if it is related, but did find CCS 5.5 high CPU load when TI Resource Explorer open. In my case I found that closing the TI Resource Explorer reduced the CCS CPU load (and this problem should be fixed in CCS 6).

Even if you have a different cause of a high CPU load, that other thread might have some ideas on how to find what component of CCS is causing the high CPU load.

Thanks - I did find your thread while researching the issue and that's what lead me to mess with the workspace. If there was other advice to root cause or fix the problem (I don't recall, now) it wasn't effective for me, unfortunately.