TMS320C6678: CPU power down status

Ilan.

What are you trying to do?

Do these links answer your questions:

https://e2e.ti.com/support/processors/f/791/t/517935

https://e2e.ti.com/support/processors/f/791/t/379766

https://e2e.ti.com/support/processors/f/791/t/279236

Tom

Hi, 

Thank you for your quick response.

My questions are related to my Original question "CPU freeze or sleep", which the CPU is seemed to be in freeze or sleep state.

I am trying to understand in what state the CPU was at that moment. For example if it was powered down unintentionally is there any sign for it.

I read these e2e cases before and i didn't get the answer.

Best regards'

Ilan

Is there any status that the CPU was in sleep state (powered down ) ?

The PSC registers for PDSTAT and MDSTAT registers for CorePacs can give you status of cores "currently" in clock or power off state. These registers will not give "post" status ie. if they were previously in power down (PD or clock off) and now enabled. 

If the CPU starves for memory resource, the TSCL continues to count ?

 Define starve? Unless the CPU is in a bad state, TSC registers should count until reset , any level of CPU activity/loading will not stop these counters. 

The PDC_INT (power down sleep interrupt), what the process that produces this event ?

 I have not seen this used much , but I expect that if you try to power down a CorePac via PSC registers , this interrupt is asserted - usually a core (host) can request clock stop etc to another core and this clock stop request will generate the PDC_INT 

One more thing , although I think you are aware of it. From your other post it looks like you are comparing Timer 8 and TSC time count - hopefully you have the calculations correct  in terms of i think the on chip timers run CPU/ 6 and TSC counts in terms of cpu cycles. 

Hi,

Thank you,

I know that the TSC input is the CPU clock and the timer 8  input is CPU/6. In relate to my other post, most of the time the timers measure the same time. till the problem happens  and after  that the CPU continues OK till it happens again.

The problem happens in other CPUs not at tne same time.

If it is hardware problem with the input CPU clock (custom board)  it would happened at all processor CPUs at the same time, or not if it depends on the CPU activity ?

Best Regards,

Ilan

Ilan,

What is your observation on these various timers when your 'problem' happens?

Tom

Hi,

Usually the CPU1 TSC and the timer 8 measure the same value for the same event, but  in one case of the bug, the Timer 8 measured 43 milli seconds and the TSC measured only 131 micorseconds. In other case CPU 2 TSC measured 300 milliseconds, while CPU 3 measured only 3 milliseconds for the same interval time.

Regards,

Ilan

Ilan,

Are you able to replicate this on an EVM?

Tom

Hו,

We run the CPU controller separately on the EVM.

And the two other CPUs (stage 1 & 2) were run on other EVM, with much slower rate of the input data.

And the bug was not detected.

To run the all application parts on the same EVM is too complicated.

Regards,

Ilan

Ilan,

Can the application be cut down so that only the parts using the timers be executed on the EVM?  We need something that can be replicated on the EVM before we can see what you are describing.  The behavior being seen does not sound possible to us.

Tom

Hi,

We did so, as i have written in my previous reply.

We have run separately on the EVM  the Controller on CPU (1) which use the Timers with the Communication Controller (RTOS included) on  the CPU (0) .

And the signal processing (stage 1 & 2)  CPU  2 & 3  with the Communication Controller (CPU 0) on different EVM.

On the custom board, we have noticed that a lot of messages (from stage 2) have been accumulated  in Stage 3 and according to the TSC of  the CPU 2 300

milisec passed (for example) and the TSC of CPU3 measured 3 millisec.

Regards,

ilan

Hi,

Is there any possibility that using the system_printf to the CCS via the JTAG can cause halt the CPU clock ?

Regards,

Ilan

Hello Ilan,

Question: Is there any status that the CPU was in sleep state (powered down ) ?
Answer: Yes, you can look at the CSR register. Please take a look at the CorePac document.

www.ti.com/lit/ug/sprugh7/sprugh7.pdf

Question: If the CPU starves for memory resource, the TSCL continues to count ?
Answer:  The counter is enabled by writing to TSCL. Counting is disabled in the following cases:
               • After exiting the reset state.
               • When the CPU is fully powered down.

Question: The PDC_INT (power down sleep interrupt), what the process that produces this event ?
Answer:  IPCGRx are the IPC interrupt generation registers to facilitate inter CorePac interrupts. The C6678 has eight IPCGRx registers (IPCGR0 through IPCGR7). These registers can be used by external hosts or CorePacs to generate interrupts to other CorePacs. A write of 1 to IPCG field of IPCGRx register will generate an interrupt pulse to CorePacx (0 <= x <= 7). You can find more description about it in C6678 datasheet.

Question: system_printf to the CCS via the JTAG can cause halt the CPU clock ?

Answer: No, it will not. But system_printf will impact the performance. In the end you will have to disable it in your production version of the code.

========================

Now in order to troubleshoot your issue, I would need some information:
 

1) On high level, can you tell me what function are you trying to achieve? I have to guess from your description on earlier E2E threads.

2) What is the issue? E.g.: my issue is at this condition, what is supposed to work but not working.

3) Is above issue reproduced on TI EVM or only on your custom board? If only on your custom board, can you describe the differences between the two assuming you use our EVM as reference when you designed your custom board. If reproduced on TI EVM, can you let me know the version of the EVM you are using?

4) What software are you using? Are you using the Processor SDK at all? The following is the latest version of SDK for C6678. Or this is totally your test code, are you calling CSL or driver APIs? http://software-dl.ti.com/processor-sdk-rtos/esd/C667x/latest/index_FDS.html

5) When this issue happened, what is the power on/off status of each core? Do you still have above issue if you keep all cores powered on all the time?

6) Are you using any peripherals? Does your application power on/off peripherals? Do you still have above issue if you keep peripherals powered on all the time? Through the GEL file, when you make target connections through CCS, it should power on all the peripherals. You can confirm that through console output.

Thank you!

With regards,

David Zhou  

One correction:

http://www.ti.com/lit/ug/sprugw0c/sprugw0c.pdf

As per section  12.2.6.2: The power-down modes found in legacy devices (set through the control status register (CSR) in the DSP) are not supported in C64x+/C66x CorePac. Please go over chapter 12 if you haven't.

regards,

David

Hi,

Thank you for the reply.

Answers:

1 & 2:

I am trying to identify the reason for the problem if it is software or hardware.

The question is how can different CPUs stop working for dozen of miilisec at different times and then return back to work. 

In one case the CPU controller (bare system) was in wait forever loop (IDLE task) and the Real Time Clock interrupt (every milisec) was not invoked at its period time (previous interrupt and other tasks ended)  and other CPU reported that the slept CPU did not read its messages.

Software option, is by (by trashing  the memory or Registers) sending the CPU to SLEEP or power down, but to wake up process should be done (BOOTING software or setting interrupt and etc.).

Hardware option, problem with the CPU clock or supply, but how it can infect one of the CPUs at time and not all the CPUs together.

3 It is reproduced  on the custom board.

Reference EVM:

CPU controller run at with the same RTC and simulated data is injected from PC.

Frames of data are injected to the Signal processing CPUs. from PC, while in the custom board FPGA injects the data.

4. The RTOS is installed only on the communication controller CPU 0 its version mcsdk_02_01_06

5. I don't think the CPUs power down, because there was not BOOT process and the software continued from the point it stopped.

6. I use SRIO, TIMERS, IPCG and the software does not initiated power down process.

Regards,

Ilan

Ilan,

Thanks for the info but I am still trying to put together the picture, I wish you can provide a block diagram.

Who is going into sleep mode? A particular DSP core, the whole DSP?

Who is waking up the sleep CPU? Another DSP core, another DSP, another CPU or FPGA or itself?

C6678 has 8 cores, do you mean one core will wake up another core? Or do you mean another CPU (whether it is TI DSP or not) will wake up a C6678 core? Or do you mean one core will wake itself up with timer interrupts?

From C6678 perspective, you should be able to put a core in IDLE state and then return it to execution when interrupts hits.

Any hardware option, if it involving redo the schematic and board layout, should be avoided to save the cost.

Typically, I would recommend you keep C6678 core 0 running at all times, you can keep the other cores in IDLE if needed to be, but core 0 can be used to wake up the other cores and core 0 can receive interrupts, whether it is timer or external driven from peripherals. Core 0 can be master in this sense.

best regards,

David Zhou

Ilan,

It's been a few days, do you still need any help here?

With regards,

David Zhou

Hi,

Sorry for my late reply and if i did not clarify the problem clearly.

I use four CPUs and the other four are spares.

These four CPUs should be active all the time from the power on to the power off.

But while the CPUS  run, somtimes there  are occasions that the two of the CPUs (i could not identified the bug in the other two CPUs)  that it seems the CPUs stop working for a period of time (not at the same times) as i described before.

It seems to be like in sleep state but i am not sure for that. 

If i focous on one of the occasions of the CPU 1 (bare system) for example:

The CPU was in loopforever waiting for the Real Time Clock interrupt (it is invoked every milisec) and it was not invoked for 43 milisec and the time elapsed in the TSC was 131 microsec.

Is there any hardware explanation for that.

Regards,

Ilan

Ilan,

Now I understand better what you are saying. If your application never does any power off/on but one of the cores (CPU in your term) stop working, then we can debug it.

Do you do your test with CCS & emulator or without it?

In either way, if you suspect core has been powered down, then you won't be able to connect emulator to it.

If you are able to connect emulator to it, then core has not been powered down. So that can be ruled out.

However, if you are not able to connect emulator when it failed, it doesn't mean core has been powered down as it is not the only reason.

Let's assume you can still connect emulator to the failed core, where does the PC register sit? Which part of the code is that? In your application, are you using TI RTOS kernel or bare metal? If using RTOS kernel, what is the version of it?

On the failed core, how many interrupts does it service? What are those interrupts and priority settings? When you enter timer interrupt service routine, is interrupt disabled? Meaning: does your application allow interrupt pre-emption? Is timer interrupt the only interrupt or could it be suspended by higher priority interrupts?

best regards,

David Zhou

Hi,

I test with CCS & emulator.

I don't think the core has been powered down, because when it resumed back it continued from the point it stopped.

The cores with the problem identification are bare.

I use two interrupts, the RTC with the highest priority (4) and the other one with the lowest priority.

The application allows interrupt pre-emption.

Regards,

Ilan

Ilan,

With your reply, it seem RTC has higher priority already.

Can you still disable all interrupts when you enter RTC and resume it when you exit?

Another possibility is the interrupt service routine, in some cases, takes longer than the allowed timer setting you have. Can you do some profiling on that to confirm? I would assume you have some background while(1) loop, so in your ISR routine, you would need to finish it as soon as you can but allow the background routines to handle the less important and less time critical tasks.

best regards,

David Zhou

Hi ,

Q: Can you still disable all interrupts when you enter RTC and resume it when you exit?

A: It Already done.

I did profiling for the  interrupt.

Now I am trying to identify if its hardware or software problem.

Regards,

Ilan

Ilan,

What is your profiling result? How did you do it?

For your convenience, here is some example code. You can also use TSCH as well (depends on how long timer is):

#include<c6x.h>

unsigned int t_start;
unsigned int t_stop;
unsigned int t_overhead;

TSCL = 0;

t_start = TSCL;

Your CODE to be profiled

t_stop = TSCL;
t_overhead = t_stop - t_start;

In your case:

if(t_overhead >= THRESHOLD)  // where THRESHOLD should be your timer count number/clock(Hz)

{

    while(1); // this means you have captured a corner case

}

I don't believe it is a hardware issue but you can check the silicon errata here:

www.ti.com/.../technicaldocuments

best regards,

David Zhou

Hi David,

The profiling was done similar to your proposal with the TSC and the Timer 8. 

The results were most of the time 1 milisec and some irregular of 43 milisec in TIMER 8 and 135 microsec in TSCL.

Now i try not to activate not all the CPUs together and to isolate the cause of the problem,

Regards,

Ilan

Ilan,

You are on the right track. The next step is to figure out where the irregularity comes from. Is it pending on resources or just taking different path in the code so it was taking longer time to execute.

best regards,

David Zhou

Ilan,

Just to follow up on this thread, are you making some progress? How can we further help you?

best regards,

David Zhou

I solve the same problem with this thread 

Mami,

Please clarify your statement.  Are you saying the you *have* the same problem or are you saying that you *solved* this problem?

Tom

Hi David,

We checked the Supplies and they seem to be OK.

Currently we run only one CPU and it still happens.

We continue in our investigation and i will inform you on our progress.

thank you for your help.

Regards,

Ilan