This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM57xx ISSUE: When the graphics changes, the Clock has a greater jitter

Expert 1385 points
Other Parts Discussed in Thread: SYSBIOS, AM5718, TMDXIDK57X-LCD, AM5728

HI,

Firstly, let's introduce my trial environment:
AM57xx A15 run Linux, and M4 run TI-RTOS, Linux will
display graphics, and M4 control GPIO pull up/down;
We use the the GPIO output 2ms PWM wave.

I have done some trial for this issue:


1. I use task_sleep in rtos side (M4 core) to sleep 1 ms and then reverse gpio。

When the graphics is not changed, the GPIO wave signal has no jitter.  But when the graphics changed,  it has a greater jitter!  (100us-300us)

2, I use gptimer to make gpio generate the 2ms PWM wave,

and the phenomenon is same , that is , when the graphics is not changed, the GPIO wave signal has no jitter.  But when the graphics changed,  it has a greater jitter!  (800us-900us)

I have done same trial on TI AM57xx GP EVM board, and has same  phenomenon.

It seems that,  when graphics changed, the clock system will have a greater jitter ?

About this CPU issue, please give me some proposal & solution !

Thank you very much!

Best Regards

Qing

  • Hi Qing,

    Can you elaborate on which GPIO are you using? Do you mux it as gpio or as gptimer to generate the pwm signal?

    Can you verify that this gpio is not used anywhere else in your system? Also could you elaborate, if M4 is controlling the gpio, and then you put the cortex in sleep with task_sleep(), do you control the gpio with the A15 (i.e. as wake up, or is my understanding wrong?)

    Best Regards,
    Yordan
  • Hi, Yordan:
    I use the GPIO 6_19 , I am sure that the gpio is not used by others.
    I configure the gpio pin as GPIO mode in linux &rtos sides.

    Best Regards
    Qing
  • 0804.main.c

    And this file is how to control the gpio.

    Thank you very much!

    Best Regards

    Qing

  • Hi Yordan,

    This is Ezen, the same team for this issue. sorry for jumping in since there is no response for a long time. Just want to let you know this is an urgent issue from end-customer and we can show you the real waveform and video directly to you by email too.

    Elaborate everything for all again:

    Customer found Ti5718's GPIO got gitter issue when IPU update screen, such as move the mouse cursor. The GPIO is controlled by M4 core independently, you will find the GPIO is emulated as a simple low speed clock waveform, and video shows the waveform is clear before any movement, but the clock waveform got some gitter symptom that become the fatal issue for our end product.

    The test result shall not related to USB communication because when the mouse cursor stop in the blank white screen area, there is no such issue at the same time. Please kindly to be noticed that only the mouse cursor moving or any screen change got this waveform issue again.

    This issue can be reprodue by EVM board by GPIO & we will post the simple script for you too.

    BR,

    Ezen

    Hi Yordan,
    This is Ezen, the same team for this issue. sorry for jumping in since there is no response for a long time. Just want to let you know this is an urgent issue from end-customer and we can show you the real waveform and video directly to you by email too.

    Elaborate everything for all again:
    Customer found Ti5718's GPIO got gitter issue when IPU update screen, such as move the mouse cursor. The GPIO is controlled by M4 core independently, you will find the GPIO is emulated as a simple low speed clock waveform, and video shows the waveform is clear before any movement, but the clock waveform got some gitter symptom that become the "real fatal" issue for our end product.

    The test result shall not related to USB communication because when the mouse cursor stop in the blank white screen area, there is no such issue at the same time. Please kindly to be noticed that only the mouse cursor moving or any screen change got this waveform issue again.

    This issue can be reprodue by EVM board by GPIO & we will post the simple script for you too.

    BR,
    Ezen

      (Before)

    (after)

    ( more description in EVM baord) :

    apply TI SDK 3.1.0.6  Linux system in A15 Core

     

    Test step:

    1. copy this attatchment script to a USB disk

     

    2. copy this file to AM57xx evm board: path is  /lib/firmware

        reboot the system and measure the GPIO6_19

     

    3. try to move mouse or change screen disply,  monitor the GPIO6_19 waveform.

      

     BR,

    Ezen

  • Hi Yordan,

    Just want to let you know this is an urgent issue from end-customer and we can show you the real waveform and video directly to you by email too.

    Elaborate everything for all again:

    Customer found Ti5718's GPIO got gitter issue when IPU update screen, such as move the mouse cursor. The GPIO is controlled by M4 core independently, you will find the GPIO is emulated as a simple low speed clock waveform, and video shows the waveform is clear before any movement, but the clock waveform got some gitter symptom that become the fatal issue for our end product.

    The test result shall not related to USB communication because when the mouse cursor stop in the blank white screen area, there is no such issue at the same time. Please kindly to be noticed that only the mouse cursor moving or any screen change got this waveform issue again.

    This issue can be reproduce by EVM board by GPIO & we will post the simple script for you too.

    ( more description in EVM board) :

    apply TI SDK 3.1.0.6  Linux system in A15 Core

    Test step:

    1. copy this attachment script to a USB disk。

    2. copy this file to AM57xx evm board: path is  /lib/firmware

       reboot the system and measure the GPIO6_19

    3. try to move mouse or change screen disply,  monitor the GPIO6_19 waveform.

    ============================ M4 firmware /* remove .txt s uffix*/=====================================

    6685.dra7-ipu1-fw.xem4.txt

    ========No display change =======

    =====  display change ========

    7848.110ca53d4e5b228e0108256575e8b53b.mp4

    Best Regards

    Qing

  • Hi all,

    Sorry for the delay.

    I've started an internal discussion about this with the design team. We'll post the feedback directly here.

    Best Regards,
    Yordan
  • Does GPIO toggle has any software dependency once it begins? 

  • Hi, manisha:
    yes, the GPIO is toggled by M4 firmware. we use the task_sleep to toggle the gpio pin.
    We used to toggle the GPIO pin by timer, but it has same jitter on the output wave!

    Best Regards
    Qing
  • Looks like GPIO has no dependency to be toggled using software. If true, then you should avoid software and use PWM to toggle the GPIO. 

  • Qing,

    Was Manisha's suggestion resolved your problem?

    Also, if you have to use SW to toggle GPIO driven LED, can you run the SW in OCMC as IPU has access to it? This can rule out any DDR contention, when you have image change and IPU task to access DDR at the same time.

    You also mentioned that to reproduce problem in EVM:
    apply TI SDK 3.1.0.6 Linux system in A15 Core

    1. copy this attachment script to a USB disk。
    2. copy this file to AM57xx evm board: path is /lib/firmware

    reboot the system and measure the GPIO6_19

    copy this attachment script to a USB disk =====> Where is this script? and what is this script for?
    dra7-ipu1-fw.xem4.txt ====> I believe this is the IPU firmware.

    Regards, Eric
  • Hi, manisha:
    I used timer to output the PWM, but it has bigger jitter on the output wave!
    And we want to know what's the root cause? and do it cause our M4 SW unstable ?

    Best Regards
    Qing
  • Hi, All:
    Thanks for your reply!

    1. Our IPU firmware is so large that there is no more OCMC ram space for it.

    When use CCS to load firmware to OCMC RAM, it's simple. but when use remoteproc subsystem (in linux )to load firmware to OCMC RAM, it's hard for me!

    In reality, we want to know what's the root cause? but not to skip it.
    And does it cause our M4 firmware have a bad real-time behaviour?


    2. this "script" is dra7-ipu1-fw.xem4.txt , you can download the firmware and rename dra7-ipu1-fw.xem4. it's IPU1 firmware.

    Thank you very much!

    Best Regards
    Qing
  • Manisha,

    I jump in also as this is my customer, the IPU executable file's map file as below. 

    MEMORY CONFIGURATION

    name             origin          length        used           unused attr fill
    ---------------------- -------- --------- -------- -------- ---- --------
    L2_ROM         00000000 00004000 000006a8 00003958 RWIX
    EXT_CODE     00004000 000fc000 000280c6  000d3f3a RW X
    L2_RAM          20000000 00010000 00000000 00010000 RWIX
    OCMC_RAM1 40300000 00080000 00000000 00080000 RWIX
    OCMC_RAM2 40400000 00100000 00000000 00100000 RWIX
    OCMC_RAM3 40500000 00100000 00000000 00100000 RWIX
    EXT_DATA      80000000 00200000 0006f6cc 00190934 RW
    EXT_HEAP     80200000 00300000 00000000 00300000 RW
    TRACE_BUF   9f000000 00060000 00008000 00058000 RW
    EXC_DATA      9f060000 00010000 00000000 00010000 RW
    PM_DATA        9f070000 00020000 0001027c 0000fd84 RW X
    SR_0               bfc00000 00100000 00100000 00000000 RW

    From config.bld file, the memory range should be virtual address, how to allocate .text to OCMC1 and IPU 64KRAM?


    /* Memory Map for IPU1
    *
    * --- External Memory ---
    * Virtual Physical Size Comment
    * ------------------------------------------------------------------------
    * 0000_4000 ???0_4000 F_C000 ( ~1 MB) EXT_CODE
    * 8000_0000 ???0_0000 20_0000 ( 2 MB) EXT_DATA
    * 8020_0000 ???0_0000 30_0000 ( 3 MB) EXT_HEAP
    * 9F00_0000 ???0_0000 6_0000 ( 384 kB) TRACE_BUF
    * 9F06_0000 ???6_0000 1_0000 ( 64 kB) EXC_DATA
    * 9F07_0000 ???7_0000 2_0000 ( 128 kB) PM_DATA (Power mgmt)
    */

  • Qing,

    You mentioned that the issue can be reproduced on TI EVM as well. Is that AM572x GP EVM? I looked at your main.c, it uses GPIO6_19 to toggle LOW/HIgh. From the EVM, the LCD module is pressed into AM572x EVM for video display. The GPIO6_19 is on connector P17, pin 28. Physically how do you tap the scope to this pin? Do you solder a wire to P17-pin 28 on EVM? I tried to reproduce the issue to debug, the main problem for me is how to get signal from GPIO6_19. I have no skill to solder a wire there.

    As a debug approach, your main.c (for GPIO testing only) is small, I believe you can just build this M4 application and load into OCMC with a JTAG/CCS. If that can be done, then we can see if the DDR access by Linux and M4 caused the problem. Also, can you give me a simple CCS project with this main.c and resource table definition so I can build the M4 application to be loaded by Linux myself?

    You also mentioned that you used GP timer to replace SYSBIOS sleep task, and you saw the same amount of waveform jitter? Can you confirm? Are you able to share the code with GP timer approach? I knew if you use the software there may be jitters, but it hard to understand with a hardware timer you still saw the issue.

    Regards, Eric
  • Hi, Eric:

       Thank you very much !

        yes, I used the GP EVM ,  and solder a wire to P17-pin 28 on EVM.

        I will  share my Code project for you!

       2783.GPIO.7z

    0601.dmTimer.7z

    About the H/W timer, I use the timer interrupt to toggle the gpio.  Maybe, outputing the timer signal directly is best.

    The CCS project as attachment!

    Thank you very much, again!

    Best Regards

    Qing

  • Qing,

    I tried to build either projects, I changed the makefile to point to my toolset installation path. I was able to compile but failed at link stage:

    <Linking>

    undefined                         first referenced

     symbol                               in file

    ---------                         ----------------

    ti_trace_SysMin_Module_startup__E C:\Data\E2E_Siebel\Video_M4\GPIO\bin\debug\co

    nfiguro\package\cfg\gpio_pem4.oem4

    ti_trace_SysMin_abort__E          C:\Data\E2E_Siebel\Video_M4\GPIO\bin\debug\co

    nfiguro\package\cfg\gpio_pem4.oem4

    ti_trace_SysMin_exit__E           C:\Data\E2E_Siebel\Video_M4\GPIO\bin\debug\co

    nfiguro\package\cfg\gpio_pem4.oem4

    ti_trace_SysMin_putch__E          C:\Data\E2E_Siebel\Video_M4\GPIO\bin\debug\co

    nfiguro\package\cfg\gpio_pem4.oem4

    ti_trace_SysMin_ready__E          C:\Data\E2E_Siebel\Video_M4\GPIO\bin\debug\co

    nfiguro\package\cfg\gpio_pem4.oem4

    error: unresolved symbols remain

    error: errors encountered during linking; "bin/debug/dra7-ipu1-fw.xem4" not

      built

    In the map file:

    9f000000  ti_trace_SysMin_Module_State_0_outbuf__A                                

    9f007ffc  ti_trace_SysMin_Module_State_0_readidx__A                              

    9f007ff8  ti_trace_SysMin_Module_State_0_writeidx__A                              

    UNDEFED   ti_trace_SysMin_Module_startup__E                                      

    UNDEFED   ti_trace_SysMin_abort__E                                                

    UNDEFED   ti_trace_SysMin_exit__E                                                

    UNDEFED   ti_trace_SysMin_putch__E                                                

    UNDEFED   ti_trace_SysMin_ready__E        

    Did I miss any steps to build this? Attached the build log and modified makefile.

    0876.Makefile3312.buildlog.txt

    Regards, Eric

  • Hi, Eric:
    Please check your ipc package. the trace buffer is dependent on ipc package (it used some based units of ipc package, not ipc).
    I guess that your ipc package is missing.
    Do you download the Ti RTOS SDK 3.1.0.6 ? I used this version!


    Best Regards
    Qing
  • Qing,

    I was able to compile and link after switch to a Linux host machine. I am working with my colleague to solder a wire to GPIO6-19.

    Regards, Eric
  • Qing,

    Sorry for the late, I was out of the office last week. I was not able to reproduce the issue. Attached a few pictures showing the setup.

    When I touched the LCD to change what displayed, I didn't see any jitter of GPIO. Let me try to see if I can attach a video.

    This is the Linux OS I used:

    root@am57xx-evm:/lib/firmware/ipc/ti_platforms_evmDRA7XX_ipu1# uname -a
    Linux am57xx-evm 4.4.32-rt41-ge26c84b0ac #1 SMP PREEMPT RT Wed Dec 14 19:34:32 EST 2016 armv7l GNU/Linux

    HW: TI AM572x GP EVM.

    IPU firmware: root@am57xx-evm:/lib/firmware/ipc/ti_platforms_evmDRA7XX_ipu1# ls -lta
    -rw-r--r--    1 root     root       2451256 Mar 29  2017 test_omx_ipu1_vayu.xem4 =====> this is the one you uploaded into the E2E.

    Regards, Eric

     

  • 5432.20170419_170858.mp4

    I also uploaded a video (MP4) clip showing my operation.

    Regards, Eric

  • Hi, Eric:

    Thank you very much !

    Which SDK version did you use ? I am on using SDK3.1.0.6.
  • Hi,

    I tried 3.1.0.6 also, no such GPIO jitter issue:

    root@am57xx-evm:~# uname -a
    Linux am57xx-evm 4.4.19-gdb0b54cdad #1 SMP PREEMPT Mon Oct 3 18:04:11 EDT 2016 armv7l GNU/Linux

    Is this the same Linux version as you tested?

    Regards, Eric
  • Hi, Eric:

    Thanks for your patiently reply!

    Now ,we are talking about jitter amplitude, From the picture you have provided, we think your test method can't reproduce this jitter , This method is used to measure the cycle, not for Jitter.

    www.diodes.com/.../AB036.pdf

    Thank you very much!



    Best Regards

    Qing
  • Qing,

    When the GPIO clock is stable without jitter, there is solid line for the rising and falling edges of GPIO toggle, as the scope constantly scan the input signal to much faster speed. If for some reasons the GPIO exhibits the jitter (e.g., heavy loaded system, the GPIO toggling time can't be guaranted), you will see some shifting along the waveform, as in your picture below:

    With this kind of scope + GPIO setup, I was able to capture the GPIO jitter for other issues (see below picture, this high speed interface with jitter in ~50 ns) in the past, as long as the operation introduce the GPIO jitter. So I think the test setup is valid.

    Then how you reproduce the issue on TI AM572x GP EVM? From your previous description, you saw the GPIO jitter when you touch the LCD screen to change graphics. Why I didn't see the issue?

    Regards, Eric

  • Hi, Eric:
    finally, we found that there are no jitter on am57xx evm board in fact.
    and we has had a wrong test method on getting the gpio jitter.

    But, we found this jitter are really on AM571x IPU sides, and we ask our local Ti FAE to test
    this on AM5718 IDK board, and it has a big jitter on GPIO6_19.
    there is no any gpio jitter on AM572x IPU.

    The test program are same as the firmware i have given you.


    Thank you very much!
    Best Regards
    Qing
  • Qing,

    Can you confirm the test is done on GPIO6_19 on AM571x IDK EVM? I looked at schematic: http://www.ti.com/lit/df/sprr242/sprr242.pdf and didn't find GPIO6_19. Where did you put a wire for GPIO waveform monitoring?

    Regards, Eric

  • Hi Eric,

    I've confirmed the test on GPIO6_19 on AM571x IDK EVM.

    Afer log in, the jitter is equal 16uS

    When execute "runOGLES2ChameleonMan.sh", the Jitter is equal 136uS.

    Hardware: AM571x Industrial Development Kit (IDK) Rev 1.3A
    Software: Linux SDK v03.01.00.06

  • <Supplementary explanation>
    AM5718 IDK must connects to TMDXIDK57X-LCD

    AM57x IDK LCD Kit
    TMDXIDK57X-LCD
    www.ti.com/.../tmdxidk57x-lcd
  • Hi,

    Thanks! I need to find a LCD for this test. Your GPIO signal is measured at the point where R241 and R245 connected, right?

    Regards, Eric
  • Hi Eric,

    Yes.

  • Hi,

    I have the board modified for GPIO6_19. Using your SW GPIO toggle code, the jitter is about 80us (less than 136us in your picture) when LCD display changed. For your dmTimer code, seems the timer only runs 2-3 seconds then stopped (tracked timerISR time stamp logged in trace buffer), I was not able to do the LCD image change test. The timer mode is Timer_RunMode_CONTINUOUS, do you have the same issue? I don't understand how the timer stopped?

    Regards, Eric
  • Qing and WTMEC_Will Hsu:

    Can you clarify the dmTimer test example?

    Regards, Eric
  • Dear Eric:
    Thank you!
    80us is the outside of our tolerance boundary. we need the same little jitter like AM572x CPU.

    This jitter has casued many problems on us and our customer, such as GPIO jitter, ISR delay, and so on.

    Let's forget the dmtimer's code(it seems for am728). and the GPIO code is our focus.

    And by the test, the jitter is really existed on am571x cpu, and am572x cpu has no such big jitter.


    And the other information, we can share you:
    we used the same test method on DSP side. and no jitter on DSP.

    Thank you very much, Eric!

    Best Regards
    Qing
  • Qing,

    Thanks for the direction! Then we will focus on GPIO code only. An important info is that issue not on DSP core, can you share the project for DSP?

    Regards, Eric
  • Dear Eric:

    the attachment:1261.GPIO_Dsp.zip

    Best Regards

    Qing

     

  • Qing,

    Thanks for sharing the project. I was not able to see the GPIO jitter when using DSP. So the jitter is only seen on AM571x M4 core. I did some clean up to remove the code for GPIO2 and GPIO6_20, and tried different Linux kernel releases. As LCD display changes refreshing the DDR memory, I also ran some Linux DDR test applications (instead of LCD display change) to see if GPIO jitter there or not. I am still actively debug this.

    Regards, Eric

  • Dear Eric:
    Yes, Thank you very much!

    Best Regards
    Qing
  • hi, Eric

    We have a discusstion for this high priority issue again since there is no feedback for a long time. Meanwhile, that is a very fatal issue for our customre's application again.

    It seems like AM5718 got this symptom but AM5728 has no this issue, and AM5728 RPU works well on SPI interface also. Base on these information, we think when system is runtime sharing DDR's slave core, the RTOS real time ability is deeply effected by A15+Linux. And we are also thinking about whether power saving design on AM5718 """might""" be the root case (or not)?

    Is it a right directly, how's your idea!? Thanks for your comments in advance.

      

    BR,

    Ezen

  • Ezen,

    I really felt sorry for the late response to this issue. I am still trying to isolate if the sharing DDR by A15 Linux and M4 causing the issue. My intents is either use CCS/JTAG to load M4 or use Linux remoteproc to load M4, but moving M4 code into OCMC. I had trouble in the approaches and am working on that.

    Regards, Eric
  • 7026.gpio.cfg6330.test_omx_ipu1_vayu.xem4

    Hi,

    I was able to move the M4 GPIO code to OCMC, I still saw the GPIO jitter, so this is not a DDR contention between A15 and M4 issue.

    Followed some discussion, I found if I put M4 core into deep sleep mode during BIOS idle task can suppress this GPIO jitter to around 20-30 us, on my AM571X IDK EVM setup. What I did is change the configuration file for M4 project, see attached.

    The IpcPower_idle() function is implemented in ipc_3_xx_xx_xx\packages\ti\pm\ipcpower.c via library. Can you update the .cfg file and try if it works on your EVM setup?

    Then to your real application, does it use SYSBIOS? and there are multiple tasks like GPIO, SPI, ISR, etc ...  there should be idle task like Task_sleep() in your GPIO toggle example to me? Do you feel put M4 into deep sleep is a problem? From TRM, we need to periodly flush the M4 unicache, this is I am trying now. I would like to know if putting M4 in deep sleep is a right direction on your real application/real board? I also attached the M4 firmware I used for test where you put under /lib/firmware

    Regards, Eric

  • Dear Eric:
    It's great for this founding, I will try it on my board. and reply you later!
    Thank you very much, Thank you!

    Best Regards
    Qing
  • Qing,

    Do you have a chance to try? Thanks!

    Regards, Eric
  • Qing,

    Also, in the gpio.cfg, I have following to put both IPU0 and IPU1 into deep sleep:

    Idle.addCoreFunc('&myIpcPower_idle', 0);
    Idle.addCoreFunc('&myIpcPower_idle', 1);

    I found out just put IPC 1 into deepsleep is sufficient to suppress the GPIO jitter on IPU0. Still we have more to understand the interaction between IPU0 and IPU1, and more experiment in our side. But I want to know if you: 1) can duplicate my results on TI EVM? 2) do you use IPU1? Any code running there? 3) put IPU1 on deepsleep by Idle.addCoreFunc('&myIpcPower_idle', 1) help to suppress the GPIO jitter on IPU0 in your real application on your board?

    Regards, Eric
  • Dear Eric:
    yes, we have done this trial. and we found just 20us GPIO jitter follow your proposal.
    It seems to be okay! And we let our customer to do it at now。 And after get the response,
    we will mark this thread as answered!
    Thank you very much! thank you !

    Best Regards
    Qing
  • Qing,

    Thanks for the trial! That is encouraging news! We still need to know what IPU1 runs on customer side? With IPU1 in deep sleep, will customer real application on IPU0 has much better controlled jitter, like GPIO and others (SPI)?

    Regards, Eric
  • Qing,

    I confused IPUs and cores in the previous update, sorry for this! The correct one would be:

    AM5718 has two IPUs (IPU1 and IPU2) and each IPU has two cores (core 0 and core 1). GPIO code runs on IPU1, some other code (maybe IVAHD) runs on IPU2 by default Linux system.

    For the added SYSBIOS configuration script:
    //Idle.addCoreFunc('&IpcPower_idle', 0);
    Idle.addCoreFunc('&IpcPower_idle', 1);

    This is to put core 1 of IPU1 into deep sleep, the GPIO test code only has one GPIO toggling task, it runs on core 0 of IPU1. It is not related to the IPU2. So what really customer using IPU2 is not relevant.

    Regards, Eric
  • Dear Eric:
    Thank you very much, I can understand it. And our customer tested and found that the gpio jitter disappeared. But they found that once ipu into the deep sleep state, IPU is hard
    to be waked up by other deivce( etc, spi, i2c interrupt). and IPU just only is waked up by timer , IPC, and so on. How can we do to avoid the state that IPU can't be waked up?

    Thank you very much, again!
    Best Regards
    Qing
  • Qing,

    Thanks for letting me know the results! In IpcPower_idle(), it calls into IPC library code to: 1) put IPU core into deep sleep 2) excute asm (" wfi") .

    I tested another case to implementing my own idle function, like this (just waiting for interrupt, without putting into deep sleep):

    A. In configuration: Idle.addCoreFunc('&myIpcPower_idle', 1);

    B. In main.c I implement this function:

    void myIpcPower_idle(void) {
     asm(" wfi");
    }

    I saw the jitter is still gone in my test setup. Can customer try this to see if it works to be waked up by other devices?

    Regards, Eric

  • Dear Eric:
    Thank you very much!
    Best Regards
    Qing
  • Qing,

    Let me know if customer can try this with their real application, and if results available? Thanks!

    Regards, Eric