This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/AM3352: DMTimer2 unexpected stop

Part Number: AM3352
Other Parts Discussed in Thread: TPS65218, TPS65910

Tool/software: Linux

hi TI engineers,

we encounter DMTimer2  unexpected stop in am3352 after run 1 or more days, we indeed seen gp_timer in /proc/interrupts never increase any more;

and this timer is initialized for clockevent in omap2_gp_clockevent_init(clkev_nr, clkev_src, clkev_prop);   //arch/arm/mach-omap2/timer.c

below is the related call stack:

omap3_gptimer_timer_init(void)   =>

__omap_sync32k_timer_init(2, "timer_sys_ck", NULL, \
                                                      1, "timer_sys_ck", "ti,timer-alwon", true);  =>

omap2_gp_clockevent_init(clkev_nr, clkev_src, clkev_prop);

after DMTimer2  unexpected stop, those things happen:

1、gp_timer in /proc/interrupts NEVER increases

2、get time form date cmd may goback some minues or seconds

3、user apps no longer output debug log in console, it seems the scheduler of kernel do not work correctly.

      but shell in console work fine, network ping is also work fine.

4、cpu load of threads in top cmd are all 0%

we have try the newest PSDk's Kernel  and other kernel version 4.4.113/4.4.155 with our own rootfs  in custom board and Beagebone Black board,  the situation is the same. eventhought i don't make any change in kernel source.

someone had posted  the similary probem at

and i had post one at

and others disscus at

here is an other one

By the way, we checked after situation come out,  ST bit of the DMTimer2's TCLR is 1 (that is Start timer)

But If we stop DMTimer2 manually in console shell by cmd:  devmem 0x48040038 32 0x0

then we can reproduced the 1/2/3 situation mentioned above, but hung while i type cmd top in console shell.

So we think DMTimer2 of my AM335x is not work correctly after run one or more days.

We also try to comment out __omap_dm_timer_override_errata() in omap2_gp_clockevent_init(), this force to enable OMAP_TIMER_ERRATA_I103_I767, but the kernel can't bootup at all.

Think you in advance.

  • Hello,

    Could you please share the input signal of your timer?

    [CLKSEL_TIMERx_CLK register has the necessary information]

    Regards,

    Krunal

  • Hi, Krunal

         Thinks for your reply.

        register[CLKSEL_TIMER1MS_CLK] is 0x00000000 , that is CLKSEL is  Select CLK_M_OSC clock

        register[CLKSEL_TIMER2_CLK ] is 0x00000001 , that is CLKSEL is  also Select CLK_M_OSC clock

       The CLK_M_OSC clock is 24MHz

     

    Below is all the  registers of  TIMER1MS dumped from shell:

    TIDR = 0x00000015
    TIOCP_CFG  = 0x00000011
    TISTAT  = 0x00000001
    TISR  = 0x00000002
    TIER  = 0x00000000
    TWER  = 0x00000000
    TCLR  = 0x00000003
    TCRR  = 0x11426DB3
    TLDR  = 0x00000000
    TTGR  = 0xFFFFFFFF
    TWPS  = 0x00000000
    TMAR  = 0x00000000
    TCAR1  = 0x00000000
    TSICR  = 0x00000004
    TCAR2  = 0x00000000
    TPIR  = 0x00000000
    TNIR  = 0x00000000
    TCVR  = 0x00000000
    TOCR  = 0x00000000
    TOWR  = 0x00000000

    and below are the register of  DMTimer2:

    TIDR  = 0x4FFF1301
    TIOCP_CFG  = 0x0000000C
    IRQ_EOI  = 0x00000000
    IRQSTATUS_RAW  = 0x00000000
    IRQSTATUS  = 0x00000000
    IRQSTATUS_SET  = 0x00000002
    IRQSTATUS_CLR  = 0x00000002
    IRQWAKEEN  = 0x00000002
    TCLR  = 0x00000001
    TCRR  = 0xFFFFFFC1
    TLDR  = 0xFFFFA240
    TTGR  = 0xFFFFFFFF
    TWPS  = 0x00000000
    TMAR  = 0x00000000
    TCAR1  = 0x00000000
    TSICR  = 0x00000004
    TCAR2  = 0x00000000

    The register values of TIMER1MS and DMTimer2 when DMTimer2 unexpected stop are all the same as the values of them when work fine, excepted TCRR .

    if you need more other informations, pls let me know.

    Thinks

  • Hello,

    In the following thread, AM335x Silicon errata Advisory 1.0.30 is explained and it is the same behavior that you are observing. If possible, please try setting the input clock source of the DMTimer as the DPLL generated 32.768 KHz clock.

    Regards,

    Krunal

  • Hi, Krunal

          Thank you for your reply.

          You mean change the CLKSEL of DTimer2 (which is clockevent of am335x) from  "timer_sys_ck" (24MHz  SEL1) to 32.768Khz(SEL3)?

          CLKSEL_TIMER2_CLK Register:

        Below is the initialization in kernel/arch/arm/mach-omap2/timer.c

          __omap_sync32k_timer_init(2, "timer_sys_ck", NULL, \
                                                          1, "timer_sys_ck", "ti,timer-alwon", true);

         and in kernel/drivers/clk/ti/clk-33xx.c

         struct ti_dt_clk am33xx_clks[] = {

        ...

        DT_CLK(NULL, "clk_32768_ck", "clk_32768_ck"),
        DT_CLK(NULL, "clk_rc32k_ck", "clk_rc32k_ck"),
        DT_CLK(NULL, "clkdiv32k_ck", "clkdiv32k_ck"),
        DT_CLK(NULL, "clkdiv32k_ick", "clkdiv32k_ick"),
        DT_CLK(NULL, "timer_32k_ck", "clkdiv32k_ick"),

       ...
       }

         which one in am33xx_clks[]  will use as CLK_32KHZ of SEL3 for DMTimer2?

         I tried  __omap_sync32k_timer_init(2, "timer_32k_ck", NULL, \
                                                          1, "timer_sys_ck", "ti,timer-alwon", true);

          and Got below message while kernel bootup.

                "OMAP clockevent source: timer2 at 32786 Hz"

           but when i read back the CLKSEL_TIMER2_CLK,  it return 0x00000001. (why?)

         When we read through the thread you mentioned and its sub-thread, the focus of  their discussion was system time jump problem.

         But The most serious thing in our problem is DMTimer2 clockevent of am335x would unexpected stop, and i think the system time back jump is just one manifestation in my situation.

         Can you send me the newest AM335x Silicon errata datasheet, or related links.

  • Hello,

    The errata is available on the following link: http://www.ti.com/lit/er/sprz360i/sprz360i.pdf

    Regards,

    Krunal

  • Hi, Krunal

         As i mentioned above,  can you give us some suggestion about why DMTimer2 would stop unexpectly (which is used for clockevent of AM3352 ). And  how to re-initialize the DMTimer2  on-the-fly after DMTimer2 unexpected stop

  • Hello,

    Here is a recommendation from our hardware expert:

    Behavior of the internal synchronous circuits is unpredictable if enough noise couples into the crystal circuit to create a glitch on the internal clock. Previous observations where the counter appeared to be counting backwards is one such example of unpredictable behavior. DMTimer2 suddenly stopping could be another example of unpredictable behavior.

    I recommend to temporarily modify a PCB assembly which is known to experience this problem by replacing the crystal circuit with an 1.8V LVCMOS oscillator that sources a reference clock with fast rise/fall edges. This would prove the issue is caused by noise coupling into the crystal circuit if the problem is resolved with this modification.

    Regards,
    Krunal
  • hi, Krunal
    we had replaced the crystal circuit with an 1.8V LVCMOS oscillator in PCB assembly , but no help to the our problem.
    Any other suggestions?
  • Hello,

    I am wondering if it would be possible for you to share pictures of your board modification.

    Regards,
    Krunal
  • Part Number: AM3352

    Tool/software: Linux

    hi TI engineers,

       we had dealed with DMTimer2 unexpected stop problem for nearly two months. Here is the thread i posted before .Recently i find out this problem is related to No initialization with PMIC in my u-boot in custom board.  So in u-boot stage,  the voltage supply to CORE and MPU from PMIC are 1.1V default. But when i configure the PMIC to supply 1.120V to CORE and 1.270V to MPU in am33xx_spl_board_init of u-boot (board.c), the problem disappeared.  

    DMTimer2 unexpected stop problem can reproduce when i delete PMIC change voltage in u-boot in BeagleBone Black.

    I don't know the exact reason.

    Can you explain why?  Thinks!

  • Part Number: AM3352

    Tool/software: Linux

    hi TI engineers,

       we had dealed with DMTimer2 unexpected stop problem for nearly two months. Here is the thread i posted before

    Linux/AM3352: DMTimer2 unexpected stop

    TI E2E Community
    Part Number: AM3352 Tool/software: Linux hi TI engineers, we encounter DMTimer2 unexpected stop in am3352 after run 1 or more days, we indeed seen gp_timer in /proc/interrupts never increase any more; and this timer is initialized for clockevent...

    .Recently i find out this problem is related to No initialization with PMIC in my u-boot in custom board.  So in u-boot stage,  the voltage supply to CORE and MPU from PMIC are 1.1V default. But when i configure the PMIC to supply 1.120V to CORE and 1.270V to MPU in am33xx_spl_board_init of u-boot (code modify see below), the problem disappeared.  

    DMTimer2 unexpected stop problem can reproduce when i delete PMIC change voltage in u-boot in BeagleBone Black.

    I don't know the exact reason.

    Can you explain why?  Thinks!

    u-boot/board\ti\am335x\board.c

    void am33xx_spl_board_init(void)
    {

    ...

        if (board_is_bone(&header) || board_is_bone_lt(&header)) {
            /* BeagleBone PMIC Code */

           ...

    +    } else if (board_is_my-custom-board(&header)) {
    +
    +        puts("Do my-custom-board tps65218 init ...\n");
    +        i2c_set_bus_num(2);
    +        //power_tps65218_init(I2C_PMIC);
    +        if (i2c_probe(TPS65218_CHIP_PM)) {
    +            puts("No tps65218 found!\n");
    +            return;
    +        }
    +        
    +        /*
    +         * Increase USB current limit to 1300mA or 1800mA and set
    +         * the MPU voltage controller as needed.
    +         */
    +        if (dpll_mpu_opp100.m == MPUPLL_M_1000) {
    +            mpu_vdd = TPS65218_DCDC_VOLT_SEL_1320MV; //1325MV
    +        } else {
    +            mpu_vdd = TPS65218_DCDC_VOLT_SEL_1270MV; //1275MV
    +        }
    +
    +        /* Set DCDC1 (CORE) voltage to 1.125V */
    +        puts("Set DCDC1 (CORE) voltage to 1.120V!\n");
    +        if (tps65218_voltage_update(TPS65218_DCDC1,
    +                        TPS65218_DCDC_VOLT_SEL_1120MV)) {  //1125MV
    +            puts("tps65218_voltage_update failure1\n");
    +        }
    +
    +        /* Set CORE Frequencies to OPP100 */
    +        puts("Set CORE Frequencies to OPP100!\n");
    +        do_setup_dpll(&dpll_core_regs, &dpll_core_opp100);
    +
    +        /* Set DCDC2 (MPU) voltage */
    +        printf("Set DCDC2 (MPU) voltage mpu_vdd=0x%x...\n", mpu_vdd);
    +        if (tps65218_voltage_update(TPS65218_DCDC2, mpu_vdd)) {
    +            puts("tps65218_voltage_update failure2\n");
    +        }
    +
    +        /* Set DCDC3 (DDR3) to 1.35V for eg9041. */
    +        puts("Set DCDC3 (DDR3) to 1.35V for my-custom-board.\n");
    +        if (tps65218_voltage_update(TPS65218_DCDC3,
    +                    TPS65218_DCDC3_VOLT_SEL_1350MV)) {
    +            puts("tps65218_voltage_update failure3\n");
    +        }
    +
    +        /*
    +         * Set LDO1 output voltage to 1.8V for my-custom-board.
    +         */
    +         puts("Set LDO1 output voltage to 1.8V for eg9041.\n");
    +         if (tps65218_reg_write(TPS65218_PROT_LEVEL_2,
    +                           TPS65218_LDO1,
    +                           TPS65218_LDO_VOLTAGE_OUT_1_8,
    +                           TPS65218_LDO_MASK))
    +        puts("tps65218_reg_write failure\n");
    +                           
    +         puts("Do eg9041 tps65218 init ... done.\n");
    +         i2c_set_bus_num(0);
    +
        } else {
            /*
             * The GP EVM, IDK and EVM SK use a TPS65910 PMIC.  For all
             * MPU frequencies we support we use a CORE voltage of
             * 1.1375V.  For MPU voltage we need to switch based on
             * the frequency we are running at.
             */

           ...

       }

        /* Set MPU Frequency to what we detected now that voltages are set */
        do_setup_dpll(&dpll_mpu_regs, &dpll_mpu_opp100);
    }

  • Hello,

    With regards to your hardware, I am wondering if you have gone through the following pages:  Schematic Checklist and Layout Checklist. Also, could you please measure the relevant voltage at the ball of the processor to confirm that the board IR drop is not contributing to the failures?

    Regards,

    Krunal