This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM5728: Issues with WDT enabled in U-boot

Part Number: AM5728


Hello Everyone,

I have a question related to the hardware watchdog timer on AM5728 enabled in the U-Boot stage.  In order to support the HW WDT in U-Boot, I made changes in U-Boot configuration and added a call to hw_watchdog_init() in board specific code in board/ti/am57xx/board.c (more on this below).  My current changes look like this:

/configs/am57xx_evm_defconfig b/configs/am57xx_evm_defconfig:

CONFIG_HW_WATCHDOG=y
CONFIG_SPL_WATCHDOG_SUPPORT=y
CONFIG_OMAP_WATCHDOG=y

board/ti/am57xx/board.c

int board_init(void)
{
#if defined(CONFIG_HW_WATCHDOG)
    printf("******** board_init(), calling hw_watchdog_init() ********\n");
    printf("**********************************************************\n");
    hw_watchdog_init();

#endif

    gpmc_init();
    gd->bd->bi_boot_params = (CONFIG_SYS_SDRAM_BASE + 0x100);

    //return 0;
    return 1;
}

Code shown above does indeed result in a board reboot at the predefined interval WDT_HW_TIMEOUT which is 60 seconds (drivers/watchdog/omap_wdt.c).  I also added temporary debugging code in drivers/watchdog/omap_watchdog.c.  Once I reboot the board, the following console output sequence is displayed:

U-Boot SPL 2018.01-g131dc82830 (Nov 04 2019 - 00:56:49)
DRA752-GP ES2.0
******** hw_watchdog_init() called! ********
************************************************
******** hw_watchdog_disable() called! ********
************************************************
******** omap_wdt_set_timeout() called! ********
************************************************
Trying to boot from MMC2_2
no pinctrl state for default mode
palmas_i2c_write_u8: chip=58 reg=51 val=43
palmas_i2c_write_u8: chip=58 reg=50 val=5
no pinctrl state for default mode

...

reading u-boot.img
reading u-boot.img
reading u-boot.img
reading u-boot.img


U-Boot 2018.01-g131dc82830 (Nov 04 2019 - 00:56:49 +0000)

CPU : DRA752-GP ES2.0
Model: TI AM5728 IDK
Board: Humatics ALB REV
DRAM: 2 GiB
******** board_init(), calling hw_watchdog_init() ********
**********************************************************
******** hw_watchdog_init() called! ********
************************************************
******** hw_watchdog_disable() called! ********
************************************************
******** omap_wdt_set_timeout() called! ********
************************************************
initcall sequence fefb1734 failed at call 80803c1d (err=1)

### ERROR ### Please RESET the board ###

>>>>>>>>>>>>>>>>>>>>>>>>>>> THE BOARD REBOOTS HERE AFTER 60 SECONDS. <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Comments on the above code.

The reason I return a "1" from board_init() above is to test the code I added.

When the extra call to hw_watchdog_init() (added by me) is not present in board_init(), regardless of whether a I return a 1 or a 0 from board_init(), the first set of calls to hw_watchdog_*() functions always appears.  I am not sure where they are coming form.  I also don't understand why, even though the HW WDT timer is not enabled, a board hangs and never restarts again.

The reason I add an explicit call to hw_watchdog_init() in board_init() is because without it, the board hangs after boot and never restarts.  That is, hw_watchdog_reset() is never called even though a call to hw_watchdog_init() is called from somewhere else and from my code.  This is despite the fact that HW WDT is now enabled in U-Boot configuration as mentioned above.  It seems that adding this call is necessary.

Finally, with code shown above and board_init() augmented to return 0, after Linux boots all the way on the board, the following message is printed indefinitely on the console:

[ 211.402546] watchdog: watchdog0: watchdog did not stop!
[ 212.477426] watchdog: watchdog0: watchdog did not stop!
[ 213.551362] watchdog: watchdog0: watchdog did not stop!
[ 214.625176] watchdog: watchdog0: watchdog did not stop!
[ 215.699052] watchdog: watchdog0: watchdog did not stop!

If someone could shed some light on my observations, I would appreciate it.

 

Regards,

Paul 

  • Hi Paul,

    Paul Burkacki said:
    When the extra call to hw_watchdog_init() (added by me) is not present in board_init(), regardless of whether a I return a 1 or a 0 from board_init(), the first set of calls to hw_watchdog_*() functions always appears.  I am not sure where they are coming form.

    The first invocation of hw_watchdog_init() happens in the context of SPL, and comes from here:

    arch/arm/mach-omap2/boot-common.c:
    
    void spl_board_init(void)
    {
    #ifdef CONFIG_SPL_SERIAL_SUPPORT
            /* Prepare console output */
            preloader_console_init();
    #endif
    #if defined(CONFIG_SPL_NAND_SUPPORT) || defined(CONFIG_SPL_ONENAND_SUPPORT)
            gpmc_init();
    #endif
    #if defined(CONFIG_SPL_I2C_SUPPORT) && !defined(CONFIG_DM_I2C)
            i2c_init(CONFIG_SYS_OMAP24_I2C_SPEED, CONFIG_SYS_OMAP24_I2C_SLAVE);
    #endif
    #if defined(CONFIG_AM33XX) && defined(CONFIG_SPL_MUSB_NEW_SUPPORT)
            arch_misc_init();
    #endif
    #if defined(CONFIG_HW_WATCHDOG)
            hw_watchdog_init();
    #endif
    #ifdef CONFIG_AM33XX
            am33xx_spl_board_init();
    #endif
    #if defined(CONFIG_SPL_BUILD) && defined(CONFIG_REMOTEPROC_TI_IPU)
            spl_boot_ipu();
    #endif
    }
    

    Paul Burkacki said:
    I also don't understand why, even though the HW WDT timer is not enabled, a board hangs and never restarts again.

    Can you please describe in more detail what you mean by "HW WDT timer not enabled".

    Paul Burkacki said:
    The reason I add an explicit call to hw_watchdog_init() in board_init() is because without it, the board hangs after boot and never restarts. 

    You say the board boots and then hangs... how far does it boot?

    Let me see if I can get a hold of an AM57xx EVM tomorrow and give this a try.

    Regards, Andreas

  • Hi Andreas,

    thank you for your reply and thank you for pointing out the location of the first call to hw_watchog_init().  That makes more sense now.

    In regard to my comment of "HW WDT timer not enabled", I made a mistake when I wrote it.  Above, I wrote

    Paul Burkacki: I also don't understand why, even though the HW WDT timer is not enabled, a board hangs and never restarts again.

    What I meant to say was:

    Paul Burkacki: I also don't understand why, even though the HW WDT timer _is_ enabled, a board hangs and never restarts again.

    So the mixup in my question was caused by my lack of understanding why the first call to hw_watchdog_init(), the one made in spl_board_init() as you pointed out, does not force a reboot of the board when board_init() is forced to return a 1.  I thought that this first call enables the WDT such that, when the predefined 60 second mark is reached, the board should be rebooted.  In effect, that first call should not require that a second call is added to AM57xx specific board_init().  As I mentioned in my original post, it seems that the extra call to hw_watchdog_init(), which I add in AM57xx specific board_init(), is required in order for the 60 second timeout to cause a reboot.

    In regard to my comment of "the board hangs after boot and never restarts".  The following is the console output I see when the call to hw_watchdog_init() is made only from spl_board_init() and I return a 1 from the AM57xx specific board_init() without making an explicit call to hw_watchdog_init().  In this case, the 60 second HW WDT timeout does not seem to force a board reboot.  Here is the console output which I observed:

    U-Boot SPL 2018.01-g131dc82830 (Nov 04 2019 - 00:56:49)
    DRA752-GP ES2.0
    ******** hw_watchdog_init() called! ************
    ************************************************
    ******** hw_watchdog_disable() called! *********
    ************************************************
    ******** omap_wdt_set_timeout() called! ********
    ************************************************
    Trying to boot from MMC2_2
    no pinctrl state for default mode
    palmas_i2c_write_u8: chip=58 reg=51 val=43
    palmas_i2c_write_u8: chip=58 reg=50 val=5
    no pinctrl state for default mode
    *** Warning - bad CRC, using default environment

    reading u-boot.img
    reading u-boot.img
    reading u-boot.img
    reading u-boot.img


    U-Boot 2018.01-g131dc82830 (Nov 04 2019 - 00:56:49 +0000)

    CPU : DRA752-GP ES2.0
    Model: TI AM5728 IDK
    Board: Humatics ALB REV
    DRAM: 2 GiB
    ******** board_init(), not calling hw_watchdog_init() ********
    **************************************************************
    initcall sequence fefb1734 failed at call 80803c1d (err=1)
    ### ERROR ### Please RESET the board ###

    Now I understand where the first set of calls to hw_watchdog_*() functions comes from.  But if I understand them correctly, they should be enough to enable HW WDT such that a reboot takes place at the 60 second mark.  Instead, the board hangs here for ever.

    One more thing I should mention, the hw_watchdog_reset() function is called repeatedly in very fast succession during boot.  This would explain why the board reboot does not occur.  However, I don't understand why this function is called so rapidly.

    Thank you again for your help.

    Regards,

    Paul

  • Hi Paul,

    Paul Burkacki said:
    Now I understand where the first set of calls to hw_watchdog_*() functions comes from.  But if I understand them correctly, they should be enough to enable HW WDT such that a reboot takes place at the 60 second mark.  Instead, the board hangs here for ever.

    I dug into this a bit deeper, and the reason that the watchdog when only turned on in SPL and not explicitly turned on again in U-Boot proper doesn't stay active lies in arch/arm/mach-omap2/hwinit-common.c, which disables the watchdog via board_init_f() -> early_system_init() -> watchdog_init(). Actually I don't think this is a good idea, as it in fact interrupts the chain the watchdog is active, plus it accesses the watchdog module outside its designated driver. So rather than explicitly turning on the watchdog again in U-Boot's board_init() I'd probably remote the call to watchdog_init() from early_system_init(). However for all practical purposes either method will work probably fine.

    For reference, here are the changes I made based on top of TI's linux-am57xx-evm-06.01.00.08 SDK to enable the watchdog in both SPL and U-Boot for AM57xx, and this is modeled after how it's done for AM335x:

    $ git show
    commit 114b2002ce0ce7525cb6f5aa5df268fd870421ae (HEAD -> processor-sdk-local)
    Author: Andreas Dannenberg <dannenberg@ti.com>
    Date:   Wed Nov 6 14:14:04 2019 -0600
    
        am57xx_evm: Enable on-chip watchdog timer in both SPL and U-Boot
        
        Signed-off-by: Andreas Dannenberg <dannenberg@ti.com>
    
    diff --git a/board/ti/am57xx/board.c b/board/ti/am57xx/board.c
    index 302b69ecf7..32581c6b9a 100644
    --- a/board/ti/am57xx/board.c
    +++ b/board/ti/am57xx/board.c
    @@ -31,6 +31,7 @@
     #include <ti-usb-phy-uboot.h>
     #include <mmc.h>
     #include <dm/uclass.h>
    +#include <watchdog.h>
     
     #include "../common/board_detect.h"
     #include "mux_data.h"
    @@ -614,6 +615,10 @@ bool am571x_idk_needs_lcd(void)
     
     int board_init(void)
     {
    +#if defined(CONFIG_HW_WATCHDOG)
    +       hw_watchdog_init();
    +#endif
    +
            gpmc_init();
            gd->bd->bi_boot_params = (CONFIG_SYS_SDRAM_BASE + 0x100);
     
    diff --git a/configs/am57xx_evm_defconfig b/configs/am57xx_evm_defconfig
    index 0bb935ac92..551f0aebe1 100644
    --- a/configs/am57xx_evm_defconfig
    +++ b/configs/am57xx_evm_defconfig
    @@ -29,6 +29,7 @@ CONFIG_SPL_DMA=y
     # CONFIG_SPL_NAND_SUPPORT is not set
     CONFIG_SPL_OS_BOOT=y
     CONFIG_SPL_SPI_LOAD=y
    +CONFIG_SPL_WATCHDOG_SUPPORT=y
     CONFIG_SPL_YMODEM_SUPPORT=y
     CONFIG_CMD_SPL=y
     # CONFIG_CMD_FLASH is not set
    @@ -94,5 +95,6 @@ CONFIG_USB_GADGET=y
     CONFIG_USB_GADGET_MANUFACTURER="Texas Instruments"
     CONFIG_USB_GADGET_VENDOR_NUM=0x0451
     CONFIG_USB_GADGET_PRODUCT_NUM=0xd022
    +CONFIG_OMAP_WATCHDOG=y
     CONFIG_CLK=y
     CONFIG_CLK_CDCE9XX=y

    I've tested that the watchdog indeed works in both SPL and U-Boot, both with the above patch, as well as the alternate method of removing the call to watchdog_init() rather than adding hw_watchdog_init() to board_init(). Also, in all cases my Kernel log looked clean, free of any watchdog-related errors.

    Paul Burkacki said:
    One more thing I should mention, the hw_watchdog_reset() function is called repeatedly in very fast succession during boot.  This would explain why the board reboot does not occur.  However, I don't understand why this function is called so rapidly.

    Yes this is to be expected. Since a watchdog timer reset should be embedded into the execution path of functional code (not triggered from another timer directly, for example), the intervals in which it is reset are by nature very irregular and typically much smaller than the watchdog timer timeout, especially for simple non-RTOS / non-OS code like U-Boot.

    Hope that helps you to get over the hump here.

    Thanks and Regards,
    Andreas

  • Hi Andreas,

    thanks for this great insight.  I was not aware that WDT was disabled during the chain of execution.  I agree with you that interrupting it is not something that is desired.  For the time being, I will stick with the changes I made so far.  However, I will keep the call to watchdog_init() in mind.  By the way, funny you should mention AM335x.  I modeled the changes I made in board_init() after exactly that code.

    Also, I found that extra console messages coming out of the kernel were caused by other changes in the source base.  So the changes I made are not the culprit.  It matches your observations which makes me feel much better about the WDT changes made in board_init(). 

    Out of curiosity, how did you test your changes?  Was it also by introducing an artificial failure in board_init() or was it by another method?

    Yes, I am over the hump now.  Thanks again for your help.

    Regards,

    Paul

  • Part Number: AM5728

    Hi Andreas,

    when I clicked the "This answer resolved my question" button, the thread was locked.  I wasn't sure if you would be able to reply to it.  That is the only reason I am once more posting that last question here about the test method you used to confirm that your WDT changes worked.  I apologize if this turns out to be spam.  I was just making sure that it was still possible for you to reply to that one question.

    Again, your previous post did resolve my question.  The one about the test method was the only one left.  Thanks again.

    Regards,

    Paul

  • Hi Paul,

    Paul Burkacki said:
    The one about the test method was the only one left. 

    I tested this in U-Boot, just like you did, by trapping the code execution in several places throughout using an endless loop that I moved around and it triggered a device RESET in all places as expected. I also tried to test it in Linux but wasn't immediately successful killing the process feeding it (also have not tried very hard). Here are some additional suggestions I just found to try out for this particular aspect if that's what you are after:

    https://unix.stackexchange.com/questions/21335/how-do-i-cause-a-watchdog-reset-of-my-embedded-linux-device

    Regards, Andreas

  • Hi Andreas,

    thank you for the detailed description.  Also, thank you for the link.  I will definitely check it out.

    Regards,

    Paul