• State
  • Locked Locked
  • Replies 6 replies
  • Subscribers 101 subscribers
  • Views 171 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Original question:

AM6548: U-Boot SPL could not load u-boot proper: QSPI: QSPI is still busy after poll for 10000 times.

AM6548: U-Boot SPL could not load u-boot proper: QSPI: QSPI is still busy after poll for 10000 times.

Andy Lee
Prodigy 10 points
Part Number: AM6548

Dear,

Sometimes our AM6548 based device could not boot from u-boot SPL to u-boot proper:

NOTICE: BL31: v2.6(release):
NOTICE: BL31: Built : 08:03:39, Oct 20 2022
I/TC:
I/TC: OP-TEE version: 3.16.0 (gcc version 10.2.1 20210110 (Debian 10.2.1-6)) #1 Thu Oct 20 08:03:39 UTC 2022 aarch64
I/TC: Primary CPU initializing
I/TC: Primary CPU switching to normal world boot

U-Boot SPL 2022.01-V01.03.01.01-0-gffc3caf (Oct 20 2022 - 08:10:44 +0000)
SYSFW ABI: 3.1 (firmware rev 0x0015 '21.9.1--v2021.09a (Terrific Lla')
Trying to boot from SPI
QSPI: QSPI is still busy after poll for 10000 times.
SPI probe failed.
SPL: failed to boot from all boot devices

ERROR ### Please RESET the board

Any suggestions? Please see github.com/.../440 for details.

  • 0
    Prodigy 10 points

    In my latest reproduced case, it went with the following patch. 

    I filtered the registers' value of abnormal case and the registers'value of normal case as follows:

    ------------------------------------------------------
CQSPI_REG_CONFIG:
 normal registers: 
  Source: cadence_qspi_apb_command_read
   CQSPI_REG_CONFIG:              80083881
  Source: cadence_qspi_apb_read_execute
   CQSPI_REG_CONFIG:              80783881
 abnormal register: 
  Source: cadence_qspi_apb_command_read
   CQSPI_REG_CONFIG:              80083881

------------------------------------------------------
CQSPI_REG_RD_INSTR:
 normal registers: 
  Source: cadence_qspi_apb_command_read
   CQSPI_REG_RD_INSTR:            00000000
  Source: cadence_qspi_apb_read_execute
   CQSPI_REG_RD_INSTR:            0800005a
   CQSPI_REG_RD_INSTR:            00000003
 abnormal register: 
  Source: cadence_qspi_apb_command_read
   CQSPI_REG_RD_INSTR:            00000000

------------------------------------------------------
CQSPI_REG_DELAY:
 normal registers: 
  Source: cadence_qspi_apb_command_read
   CQSPI_REG_DELAY:               05090309
  Source: cadence_qspi_apb_read_execute
   CQSPI_REG_DELAY:               09090909
 abnormal register: 
  Source: cadence_qspi_apb_command_read
   CQSPI_REG_DELAY:               05090309

------------------------------------------------------
CQSPI_REG_INDIRECTRDSTARTADDR:
 normal registers: 
  Source: cadence_qspi_apb_read_execute
   CQSPI_REG_INDIRECTRDSTARTADDR: 00000000
  Source: cadence_qspi_apb_read_execute
   CQSPI_REG_INDIRECTRDSTARTADDR: 00000080
   CQSPI_REG_INDIRECTRDSTARTADDR: 00380000
 abnormal register: 
  Source: cadence_qspi_apb_command_read
   CQSPI_REG_INDIRECTRDSTARTADDR: 00000000

------------------------------------------------------
CQSPI_REG_CMDCTRL:
 normal registers: 
  Source: cadence_qspi_apb_command_read
   CQSPI_REG_CMDCTRL:             9fd00000
  Source: cadence_qspi_apb_read_execute
   CQSPI_REG_CMDCTRL:             00000000
  Source: cadence_qspi_apb_command_write
   CQSPI_REG_CMDCTRL:             06000000
   CQSPI_REG_CMDCTRL:             01008000
  Source: cadence_qspi_apb_command_read
   CQSPI_REG_CMDCTRL:             05800000
 abnormal register: 
  Source: cadence_qspi_apb_command_read
   CQSPI_REG_CMDCTRL:             9fd00000

------------------------------------------------------
CQSPI_REG_CMDREADDATALOWER:
 normal registers: 
  Source: cadence_qspi_apb_read_execute
   CQSPI_REG_CMDREADDATALOWER:    001840ef
  Source: cadence_qspi_apb_command_write
   CQSPI_REG_CMDREADDATALOWER:    00000000
 abnormal register: 
  Source: cadence_qspi_apb_command_read
   CQSPI_REG_CMDREADDATALOWER:    001840ef

    abnormal register: 
      Source: cadence_qspi_apb_command_read
       CQSPI_REG_CONFIG:              80083881

    A bit strange is noted: bit 31 should be 0 in the error case because it is CQSPI_REG_CONFIG_IDLE_LSB, and that is the condition that CQSPI_REG_IS_IDLE is testing for.

    Here is the log of abnormal case:

    Trying to boot from SPI
QSPI: QSPI is still busy after poll for 10000 times.
Source: cadence_qspi_apb_command_read
CQSPI_REG_CONFIG:              80083881
CQSPI_REG_RD_INSTR:            00000000
CQSPI_REG_WR_INSTR:            00000002
CQSPI_REG_DELAY:               05090309
CQSPI_REG_RD_DATA_CAPTURE:     00000005
CQSPI_REG_SIZE:                00101002
CQSPI_REG_SRAMPARTITION:       00000080
CQSPI_REG_INDIRECTTRIGGER:     00000000
CQSPI_REG_REMAP:               00000000
CQSPI_REG_MODE_BIT:            00000200
CQSPI_REG_SDRAMLEVEL:          00000000
CQSPI_REG_WR_COMPLETION_CTRL:  00010005
CQSPI_REG_IRQSTATUS:           00000000
CQSPI_REG_IRQMASK:             00000000
CQSPI_REG_INDIRECTRD:          00000000
CQSPI_REG_INDIRECTRDWATERMARK: 00000000
CQSPI_REG_INDIRECTRDSTARTADDR: 00000000
CQSPI_REG_INDIRECTRDBYTES:     00000000
CQSPI_REG_CMDCTRL:             9fd00000
CQSPI_REG_INDIRECTWR:          00000000
CQSPI_REG_INDIRECTWRWATERMARK: ffffffff
CQSPI_REG_INDIRECTWRSTARTADDR: 00000000
CQSPI_REG_INDIRECTWRBYTES:     00000000
CQSPI_REG_CMDADDRESS:          00000000
CQSPI_REG_CMDREADDATALOWER:    001840ef
CQSPI_REG_CMDREADDATAUPPER:    00000000
CQSPI_REG_CMDWRITEDATALOWER:   00000000
CQSPI_REG_CMDWRITEDATAUPPER:   00000000
CQSPI_REG_OP_EXT_LOWER:        13edfa00
CQSPI_REG_PHY_CONFIG:          00000000

  • 0 in reply to Andy Lee
    TI__Guru 55516 points

    Hi Andy,

    This looks like a duplicate of issue e2e.ti.com/.../am6548-u-boot-spl-could-not-load-u-boot-proper-qspi-qspi-is-still-busy-after-poll-for-10000-times , please confirm. There was already a lengthy discussion on this.

    Also I know you are using your own/upstream sources for U-Boot etc., but can you please create a minimal system based on our just-released AM65x Linux SDK v9.1 (https://www.ti.com/tool/PROCESSOR-SDK-AM65X). It'll be a good reference point for comparison whether the same issue manifests itself or not. Do you have a way to re-create this issue by accelerated cycling/loop testing, or is it still something that only happens after 90(?) days on occasion?

    As for your register values, do you know there is valid QSPI communication happening on the U-Boot SPL A53 stage before the busy timeout? Meaning are there QSPI operations that are successful and return meaningful register values, before the issue happens? Just trying to gauge if in failure case the QSPI is is behaving abnormally right from the beginning of U-Boot SPL A53, or starts doing so only sometime during execution.

    Regards, Andreas

  • 0 in reply to Andreas Dannenberg
    Prodigy 10 points

    Hi Andreas,

    Andreas Dannenberg said:
    This looks like a duplicate of issue e2e.ti.com/.../am6548-u-boot-spl-could-not-load-u-boot-proper-qspi-qspi-is-still-busy-after-poll-for-10000-times , please confirm.

    Yes, it's the same issue. The original issue has been locked, so I created a related question.

    Andreas Dannenberg said:
    Do you have a way to re-create this issue by accelerated cycling/loop testing, or is it still something that only happens after 90(?) days on occasion?

    Unfortunately, no for now. From my experiences, it requires about 50 days to reproduce it.

    Andreas Dannenberg said:
    As for your register values, do you know there is valid QSPI communication happening on the U-Boot SPL A53 stage before the busy timeout? Meaning are there QSPI operations that are successful and return meaningful register values, before the issue happens? Just trying to gauge if in failure case the QSPI is is behaving abnormally right from the beginning of U-Boot SPL A53, or starts doing so only sometime during execution.

    From the log and my current understanding, no valid QSPI communication happened before the busy timeout in the abnormal case.

    However, could you suggest me some debug ways to locate the root cause? Such as a patch to show more information when it is reproduced again.

    And any workaround can be considered? I tried `do_reset()` but it didn't work, the same issues happen on next boot; hard reset works for this problem, but no way to trigger it in software side.

    Thanks.

  • 0 in reply to Andy Lee
    Prodigy 10 points
    Andy Lee said:
    From the log and my current understanding, no valid QSPI communication happened before the busy timeout in the abnormal case.

    Sorry, correct me here, there are some valid QSPI communications happed before the busy timeout, which is in the earlier phase. In the early seboot phase, it reads some contents from the Flash through QSPI communication.

  • 0 in reply to Andy Lee
    TI__Guru 55516 points
    Andy Lee said:
    In the early seboot phase

    What is the "seboot" phase?

    I'm specifically interested if U-Boot SPL A53 can do some valid transactions before the failure. Because each stage of U-Boot re-initializes (probes) the peripheral, so even if U-Boot SPL R5 was able to load U-Boot SPL A53 from QSPI, this by itself doesn't mean QSPI was ever working properly in U-Boot SPL A53 itself, hence my question.

    I'm on business travel right now but I'll try to spend some time next week to re-familiarize myself with the init flow to perhpaps make a more targeted suggestion to debug. In the meantime please consider my other comments.

    Regards, Andreas

  • 0 in reply to Andreas Dannenberg
    Prodigy 10 points
    Andreas Dannenberg said:
    I'm on business travel right now but I'll try to spend some time next week to re-familiarize myself with the init flow to perhpaps make a more targeted suggestion to debug. In the meantime please consider my other comments.

    Thanks for your supporting. I did considered your other comments, anyway AM65x Linux SDK v9.1 is not easy for us to archive. Let's see what I can do about this.

    Andreas Dannenberg said:

    What is the "seboot" phase?

    I'm specifically interested if U-Boot SPL A53 can do some valid transactions before the failure.

    By seboot I meant the R5 part, U-boot SPL A53 didn't do any valid transactions before the failure.

    Thanks.