PROCESSOR-SDK-J784S4: how about the porting flow of R5

yinli liang said:

do you know is there any other host will start R5F, e.g. UBOOT?

Yes. U-Boot will load main domain r5f firmware is the environment variable : dorprocboot is set to 1.

You can check using the below command:

printenv dorprocboot

If the above returns 1 then we are sure that U-Boot is definitely starting the main domain R5F.

Also what is your boot flow? Are you using SPL/U-boot or SBL?

- Keerthy

Are you using SPL/U-boot or SBL? => yes, we use the U-Boot, 

I tried on our platform, the dorprocboot is 0

=> printenv dorprocboot
dorprocboot=0

the detail bootup log is at here:

U-Boot SPL 2021.01-g62a9e51344 (May 02 2023 - 21:12:41 +0000)
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
SYSFW ABI: 3.1 (firmware rev 0x0008 '8.6.3--1-g2249f (Chill Capybara')
SPL initial stack usage: 13472 bytes
Trying to boot from MMC2
Starting ATF on ARM64 core...

NOTICE:  BL31: v2.8(release):v2.8-226-g2fcd408bb3-dirty
NOTICE:  BL31: Built : 21:12:26, May  2 2023
I/TC:
I/TC: OP-TEE version: 3.20.0 (gcc version 9.2.1 20191025 (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)))
I/TC: WARNING: This OP-TEE configuration might be insecure!
I/TC: WARNING: Please check https://optee.readthedocs.io/en/latest/architecture/porting_guidelines.html
I/TC: Primary CPU initializing
I/TC: SYSFW ABI: 3.1 (firmware rev 0x0008 '8.6.3--1-g2249f (Chill Capybara')
I/TC: HUK Initialized
I/TC: Activated SA2UL device
I/TC: Fixing SA2UL firewall owner for GP device
I/TC: Enabled firewalls for SA2UL TRNG device
I/TC: SA2UL TRNG initialized
I/TC: SA2UL Drivers initialized
I/TC: Primary CPU switching to normal world boot

U-Boot SPL 2021.01-g62a9e51344 (May 02 2023 - 21:12:53 +0000)
SYSFW ABI: 3.1 (firmware rev 0x0008 '8.6.3--1-g2249f (Chill Capybara')
Trying to boot from MMC2


U-Boot 2021.01-g62a9e51344 (May 02 2023 - 21:12:53 +0000)

SoC:   J784S4 SR1.0 GP
Model: Texas Instruments J784S4 EVM
Board: J784S4X-EVM rev E3
DRAM:  32 GiB
idle-statesFlash: 0 Bytes
MMC:   mmc@4f80000: 0, mmc@4fb0000: 1
Loading Environment from MMC... OK
In:    serial@2880000
Out:   serial@2880000
Err:   serial@2880000
am65_cpsw_nuss ethernet@46000000: K3 CPSW: nuss_ver: 0x6BA02102 cpsw_ver: 0x6BA82102 ale_ver: 0x00293904 Ports:1 mdio_freq:10000
Unidentified board claims J784S4X-EVM in eeprom header
Net:   eth0: ethernet@46000000port@1
Hit any key to stop autoboot:  0
=> printenv dorprocboot
dorprocboot=0
=>

ask one more question, if we found r5 is still running, how to reset it? 

Hello,

The Linux powers up the R5F and loads the firmware under /lib/firmware. You can for testing sake move the /lib/firmware folder on the rootfs of the target folder to /lib/firmware_bk

Check if still the main domain R5Fs are powered on. Note that MCU domain R5F is mandatory to be on and should run the device manager firmware.

- Keerthy

hello, 

I have tested as your suggestion, move the /lib/firmware to /lib/firmwares_bak. in this case, after Linux start up, there is none any log from R5 serial port. so the R5 is not loaded and started by Linux. 

but when running in our software system (started in uboot), I query the R5 status through the SCI command (messageid =0x201), the current_state in the response message is 1, which means the R5 is already running. but compared with TDA4VM, the current_state is 0 on TDA4VM.

so I am confused why the R5 is running on TDA4VH? who starts the R5? or is there any init flow must be done before configuring the R5?

paste the message at here: 

https://software-dl.ti.com/tisci/esd/latest/2_tisci_msgs/pm/devices.html

Message Data Structures

struct tisci_msg_get_device_req

Request to get device based on id.

struct tisci_msg_get_device_resp

Response to get device request.

Hi,

Couple of questions here.

1. Are you observing that TDA4VM has r5f disabled but TDA4VH has the r5f enabled?
2. MCU domain r5f will need to run the device manager firmware. Just want to double confirm that it's the main domain r5f that you are observing to be on for TDA4VH?

Best Regards,

Keerthy 

Hi, 

1. Are you observing that TDA4VM has r5f disabled but TDA4VH has the r5f enabled？

=> yes, on our software platform, I query the R5 status through the SCI command (messageid =0x201),  the current_state is 0 on TDA4VM, but it's  1 on TDA4VH

2. MCU domain r5f will need to run the device manager firmware. Just want to double confirm that it's the main domain r5f that you are observing to be on for TDA4VH? 

=> yes, it's the main domain, not the mcu domain r5f.

you can see form the picture, when Linux boot, the mcu r5f is already powered on, but the main r5f is not powered on and configured to rempteproc mode.

so my question is , why during the init flow of our software, when first time to query the r5 status by SCI command(0x201), the current_state is 0x1?

is there any other operation needs to be done before querying the r5 status ?

Hi Yinli,

yinli liang said:

you can see form the picture, when Linux boot, the mcu r5f is already powered on, but the main r5f is not powered on and configured to rempteproc mode.

Correct, MCU R5F core will always be booted by R5 SPL (different host id) and will be running the DM Server or SciServer. So, you will always see the following trace for MCU R5F (41000000.r5f) like in your above picture

"R5F core may have been powered on by a different host, programmed state (%d) != actual state (%d)"

The MAIN R5F cores are powered-off and should continue to be in reset by default. If no one has booted then MAIN R5Fs, then they should continue to be in reset.

Are you sure you are using the right device ids for TDA4VH in your query? TDA4VH has more MAIN R5F clusters/cores than the TDA4VM, and the TI-SCI device ids are not the same. Do you see this behavior only on a specific MAIN R5F or all of them?

I suspect that some of the device-ids may not have been updated for TDA4VH during your porting, and may have inadvertently resulted in powering on one of the MAIN R5Fs while powering up some other module. 

regards

Suman

Hi Suman, 

Are you sure you are using the right device ids for TDA4VH in your query?

=> I checked the proc id and dev id, I use the id by parsing the dts nodes, So I think it should be correct. 

 Do you see this behavior only on a specific MAIN R5F or all of them?

=> all MAIN R5F cores have the same behavior, all 6-cores are running. 

Hi Yinli,

This is very strange, and not expected. 

Please read the following PSC MDCTL and MDSTAT registers which correspond to the MAIN R5F cores.

The general formula is as follows:

MDCTL = 0x400000 + 0xA00 + (psc_id * 4)

MDSTAT = 0x400000 + 0x800 + (psc_id * 4)

Following are the PSC ids for the MAIN R5F cores:

MAIN R5FSS0 :  Core0 = 93; Core1 = 94

MAIN R5FSS1 : Core0 = 96; Core1 = 97

MAIN R5FSS2 : Core0 = 122; Core1 = 123

regards

Suman

Hi Suman, 

Please read the following PSC MDCTL and MDSTAT registers which correspond to the MAIN R5F cores.

=> the MAIN R5FSS0 :  Core0 = 93; Core1 = 94, so we can get the address as: 

    the MDCTL of core 0 is  0x400 000  + 0xA00 + 0x174 = 0x400 B74

     the MDSTAT of core 0 is 0x400000 + 0x800 + 0x174 =0x400 974 

the dump result is as below picture shows, I dumped the register in the Uboot, the MDCTL value  is 0x100, the MDSTAT  value is 0xa00.

compared the dump result on TDA4VM: (assume it has the same psc_id on TDA4VM for R5FSS0)

the MDCTL value is 0x000, the MDSTAT value is 0xa00  => compared on the  TDA4VH, the MDCTL of R5FSS0 is 0x100

I cannot find the MDCTL register bits-description, and don't know the meaning of the bit8 of MDCTL .

Could you help to check the difference, and give some comment about the difference. thanks a lot for your kindly support.

Hi Suman, 

The core->ti_sci->ops.dev_ops.is_on() alone is not enough to distinguish IPC-only mode

=> Yes, I know it's now enough, but my problem is, in our software flow, it will enter the mismatched mode...

this picture shows our software flow, the left is the linux reference implementation, and  the right side is our software flow. 

and here is our log:

the c_state=1, ret=0, halted=1, and it enter the mismatched mode...

 is there any other init flow need to be done before querying and configuring  the R5 mode ? 

Hi Yinli,

The c_state variable should be 0 for the register settings you have. Please confirm that you are not bootling the remote processors from U-Boot.

Is it possible for you to read the same registers at your OS console instead of at U-Boot? Please disable your remoteproc drivers for this.

regards

Suman

Hi Suman, 

Please confirm that you are not bootling the remote processors from U-Boot.

=> yes, I attached the full log since board power on /cfs-file/__key/communityserver-discussions-components-files/791/2086.dump_5F00_from_5F00_uboot.txt

Is it possible for you to read the same registers at your OS console instead of at U-Boot? Please disable your remoteproc drivers for this.=> disable remoteproc drivers  and dump regs. /cfs-file/__key/communityserver-discussions-components-files/791/dump_5F00_from_5F00_os.txt

[MAIN R5FSS0 core0]psc_id = 93, MDCTLVirAddr=0xffff800002011b74, MDSTATVirAddr=0xffff800002012974,MDCTL: [0x00400b74]=0x00000100, MDSTAT: [0x00400974]=0x00000a00

I have tried on twice tda4vh boards, they have the same status. 

and also I have a question, the c_state is  from the SCI message, and  is there any register can be read to check the value of c_state.

It seems the uboot log file cannot be inserted, I pasted at here:

U-Boot SPL 2021.01-g62a9e51344 (May 02 2023 - 21:12:41 +0000)
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
ti_i2c_eeprom_am6_get: Ignoring record id 255
SYSFW ABI: 3.1 (firmware rev 0x0008 '8.6.3--1-g2249f (Chill Capybara')
SPL initial stack usage: 13472 bytes
Trying to boot from MMC2
Starting ATF on ARM64 core...

NOTICE: BL31: v2.8(release):v2.8-226-g2fcd408bb3-dirty
NOTICE: BL31: Built : 21:12:26, May 2 2023
I/TC:
I/TC: OP-TEE version: 3.20.0 (gcc version 9.2.1 20191025 (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10))) #1 Tue May 2 21:12:33 UTC 2023 aarch64
I/TC: WARNING: This OP-TEE configuration might be insecure!
I/TC: WARNING: Please check optee.readthedocs.io/.../porting_guidelines.html
I/TC: Primary CPU initializing
I/TC: SYSFW ABI: 3.1 (firmware rev 0x0008 '8.6.3--1-g2249f (Chill Capybara')
I/TC: HUK Initialized
I/TC: Activated SA2UL device
I/TC: Fixing SA2UL firewall owner for GP device
I/TC: Enabled firewalls for SA2UL TRNG device
I/TC: SA2UL TRNG initialized
I/TC: SA2UL Drivers initialized
I/TC: Primary CPU switching to normal world boot

U-Boot SPL 2021.01-g62a9e51344 (May 02 2023 - 21:12:53 +0000)
SYSFW ABI: 3.1 (firmware rev 0x0008 '8.6.3--1-g2249f (Chill Capybara')
Trying to boot from MMC2


U-Boot 2021.01-g62a9e51344 (May 02 2023 - 21:12:53 +0000)

SoC: J784S4 SR1.0 GP
Model: Texas Instruments J784S4 EVM
Board: J784S4X-EVM rev E3
DRAM: 32 GiB
idle-statesFlash: 0 Bytes
MMC: mmc@4f80000: 0, mmc@4fb0000: 1
Loading Environment from MMC... OK
In: serial@2880000
Out: serial@2880000
Err: serial@2880000
am65_cpsw_nuss ethernet@46000000: K3 CPSW: nuss_ver: 0x6BA02102 cpsw_ver: 0x6BA82102 ale_ver: 0x00293904 Ports:1 mdio_freq:1000000
Unidentified board claims J784S4X-EVM in eeprom header
Net: eth0: ethernet@46000000port@1
Hit any key to stop autoboot: 0
=>
=>
=> md 0x00400b74 0x20
00400b74: 00000100 00000100 00000100 00000100 ................
00400b84: 00000100 00000100 00000100 00000100 ................
00400b94: 00000100 00000100 00000100 00000100 ................
00400ba4: 00000100 00000100 00000100 00000100 ................
00400bb4: 00000100 00000100 00000100 00000100 ................
00400bc4: 00000100 00000100 00000100 00000100 ................
00400bd4: 00000100 00000100 00000100 00000103 ................
00400be4: 00000103 00000100 00000100 00000100 ................
=> md 0x00400974 0x20
00400974: 00000a00 00000a00 00000a00 00000a00 ................
00400984: 00000a00 00000a00 00000a00 00000a00 ................
00400994: 00000a00 00000a00 00000a00 00000a00 ................
004009a4: 00000a00 00000a00 00000a00 00000a00 ................
004009b4: 00000a00 00000a00 00000a00 00000a00 ................
004009c4: 00000a00 00000a00 00000a00 00000a00 ................
004009d4: 00000a00 00000a00 00000a00 00011f03 ................
004009e4: 00011f03 00000a00 00000a00 00000a00 ................
=>

Hi Yinli,

yinli liang said:

and also I have a question, the c_state is  from the SCI message, and  is there any register can be read to check the value of c_state.

pSciHandle->ops.pmOps.isDeviceOn (pSciHandle,  (UINT8) pDrvCtrl->tiSciDevId, &r_state, &c_state);

r_state and c_state are just local variables. The Linux implementation only updates these if valid pointers are provided. The K3 remoteproc driver implementation initializes these variables to false (the is_on takes in bool pointers) and sets these based on the TI-SCI status received.

Your OS function implementation is taking in a structure argument, and relying on the structure elements. Are you passing in a new structure pointer for each processor, and/or are you storing some stale values in the is_running field? 

regards

Suman

Hi Suman, 

Are you passing in a new structure pointer for each processor,

=> yes, actually, every core has its specific structure, so none risk 

and/or are you storing some stale values in the is_running field? 

=> no, don't store the values to the field

our local implementation is here:  I think there is none difference compared with Linux implementation.

LOCAL STATUS tisciPmIsDeviceOn
    (
    const TISCI_HANDLE * pHandle,
    UINT32               id,
    BOOL *               pSwOn,
    BOOL *               pHwOn
    )
    {
    UINT8                       programmedState;
    UINT8                       currentState;

    if (tisciPmGetDeviceStatus (pHandle, id, NULL, NULL,
                                &programmedState, &currentState) == ERROR)
        {
        return ERROR;
        }

    if (pSwOn)
        {
        *pSwOn = (programmedState == MSG_DEVICE_SW_STATE_ON) ? TRUE : FALSE;//MSG_DEVICE_SW_STATE_ON = 2
        }
    if (pHwOn)
        {
        *pHwOn = (currentState == MSG_DEVICE_HW_STATE_ON) ? TRUE : FALSE;//MSG_DEVICE_HW_STATE_ON = 1
        }

    return OK;
    }

I provided the registers status on the previous reply, and I want to confirm that, the register status is correct or not?

I means if the PSC MDCTL =0x100 and MDSTAT =0xa00 is correct status when enter our OS

maybe there is something wrong in the SCI communication flow, I should check more about the SCI communication codeflow again

Hi Yinli,

Yes, the h/w registers are in the correct programmed state. So, this is all pointing to some mistranslation in a s/w layer somewhere.

What's the value of your OK macro?

What firmware are you running on MCU1_0 - your own firmware or TI provided firmware?

regards

Suman

Hi Suman, 

What's the value of your OK macro? => it's 1, it is used to mark the sci cmd statuts. the cpu status is saved in *pSwOn and *pHwOn

if (pSwOn)
{
*pSwOn = (programmedState == MSG_DEVICE_SW_STATE_ON) ? TRUE : FALSE;//MSG_DEVICE_SW_STATE_ON = 2
}
if (pHwOn)
{
*pHwOn = (currentState == MSG_DEVICE_HW_STATE_ON) ? TRUE : FALSE;//MSG_DEVICE_HW_STATE_ON = 1
}

What firmware are you running on MCU1_0 - your own firmware or TI provided firmware?

=> you means the the mcu r5 cores (the node defined in the  k3-j784s4-mcu-wakeup.dtsi)

firmware-name = "j784s4-mcu-r5f0_0-fw";=> mcu_r5fss0_core0: r5f@41000000

firmware-name = "j784s4-mcu-r5f0_1-fw"; => mcu_r5fss0_core1: r5f@41400000

we mark it's status to be disabled and don't do any operation for these two cores. is there anything wrong?

&mcu_r5fss0 {
status = "disabled";
};

Hi Yinli,

yinli liang said:

as discussed, the h/w registers are  in the correct programmed state, I will resolve this ticket and check the software flow by myself. 

Thank you, all the h/w registers are in expected state, so this is definitely some mistranslation in the s/w layer somewhere. Note that the Linux s/w does use a ret value of 0 in various TI-SCI calls.

yinli liang said:

it runs the SCI server, so we think we should not update it's firmware in our OS? 

so we just mark the node is disabled, then our OS don't do any operation for this node.

if our behavior has something wrong, please correct me. 

The MCU1_0 is booted at R5 SPL stage itself. The R5 SPL loads and jumps to the MCU1_0 firmware after it launches the A72 cores. The MCU1_0 firmware is built into the tispl.bin binary, and is incorporated into the binary during the A72 U-Boot build steps.

The Linux device-tree firmware names or links are not referenced for MCU1_0 as such. Disabling the nodes, only disables the conventional IPC stack (rpmsg stack), and won't allow userspace applications to communicate with MCU1_0. This does not disable the TI-SCI interface however.

The SDKs use an example IPC firmware image ipc_echo_testb_mcu1_0_release_strip.xer5f as the DM firmware image. This firmware image is built from PDK and is provided under the <ti-processor-sdk-linux-j784s4-evm-08_06_01_02>/board-support/prebuilt-images/ folder.

regards

Suman

Hi Suman, 

thanks again for your kindly explanation and great support. I have no questions now.