This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM623: AM623+mcu+sdk

Part Number: AM623

Run the ipc_rpmsg_echo_linux example in the sdk on the m4 kernel of am623, and find that the program is stuck in the RPMessage_waitForLinuxReady function. In this case, what should the linux side do to make this function pass normally.  (sdk:mcu_plus_sdk_am62x_08_04_00_16)

  • Hello,

    How are you running the example?

    Please start by following the steps documented here:
    https://dev.ti.com/tirex/explore/node?node=A__AISILbMWZ4d84U2oSmXcdA__linux_academy_am62x__XaWts8R__LATEST (booting the correct firmware to the M4)
    https://dev.ti.com/tirex/explore/node?node=A__ACKtJpWz8b53RwT-PJ4qmA__linux_academy_am62x__XaWts8R__LATEST (steps to run the IPC example)

    Regards,

    Nick

  • After the development board is powered on, the m4 kernel is connected through the JTAG of the development board, and the computer is running the m4 kernel example through ccs. During debugging, the program blocks the RPMessage_waitForLinuxReady function.

    Do you mean that I can only send the m4 execution firmware through a53 in the way you said to run it? Does it mean that the ipc example cannot be debugged on the m4 kernel through ccs

  • Hello,

    I am not clear from your description. Is Linux running or not when the development board is powered on?

    This process should work for your debug:

    1) Boot Linux as per https://software-dl.ti.com/mcu-plus-sdk/esd/AM62X/08_04_00_16/exports/docs/api_guide_am62x/CCS_LAUNCH_PAGE.html , section "Load and run example binaries" > "if initialization is done by following steps in SOC Initialization Using SPL"

    2) Connect to the M4F CPU. However, do NOT reset the CPU. Instead, Load Symbols and point to the source code that is already running on the M4F

    3) Now you should be able to see the code that is running on the M4F, that Linux loaded with the RemoteProc driver during the Linux boot process

    What background info do you need to know? 

    When the M4F binary is loaded, it should place the resource table at a known location in memory (you can see this in the DDR allocation of the M4's linker.cmd file). This happens regardless of whether the M4F is loaded by Linux, SBL, or CCS.

    HOWEVER, the resource table is missing critical information (the location of the VRINGS that are used to pass RPMsg packets between cores). The Linux RemoteProc driver is the one that adds that missing information to the resource table. The M4F cannot start doing RPMsg until after Linux updates the resource table.

    My guess is that you are not getting Linux to fill in that resource table information after CCS initializes the M4F, so the M4 is just waiting for that information to get written.

    How can you help me improve the documentation? 

    My suspicion is that if you follow the exact steps at https://software-dl.ti.com/mcu-plus-sdk/esd/AM62X/08_04_00_16/exports/docs/api_guide_am62x/CCS_LAUNCH_PAGE.html , including resetting and reloading the M4F (instead of just loading symbols), then the RPMsg example will get stuck at the exact place that you are observing. Can you confirm that for me?

    If that suspicion is true, I need to work with the MCU+ developers to update that documentation so that it works out of the box.

    Regards,

    Nick

  • I'm sorry that I didn't make it clear. After my development board is powered on, linux will be started through sd card, and then m4 kernel will also be started. At this point I went through software-dl.ti.com/.../CCS_LAUNCH_PAGE.html The above step is to debug m4 kernel on ccs, other routines are running normally, indicating that m4 kernel is started normally, only RPMSG example runtime is stuck in RPMessage_waitForLinuxReady function, I am not clear about the resource table you mentioned is missing key information, So I feel that your suspicion should be correct. If so, how should I operate in order to run normally

  • Hello,

    Please try Option 1 first and let me know if it works.

    Option 1: connect CCS to watch M4 firmware that was loaded by Linux

    Reference section "This process should work for your debug" in my previous response. You can find load symbols by going Load > load symbols.

    Option 2: use CCS to load new firmware onto the M4.

    for Option 2, you would need the Linux IPC drivers to re-run so they can add in the resource table information. These steps might work. I am double-checking to see if the remoteproc module also needs to get reloaded.

    >> On Linux, remove the Linux IPC module
    rmmod virtio_rpmsg_bus
    rmmod rpmsg_char

    >> From CSS: do an M4 CPU reset, load the firmware, and run the firmware

    >> Then on Linux, restart the IPC module:
    modprobe virtio_rpmsg_bus

    modprobe rpmsg_char

    Regards,

    Nick

  • I have done the content in option 1, the program is indeed stuck, do I need to try option 2?

  • Hello,

    What do you mean by "stuck"? Can you provide screenshots of what you are seeing and what you are doing for option 1?

    Option 1 is the easier and quicker option.

    There is another customer who just got option 2 working. See this response:
    https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1180949/am625-ccs-simulates-ipc-exception/4448576#4448576

    Regards,

    Nick

  • ...I added a few print messages before and after RPMessage_waitForLinuxReady function. When I run it, I do not reset the cpu, I go straight to download and resume without any print messages, but after resetting the cpu and downloading and resum, I will prompt you to print messages. Figure 1 shows the contents of the program. Figure 2 is not resetting the cpu and Figure 3 is resetting the cpu

  • Why can't I succeed with option 2? Can you explain the specific steps in detail? How can m4 run after ipc driver is uninstalled and reloaded?

  • Hello,

    Please give me a couple of days to run tests on my end. I want to verify that the information I am giving you is correct.

    Regards,

    Nick

  • Hello,

    Thank you for your patience. OPTION 1 (where Linux loads the cores) is working exactly as I expected it to. More information below. Note that in this option, CCS does NOT restart the M4F, and CCS does NOT load new firmware into the M4F.

    I was not able to find the source code for the prebuilt binary ipc_echo_baremetal_test_mcu2_0_release_strip.xer5f that is packaged in the Linux SDK 8.5 default image filesystem under /lib/firmware/pdk-ipc. I am asking followup questions to the developers. In the meantime, I ran tests with the MCU+ IPC project examples/drivers/ipc/ipc_rpmsg_echo_linux/am62x-sk/m4fss0-0_freertos.

    OPTION 1: attach to firmware that is already running from Linux

    First, boot Linux and make sure that the RemoteProc driver is pointing to the correct firmware: https://dev.ti.com/tirex/explore/node?node=A__AISILbMWZ4d84U2oSmXcdA__linux_academy_am62x__XaWts8R__LATEST (booting the correct firmware to the M4)

    Run tests to make sure that the example is running properly:
    https://dev.ti.com/tirex/explore/node?node=A__ACKtJpWz8b53RwT-PJ4qmA__linux_academy_am62x__XaWts8R__LATEST (steps to run the IPC example)

    Second, attach CCS

    Plug in the JTAG cable, and start the AM62x Target Configuration.

    Connect to the core BLAZAR_Cortex_M4F_1

    Load Symbols, as shown in this image:

    Select the .out file to load:

    You can now step through the code already running on the M4F.

    The example I listed above is not stuck on my side. I can even test from the Linux side as described in the Linux academy: https://dev.ti.com/tirex/explore/node?node=A__ACKtJpWz8b53RwT-PJ4qmA__linux_academy_am62x__XaWts8R__LATEST

    When the core is running in CCS, the Linux example runs. When the core is paused in CCS, the Linux example does not move forward until the core is unpaused. When I step through the CCS code, the Linux example will move forward one "echo" at a time until all the echos are completed.

    Regards,

    Nick

  • For now, I would probably suggest going with OPTION 1, and just adding software breakpoints into your M4F code so that the M4F stays at the point where you want to start debugging until you can connect to CCS.

    A software breakpoint can be as easy as setting a variable value, then waiting until the variable value is changed in CCS, or as complicated as reading a value in a known memory location that you can write from Linux userspace.

    example software breakpoint:

    breakpoint1 = 0;
    while( breakpoint1 != 1 ) {}

    Regards,

    Nick

  • I have two points to make.

    1. I have a new problem. When I run the ipc_rpmsg_echo_linux example at the beginning, some corresponding information will be output, but recently when I run the ipc_rpmsg_echo_linux example, there is no information output at all. However, when I ported the contents of the ipc_rpmsg_echo_linux example to another empty project, the output was fine (again stuck in the reday function); I don't know why. I want to know if this phenomenon will affect my operation here;

    2. When I transplant the ipc_rpmsg_echo_linux example to an empty project and run it, I still follow the operation mode of option 1, and the phenomenon as shown in the screenshot will appear. Do you understand this phenomenon

  • Hello,

    I am on vacation today and the next couple of days. Please ping the thread if I have not replied by the middle of next week.

    Regards,

    Nick

  • I follow option 1,My operation is to have a press f6 key to step over,The program is stuck in the for loop

  • Hello,

    The behavior in the first video is typically what I see if the symbols that are loaded do not match the firmware that is running on the core: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1180017/am623-am623-mcu-sdk/4457630#4457630

    Based on the second video, it sounds like you got the symbols loaded properly. When I would just randomly stop the M4 core, I saw similar behavior as you did in your second video. Please try using CCS breakpoints to get the code to stop while it is in the desired code.

  • Setting a breakpoint in CCS

    First, go back to the "CCS Edit" view and open the file where you want to place the breakpoint. For this example, I opened ipc_rpmsg_echo.c.

    Next, go back to the "CCS Debug" view.

    Go to the line of code where you want CCS to halt the program. Right click on the line number, and select Breakpoint (Code Composer Studio) > Breakpoint

    You should now see a blue circle on that line number. The blue circle tells you that you have a breakpoint there.

    Testing the breakpoint

    Now let's run some tests.

    In CCS, make sure that the M4F core is running (not suspended)

    In the linux terminal, I run the rpmsg_char_simple command:
    rpmsg_char_simple -r 9 -n 5

    In CCS, you should now see that the M4F is suspended at the breakpoint that you set. You can now move through the IPC code.

    Regards,

    Nick

  • 1.Do you mean that the firmware monitored on the first video is not the same as the firmware running on the kernel? So how do I download the firmware? I thought I could just run the load program on ccs and download the firmware.;

    2.One more question: did ccs download the program to emmc or to ospi connected flash, this question is very important to me

  • 1 我猜TI工程师的意思是  你flash上的M核应用程序和你ccs仿真的应该是一个。

    2 可以用xmodem FTP SFTP多种方式将M程序替换

  • Hello,

    1)

    Just take the .out file that you built in CCS, and put the binary in the Linux filesystem. You tell Linux to load the new binary by following the steps in the Linux Academy: https://dev.ti.com/tirex/explore/node?node=A__AISILbMWZ4d84U2oSmXcdA__linux_academy_am62x__XaWts8R__LATEST

    The binary program is stored wherever your Linux filesystem is stored. So in the out-of-the-box EVM, the program is saved to the SD card.

    When CCS loads symbols, it is NOT actually downloading anything onto the EVM. It is just telling CCS what software is running on the EVM.

    2)

    This forum is English language only. Questions can be asked in Chinese here: https://e2echina.ti.com/

    Normally we delete questions and responses that are written in other languages, but I'll leave Yu's response up for now in case it is helpful.

    Regards,

    Nick

  • 1 After loading symbols, two breakpoint were set, one before rpmessage_waitforlinuxready and one after rpmessage_waitforlinuxready.

    2 1After loading the program, the program stuck in rpmessage_waitforlinuxready, but after loading symbols,, directly into the task for loop. 

  • ok, I see what you mean

  • My situation is a little special. The ospi line of the development board is connected to another place when debugging by others, and it is not connected with flahs. Will it affect my ability to do the content of option 1

  • Yu,

    1) Are you testing with the unmodified ipc_rpmsg_echo_linux code, or is this custom firmware?

    2) What is the exact process you are following to initialize the code? If Linux is loading the firmware (NOT CCS), then by the time CCS attaches to the core the M4 should have already gone by rpmessage_waitforlinuxready.

    Take a look at the timeline I list here: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1180949/am625-ccs-simulates-ipc-exception/4449879#4449879 

    After step e), the M4 firmware starts running. It runs to rpmessage_waitforlinuxready. Then the M4 waits for Linux to update the resource table.

    If Linux is initializing the M4, then Linux will do step f) to update the resource table.

    After step f), rpmessage_waitforlinuxready sees the updated resource table, and continues running.

    If you want the M4F to stop BEFORE rpmessage_waitforlinuxready until you have attached CCS, I would write a software breakpoint (NOT a CCS breakpoint) into the firmware before rpmessage_waitforlinuxready. Note that you will want to compile the new code, and copy the new firmware into the Linux filesystem to be loaded.

    I describe writing a software breakpoint here: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1180017/am623-am623-mcu-sdk/4457466#4457466

    Regards,

    Nick

  • ,

    I do not understand your question.

    Is Linux still able to run and access the Linux filesystem while you are debugging? If Linux can access the Linux filesystem, then Linux can load firmware into the M4F.

    Regards,

    Nick

  • ok, let's not worry about this problem, I have looked at the file AM62-MMC-M4F0_0-FW running on my kernel, do I need to change it? What is the difference between am62-MMC-M4F0_0-fw and.out;

  • Please refer to the Linux Academy for how to change the firmware getting loaded by the kernel. You can point the symbolic link to the .out file after copying the .out file to the Linux filesystem.

    dev.ti.com/.../node

    Regards,

    Nick

  • 1 I am using the development board for ti am625-sk 。

    2 The version of the sdk is mcu_plus_sdk_am62x_08_04_00_16 and I have not modified anything 。

    3 I tested both the am62-mcu-m4f0_0-fw and * out and both failed 。

    4 In the simulation, it seems as if the program did not enter the main function, directly into the for cycle。

    Are there any steps wrong in my operation?

     

  • Hello,

    In the first video you shared, I would not expect any of your breakpoints to get hit. The program would have already gone past that point by the time CCS connected to it. I would expect that you could place a breakpoint in the while() loop that my breakpoint was in for this response, and then you should see the breakpoint trigger when going through the steps in this response: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1180017/am623-am623-mcu-sdk/4461332#4461332

    In the second video you shared, please note that "graceful shutdown" is not supported on AM62x & AM64x as of SDK 8.5. Please reference https://dev.ti.com/tirex/explore/content/linux-academy-AM62X-8.4.1.3.v2/ch-eval/eval-booting-remote-cores.html#booting-remote-cores-from-the-linux-console-or-user-space 

    What does "graceful shutdown" mean? Sometimes when Linux shuts down a remote core with echo stop, the remote core will be in a known good state. However, sometimes the remote core will be stopped in a bad state. If you try to re-start the remote core in a bad state, the processor might crash.

    Regards,

    Nick

  • 1 I set the breakpoint in while (1) loop and then enter ". / Rpmsg _ char _ simple-r 9 ", the program was indeed stopped at the breakpoint,it worked.

    2 This method can only work for freertos's while (1). Can't be from the main function is ok? 

    3 If it is a Baremetal running program, without freertos, is it also working in while (1)? 

  • I didn't quite understand what the ti staff meant, may I know how you did the first one

  • Hello yu & ,

    I assumed you were working on this project together.

    Yu, in the future, if you have a question, please create a new thread instead of asking questions on someone else's thread. (however, if you have suggestions or input for someone else, please feel free t reply to their threads!) It is hard for me and for future customers to keep track of multiple questions coming from multiple people.

    yu's questions: 

    Glad to hear that the program is running as expected.

    Your other questions are about how RTOS tasks work, and how that differs from how baremetal works.

    In general, RTOS main functions start up a series of tasks that run (the while() loop is within one of those tasks). The RTOS then decides which tasks get processing time on the M4 or R5 at what time. Baremetal will look different, because there is no OS that is determining which task gets processing time.

    Please create a separate thread if you want to ask followup questions about RTOS, or whether there is a baremetal IPC example project.

    Regards,

    Nick

  •  question:

    I will combine several of my previous responses into a single reply, and add additional information. Hopefully this is helpful for you.

    This is the process to load M4F firmware from Linux, then attach CCS to the firmware that is already running

    Step 1: build the project in CCS

    Take the MCU+ IPC project examples/drivers/ipc/ipc_rpmsg_echo_linux/am62x-sk/m4fss0-0_freertos.

    Open the project in CCS. Build the project.

    You can find steps for how to build a project in CCS here: https://software-dl.ti.com/mcu-plus-sdk/esd/AM62X/08_05_00_14/exports/docs/api_guide_am62x/GETTING_STARTED_BUILD.html 

    Step 2: copy the binary into the Linux filesystem, and update the links

    Copy the .out file from CCS into the Linux filesystem somewhere under /lib/firmware.

    Boot up the EVM. In the terminal, update the M4 symbolic link to point to the new .out file.

    Steps for how to update the symbolic link are here: https://dev.ti.com/tirex/explore/node?node=A__AISILbMWZ4d84U2oSmXcdA__linux_academy_am62x__XaWts8R__LATEST

    Step 3: make sure the firmware is running properly 

    Reboot the EVM.

    Run tests to make sure that the RPMsg example is running properly. Follow the steps here:
    https://dev.ti.com/tirex/explore/node?node=A__ACKtJpWz8b53RwT-PJ4qmA__linux_academy_am62x__XaWts8R__LATEST (steps to run the IPC example)

    Step 4: attach CCS

    Plug in the JTAG cable, and start the AM62x Target Configuration.

    Connect to the core BLAZAR_Cortex_M4F_1

    Load Symbols, as shown in this image:

    Select the same .out file that was copied to the Linux filesystem, and Linux loaded to the M4:

    Step 5: Set a breakpoint in CCS

    Open ipc_rpmsg_echo.c in the "CCS Debug" view.

    Go to the "ipc_recv_task_main".

    Do you see the "while(1)" loop? This is where the ipc_recv_task_main task will be when the RTOS runs the task.

    Set a breakpoint somewhere INSIDE the while(1) loop:

    Go to the line of code where you want CCS to halt the program. Right click on the line number, and select Breakpoint (Code Composer Studio) > Breakpoint

    You should now see a blue circle on that line number. The blue circle tells you that you have a breakpoint there.

    Step 6: Testing the breakpoint

    Now let's run some tests.

    In CCS, make sure that the M4F core is running (not suspended)

    In the linux terminal, I run the rpmsg_char_simple command:
    rpmsg_char_simple -r 9 -n 5

    In CCS, you should now see that the M4F is suspended at the breakpoint that you set.

    In the Linux terminal, you should see that the terminal has paused partway through the test. The Linux terminal will not get any RPMsg example updates until you unpause the M4.

    You can now move through the IPC code in CCS. As you move through the code, you will see more lines show up in the Linux terminal.

    Regards,

    Nick

  • 1.OK  If there are other problems, I will open a separate thread.

    2 For now, I recommend method 2, which needs to reload the modules, but can simulate all the programs 

  • Ok, thank you very much for the solution. Could you please keep this thread? If there are any questions in the future, I would like to leave a message here

  • I have turned your question into an FAQ here:

    https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1188177/faq-am62x-am64x-how-to-use-ccs-to-debug-a-running-m4f-core-that-was-started-by-linux

    If you have a followup question, feel free to reply to this thread or create a new thread. The thread will lock after 30 days if nobody replies, but in that case just create a new thread and provide a link to this thread.

    Regards,

    Nick