Linux/EVMK2H: Using QMSS and PKTDMA between ARM and DSP while running Linux on ARM

Hi,

We're looking into this. Feedback will be posted here.

Best Regards,
Yordan

Hi, Moein,

The standard way of inter-core communication is using IPC over RPMsg. If you insists of using QMSS, there is a QMSS_LLD and its examples in processors.wiki.ti.com/.../Processor_SDK_Resource_Manager

It does a qmss_put to transmit on one end, and qmss_get to receive on the other end.

XGE functionality is irrelevant to inter-core communication. I don't think we have any XGE RTOS example. I'll need to check with RTOS engineer.

Rex

Thank you very much Rex, your link clarified several things for me. 

Using the DSP for XGE packet processing is my main goal, and since the interface to XGE is PKTDMA I wanted to use the same interface for inter-core communication which can also be used among DSP cores.

Also still I don't know how to properly setup the QMSS from ARM side, possibly I need some driver modifications. I changed the rx-queue for XGE in DTS file to use low priority queues already setup in the same DTS file. Now the queue numbers are updated (used printk during setup to make sure) but still ARM has the full control and does not send the packets to DSPs.

Moein

Hi, Moein,

Low priority queue may not work. It generates an interrupt 226 which is defined in K2H datasheet as ARM Navigator interrupt. You probably should define a rx flow using general purpose queue for the desination DSP. Also, a tx queue for the DSP which the XGE will receive from.

You may have read the Multicore Navigator User's Guide, www.ti.com/.../sprugr9h.pdf. It has detail and useful info.

Rex

Hi Rex,

I'm a little bit confused, in the Multicore Navigator User's Guide I see these queue allocations:

Which there is separate interrupts for ARM (8704 to 8735) and actually XGE was using these interrupts for RX queue. On low priority and high priority it says core-specific and later it shows how it maps to DSP cores:

But as you mentioned the low priority queues 480 to 512 are mapped to interrupt 226 and it is sent to ARM processor. I see this in the DTS file:

accumulator {

acc-low-0 {

qrange = <480 32>;
accumulator = <0 47 16 2 50>;
interrupts = <0 226 0xf01>;
multi-queue;
qalloc-by-id;

};

};

Which K2h Datasheet refers to 226 for the low priority queue? Can I change this DTS file to allocate high priority queue interrupts for the DSP? Or even some general purpose queues for now.

Or ARM cannot setup the QMSS for DSP and DSP needs to run the QMSS initialization for its interrupts itself? If that's the case then I would have the same question as the un-answered thread I refereed to, how to initialize QMSS from both ARM and DSP?

BTW I wasn't able to build the PDK example projects, only the "evmk2h platfrom" one is a CCS project and the RM or QMSS test projects in drv folder are some source files, mixed with both ARM and DSP codes. Running make would only build some ".oa15fg" files in the lib directory, which I'm not sure these are the same ".out" files for the ARM that can be found in the processor SDK root file system under /usr/bin or not . I am looking into figuring it out, but if you know of an explanation how to use these PDK examples I would appreciate it.

Thanks,

Moein

Hi, Moein,

In the SoC datasheet, www.ti.com/.../66ak2h12.pdf, Table 8-23 defines ARM interrupts, page 103 for #226.

In the Queue Maps section (5.1) of the CPPI (or Multicore Navigator) User's Guide, the queues defined in Table 5-2 for Keystone-2 are designated (hard coded) to particular IPs. For example, 8704-8735 are designated to CPSW and defined in netcp.dtsi. You can see both GBE and XGE use them. 8752-8759 are designated to XGE which you can also find corresponding configuration in dtsi file. In the table, none of them are designated to DSP. Therefore, you need to use the gerneral purpose queue.

The above setup is in kernel space. Using RM and QMSS_LLD, it is in user space. The QMSS_LLD provides the API which hides all these configuration, so user can call qmss_put to transmit and the other side does qmss_get to get the message. The binaries for the examples are included in the tisdk-server-extra-rootfs filesystem tarball. If you want to build them, you may need to run pdkProjectCreate script and setup the build env using the pdksetupenv script. You may find some instructions in RTOS PDK for it. In the RM User's Guide, there are instructions on how to run the examples, processors.wiki.ti.com/.../Processor_SDK_Resource_Manager

Rex

Hi Rex,

I see. Table 8-22 has interrupts for core-pac and the only navigator interrupts are the high priority one such as:

Event 52, QMSS_INTD_2_HIGH_N: Navigator 2 accumulated hi-priority interrupt 0

My goal is to change the routing from "XGE to ARM" to "XGE to DSP" not to add a redundant latency going through ARM. So I think now the question is how should I setup the high priority interrupt, I'm fine if I need to modify kernel and rebuilt it. I'm actually looking for where this qmss setup steps are performed so I can modify them. Can I do some modifications to the ARM dtsi file to change the setup and only use QMSS_LLD on the DSP side to send/receive data, and if not which kernel modules should I modify on each side? 

I couldn't find XGE support on the DSP side, so that's why I'm trying to set it up through ARM. If XGE setup is possible through DSP and still I can use ARM cores for booting Linux and talk to DSPs over IPC or QMSS that fulfills my goal too. I would appreciate it if you point how to do it if it's doable.

Thank you for the guidance on the PDK.

Best,

Moein

Hi, Moein,

PktDMA provides a flexible approach but care should be taken to avoid any existing usage of resources. High priority queue can be one of the solutions as long as it is not used by others. General purpose queues can also be an option. There isn't a set rules of what you must use. Setting up the queues on linux side doesn't mean DSP will automatically get it. You will need code to consume the queue. You may want to consider how to manage the interrupt flood on XGE if using interrupt.

RTOS XGE example only set up SerDes for the BER test. It doesn't set up for receiving and transmitting packets. It also needs to set up queues for transmit/receive. This queue set up also applies to GbE which may be useful as a reference.

Rex

Hi Rex,

I still can't find the examples you mentioned or the one described in the "Processor_SDK_Resource_Manager". It says: "Import the rmK2xArmv7LinuxDspClientTestProject into CCS and build the project" but I could not find such project and there is no css project inside the drv folder of pdk. 

Based on your pointer to the setupenv script I ran the "make pdk" from the rtos folder but it ran into some errors. For now I did update two of the software versions as bellow but I still get errors during build:

1)Added this line to ~/ti/processor_sdk_rtos_k2hk_4_01_00_06/makefile:
   export IPC_INSTALL_PATH="${SDK_INSTALL_PATH}/ipc_3_46_02_04"

2) In ~/ti ran: sudo grep -rl "bios_6_46_04_53" . | xargs sed -i 's/bios_6_46_04_53/bios_6_46_05_55/g'

errors: 

# Compiling k2h:c66x:release:osal_tirtos: src/tirtos/SemaphoreP_tirtos.c

/home/moein/ti/ti-cgt-c6000_8.1.3/bin/cl6x -ppd=/home/moein/ti/pdk_k2hk_4_0_7/packages/ti/binary/ti/osal/osal_tirtos/obj/k2h/c66/release/.deps/SemaphoreP_tirtos.P -DMAKEFILE_BUILD -mv6600 --abi=eabi -q -mi10 -mo -pden -pds=238 -pds=880 -pds1110 --program_level_compile -g --endian=little -eo.oe66 -ea.se66 --emit_warnings_as_errors -o3 --optimize_with_debug -DSOC_K2H -Dxdc_target_types__=ti/targets/elf/std.h -I/home/moein/ti/ti-cgt-c6000_8.1.3/include -I. -Isrc/tirtos -Isrc/tirtos/muxintcp/v0 -I/home/moein/ti/xdctools_3_32_01_22_core/packages -I/home/moein/ti/pdk_k2hk_4_0_7/packages -I/home/moein/ti/bios_6_46_04_53/packages -fr=/home/moein/ti/pdk_k2hk_4_0_7/packages/ti/binary/ti/osal/osal_tirtos/obj/k2h/c66/release -fs=/home/moein/ti/pdk_k2hk_4_0_7/packages/ti/binary/ti/osal/osal_tirtos/obj/k2h/c66/release -fc src/tirtos/SemaphoreP_tirtos.c
"/home/moein/ti/xdctools_3_32_01_22_core/packages/xdc/std.h", line 80: fatal error #1965: cannot open source file "ti/targets/elf/std.h"
1 catastrophic error detected in the compilation of "src/tirtos/SemaphoreP_tirtos.c".
Compilation terminated.

In another approach I searched for any project in the pdk folder and found the evmk2h platform test project. That one did not build and after figuring out the link errors and solving them now during loading the project to DSP I get this (I tried CCS both on windows and linux):

C66xx_0: Trouble Reading Memory Block at 0x8000 on Page 0 of Length 0x4: (Error -1190 @ 0x8000) Unable to access device memory. Verify that the memory address is in valid memory. If error persists, confirm configuration, power-cycle board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 7.0.48.0)
C66xx_0: File Loader: Verification failed: Target failed to read 0x00008000
C66xx_0: GEL: File: C:\ti\pdk_k2hk_4_0_7\packages\ti\platform\evmk2h\platform_test\Debug\platform_test_evmk2h.out: Load failed.

I would appreciate it if you point me where to find those projects and solve these problems if necessary, like another path or version setup file that I'm missing.

Thanks,

Moein