Keystone II 66ak2h qmss LLD used by both DSP and ARM

Dan Skehan7

Other Parts Discussed in Thread: SYSBIOS

Hi,

In the past I have ran with a Keystone I, but am not using a Keystone II device. I would like to have the DSPs use QMSS LLD similarly to how I used it before and I would like to have the ARM Linux use the QMSS LLD as well.

At the moment I have the same code I had working for the Keystone I DSPs running (with minor mods to account for the ARM). The DSP initialize QMSS like they used to and work fine. However if I have the ARM Linux attempt to open a queue (Qmss_queueOpen) after the DSP initialization it fails.

The Qmss_queueOpen() fails with an error code of -130 (QMSS_NOT_INITIALIZED). After searching through the QMSS LLD, I found that this function must be failing when it checks qmssLObjIsValid[QMSS_MAX_SUBSYS] = { 0, 0 }; If compiled for the DSP, a #ifdef _TMS320C6X flag puts this variable in a specific data section for each of the DSPs to find. This is not the case for the ARM build.

How is the ARM Linux supposed to know if a DSP initialized the QMSS or vice versa. I noticed this variable has a space for 2 flags. Does the ARM Linux do a separate QMSS initialization? If so, how do the two coexist?

Thanks in advance,

-Dan

over 7 years ago

0 Yordan Kovachev over 7 years ago

TI__Guru**** 161600 points

Hi Dan,

I've notified the 66AK2H experts. Their feedback will be posted here.

Best Regards,
Yordan

0 Garrett Ding over 7 years ago in reply to Yordan Kovachev

TI__Mastermind 43296 points

Dan,

Have you tried resource manager (RM) provided in Processor-SDK for managing QMSS in ARM + DSP?

The Resource Manager (RM) is delivered as part of Processor-SDK as a means for managing system resource contentions.

A brief introduction of RM, including test case, is here: http://processors.wiki.ti.com/index.php/Processor_SDK_RTOS_RM

The more detailed RM & QMSS LLD info is here: http://processors.wiki.ti.com/index.php/MCSDK_UG_Chapter_Developing_System_Mgmt#QMSS_LLD

RM Supported LLDs

At this point in time the following LLDs have had RM support added:

- QMSS
- CPPI
- PA

Regards,

Garrett

0 Dan Skehan7 over 7 years ago in reply to Garrett Ding

Intellectual 560 points

Hi Garrett,

Thank you for your reply.

I followed the instructions from:

http://processors.wiki.ti.com/index.php/MCSDK_UG_Chapter_Developing_System_Mgmt#QMSS_LLD

I have started by changing one thing at a time. i initialized RM and added the RM handle to the QMSS config for the DSP cores. When I run this core 0 and 1 come up fine, but all the other DSP cores abort and dump their registers.

This behavior seems to match what I have read for the ARM/DSP config in:

http://processors.wiki.ti.com/index.php/Processor_SDK_RTOS_RM

"Application successfully completes with no memory leaks on two DSP cores."

It appears to me that the RM does not work with more than 2 cores. Is this true? or is there a different config needed?

-Dan

0 Garrett Ding over 7 years ago in reply to Dan Skehan7

TI__Mastermind 43296 points

Dan,

The default RM DTB files should have 8 core support out of the box. The “successfully completes on 2 cores” text is the output of the RM tests/examples which run on two cores. The test/example outputs are not reflective of the total number of cores RM supports.

Do you have the DSP failure log? We can look into it and try to find the root cause.

Regards,
Garrett

0 Dan Skehan7 over 7 years ago in reply to Garrett Ding

Intellectual 560 points

Hi Garrett,

Thank you for your quick reply.

So far I am running a DSP application that has worked for me for a long time. The only difference is that I added the RM and gave it's handle to QMSS using the following during initialization:

int initRm (void)
	{
	    Rm_InitCfg         rmInitCfg;
	    int32_t            result;

	    /* Initialize the RM Client - RM must be initialized before anything else in the system */
	    memset(&rmInitCfg, 0, sizeof(rmInitCfg));

	    rmInitCfg.instName = rmClientName;
	    rmInitCfg.instType = Rm_instType_CLIENT;
	    rmClientHandle = Rm_init(&rmInitCfg, &result);

	    printf("\n\nInitialized %s\n\n", rmClientName);

	    /* Open Client service handle */
	    rmServiceHandle = Rm_serviceOpenHandle(rmClientHandle, &result);

	    return 0;
	}

...

if (p_qmss_cfg->master_core)
    {

...
  
      /* Register RM Service handle with QMSS */
        qmssGblCfgParams.qmRmServiceHandle = rmServiceHandle;
        result = Qmss_init (&qmssInitConfig, &qmssGblCfgParams);

        if (result != QMSS_SOK)
        {
            printf ("Error initializing Queue Manager SubSystem, Error code : %d\n", result);
            return -1;
        }

      /* Start Queue manager on this core */
      Qmss_start ();
    }
    else
    {
		/* Register RM service handle with QMSS */
		qmssStartCfg.rmServiceHandle = rmServiceHandle;
		result = Qmss_startCfg(&qmssStartCfg);
	}

I got the following output before several cores crashed:

2 Resource entries at 0x800000
***************************Starting with Core: #0*******************************
core:0, (hyplnkTest.c), 488...
Waiting for IPC Start
2 Resource entries at 0x800000
***************************Starting with Core: #1*******************************
core:1, (hyplnkTest.c), 488...
Waiting for IPC Start
2 Resource entries at 0x800000
***************************Starting with Core: #2*******************************
core:2, (hyplnkTest.c), 488...
Waiting for IPC Start
A0=0x0 A1=0x0
A2=0xc12eba4 A3=0x0
A4=0x0 A5=0xc
A6=0x1000 A7=0x1844018
A8=0x100 A9=0x80d8d0
A10=0x80a7e8 A11=0xc109748
A12=0x0 A13=0x0
A14=0x0 A15=0x0
A16=0x0 A17=0xc137780
A18=0x4c A19=0x80adbc
A20=0x0 A21=0x80adbc
A22=0x0 A23=0x20
A24=0x0 A25=0x0
A26=0x80adbc A27=0x80adbc
A28=0x80c398 A29=0x0
A30=0x14 A31=0x7f
B0=0x0 B1=0x0
B2=0x0 B3=0x0
B4=0xd B5=0xd
B6=0x1848030 B7=0x0
B8=0x0 B9=0x1
B10=0x0 B11=0x0
B12=0x0 B13=0x1
B14=0x80e300 B15=0x80d900
B16=0xc137138 B17=0x0
B18=0xf400d0c B19=0x15000100
B20=0x0 B21=0x69
B22=0x1 B23=0xfffffffd
B24=0x0 B25=0x2 Resource entries at 0x800000
***************************Starting with Core: #4*******************************
core:4, (hyplnkTest.c), 488...
Waiting for IPC Start
A0=0x1 A1=0x0
A2=0xc12eba4 A3=0xfff5
A4=0xfff5 A5=0xc
A6=0x1000 A7=0x1844018
A8=0x100 A9=0x80d8d0
A10=0x0 A11=0xc0431bc
A12=0x0 A13=0x80a7e4
A14=0x80a7fd A15=0xffffffe1
A16=0x0 A17=0xc137780
A18=0x4c A19=0x80adbc
A20=0x0 A21=0x80adbc
A22=0x0 A23=0x20
A24=0x1 A25=0x0
A26=0x80adbc A27=0x80adbc
A28=0x80c398 A29=0x0
A30=0x14 A31=0xc0431cc
B0=0x0 B1=0x0
B2=0x0 B3=0x0
B4=0x80a7f1 B5=0xd
B6=0x1848030 B7=0x0
B8=0x0 B9=0x1
B10=0x80a7e4 B11=0x0
B12=0xc137924 B13=0x90
B14=0x80e300 B15=0x80d8c8
B16=0xc137138 B17=0x0
B18=0xf400d0c B19=0x15000100
B20=0x0 B21=0x69
B22=0xffffffff B23=0xfffffffd
B24=0x1 B25=0x66f90397
B26=0x80adbc B27=0x1
B28=0x80b980 B29=0x80c240
B30=0x0 B31=0xc043590
NTSR=0x1000c
ITSR=0x0
IRP=0x90
SSR=0x0
AMR=0x0
RILC=0xc0431bc
ILC=0x0
Exception at 0x0
EFR=0x2 NRP=0x0
Internal exception: IERR=0x1
Instruction fetch exception
ti.sysbios.family.c64p.Exception: line 256: E_exceptionMax: pc = 0x00000000, sp
= 0x0080d8c8.
xdc.runtime.Error.raise: terminating execution
2 Resource entries at 0x800000
***************************Starting with Core: #5*******************************
core:5, (hyplnkTest.c), 488...
Waiting for IPC Start
A0=0x1 A1=0x0
A2=0xc12eba4 A3=0xfff5
A4=0xfff5 A5=0xc
A6=0x1000 A7=0x1844018
A8=0x100 A9=0x80d8d0
A10=0x0 A11=0xc0431d0
A12=0x0 A13=0x80a7e4
A14=0x80a7fd A15=0xffffffe1
A16=0x0 A17=0xc137780
A18=0x4c A19=0x80adbc
A20=0x0 A21=0x80adbc
A22=0x0 A23=0x20
A24=0x1 A25=0x0
A26=0x80adbc A27=0x80adbc
A28=0x80c398 A29=0x0
A30=0x14 A31=0xc0431e0
B0=0x0 B1=0x0
B2=0x0 B3=0x0
B4=0x80a7f1 B5=0xd
B6=0x1848030 B7=0x0
B8=0x0 B9=0x1
B10=0x80a7e4 B11=0x0
B12=0xc137924 B13=0x90
B14=0x80e300 B15=0x80d8c8
B16=0xc137138 B17=0x0
B18=0xf400d0c B19=0x15000100
B20=0x0 B21=0x69
B22=0xffffffff B23=0xfffffffd
B24=0x1 B25=0x7e5bd37a
B26=0x80adbc B27=0x1
B28=0x80b980 B29=0x80c240
B30=0x0 B31=0xc043690
NTSR=0x1000c
ITSR=0x0
IRP=0x90
SSR=0x0
AMR=0x0
RILC=0xc0431d0
ILC=0x0
Exception at 0x0
EFR=0x2 NRP=0x0
Internal exception: IERR=0x1
Instruction fetch exception
ti.sysbios.family.c64p.Exception: line 256: E_exceptionMax: pc = 0x00000000, sp
= 0x0080d8c8.
xdc.runtime.Error.raise: terminating execution
2 Resource entries at 0x800000
***************************Starting with Core: #6*******************************
core:6, (hyplnkTest.c), 488...
Waiting for IPC Start
A0=0x0 A1=0x0
A2=0xc12eba4 A3=0x0
A4=0x0 A5=0xc
A6=0x1000 A7=0x1844018
A8=0x100 A9=0x80d8d0
A10=0x80a7e8 A11=0xc109748
A12=0x0 A13=0x0
A14=0x0 A15=0x0
A16=0x0 A17=0xc137780
A18=0x4c A19=0x80adbc
A20=0x0 A21=0x80adbc
A22=0x0 A23=0x20
A24=0x1 A25=0x0
A26=0x80adbc A27=0x80adbc
A28=0x80c398 A29=0x0
A30=0x14 A31=0x7f
B0=0x0 B1=0x0
B2=0x0 B3=0x0
B4=0xd B5=0xd
B6=0x1848030 B7=0x0
B8=0x0 B9=0x1
B10=0x0 B11=0x0
B12=0x0 B13=0x1
B14=0x80e300 B15=0x80d900
B16=0xc137138 B17=0x0
B18=0xf400d0c B19=0x15000100
B20=0x0 B21=0x69
B22=0x1 B23=0xfffffffd
B24=0x0 B25=0x

I have never used the RM before. Please let me know if I need to edit the configuration

Thank you,

-Dan

0 Dan Skehan7 over 7 years ago in reply to Dan Skehan7

Intellectual 560 points

Hi,

I have an update. I was getting a little desperate and tried tweaking a few things. I ended up changing my compiler version from 7.4.1.2 to 7.4 and this helped tremendously. Now none of the cores crash and dump their registers like before. Now each of the DSP cores appear to get past the RM initialization with no errors. However, I still have a DSP core attempting to setup the QMSS using QMSS init like I have previously on Keystone I and the addition of the RM has QMSS_init failing. The error is likely coming from the qmss_drv.c where QMSS control of QM was denied by RM. Can DSP core not be the initializer of the QMSS when using the RM?

-Dan

0 Garrett Ding over 7 years ago in reply to Dan Skehan7

TI__Mastermind 43296 points

Hi Dan,

The compiler version is specified in Processor SDK RTOS release note - TI C6000 Compiler 8.1.0, .

I am just seeing your Feb. 1 post. I didn't get automated mail from the system somehow : -(

Are you certain the QMSS_init failure is from the piece of code below in qmss_drv.c?

    if (qmssGblCfgParams->qmRmServiceHandle)
    {
        int32_t qmControl = 0;

        /* Attempt to allocate control of all the QMs. Right now both JOINT and SPLIT modes use all QMs */
        if ((!Qmss_rmServiceGroups((Rm_ServiceHandle *)qmssGblCfgParams->qmRmServiceHandle, Rm_service_RESOURCE_ALLOCATE_INIT,
                                   offsetof(Qmss_GlobalConfigGroupRm, rmQmControl), &qmControl, 1, 0, NULL, qmssGblCfgParams)))
        {
            return QMSS_QM_CONTROL_DENIED;
        }
    }

Regards,

Garrett

0 Dan Skehan7 over 7 years ago in reply to Garrett Ding

Intellectual 560 points

Hi Garrett,

Thank you for replying.

Yes, I have confirmed that is the error code I am seeing.

When I do not pass a RM handle to QMSS_init(), the DSP cores initialize fine and proceed to run as I expect.

What would cause the RM to deny resources?
I currently only have core 0 performing the QMSS_init() and its rmClientName is "RM_Client0" which I got from the policy_dsp_arm.dts valid-instances.
Do I need to first perform an action on the server side on the ARM cores?

-Dan

0 ran35366 over 7 years ago in reply to Dan Skehan7

TI__Genius 12805 points

Dan, this might help:

Is the user starting a RM Server on at least one of the cores (usually core 0)? Their code snippet only shows RM clients being created. In DSP scenarios, one core should be running an RM server and the remaining running clients. The server and clients are connected over an IPC transport for communication. We have examples which use the test/rm_transport_setup.c utility to set up IPC transports between the clients and the server. Many PDK component examples use this utility. In fact, the QMSS InfraDmaMC example is a really good example of how to setup and use RM within a multi-core application.

https://bitbucket.itg.ti.com/projects/PROCESSOR-SDK/repos/qmss-lld/browse/example/InfraDmaMC/src/infrastructure_multicoremode.c

https://bitbucket.itg.ti.com/projects/PROCESSOR-SDK/repos/qmss-lld/browse/example/InfraDmaMC/

for all the example source

The latter is a 4 core example. If the user can get 4 cores working with RM they should easily be able to extend to 8.

Ran

0 Dan Skehan7 over 7 years ago in reply to ran35366

Intellectual 560 points

Hi Ran,

The RM server is running on the ARM. That is why all of the DSP code are clients.

The links you posted will not load for me. I think I have the equivalent in the PDK. I will attempt to get it working. It looks very similar to my existing code. I'll see if I can find a difference.

-Dan

0 ran35366 over 7 years ago in reply to Dan Skehan7

TI__Genius 12805 points

Sure, I will wait for your observations

Ran

0 Dan Skehan7 over 7 years ago in reply to ran35366

Intellectual 560 points

Hi Ran,

I have been having trouble getting that example to work. When I run the Linux portion of the example I get the following output:

./qmInfraDmaMC.out
No command line given. Forking off 4 tasks
coreNum: 1; pid=1998
coreNum: 0; pid=1997
coreNum: 2; pid=1999
coreNum: 3; pid=2000

Initialized RM_Client0

Initialized RM_Client2

Initialized RM_Client1

Initialized RM_Client3

CMEM Error: init: Failed to open /dev/cmem: 'No such file or directory'
Failed to initialize CMEM
ERROR: "Top Level Test" fw_memAllocInit failed

I did not find too much for instructions. Am I supposed to supply command line arguments?

Thank you,

Dan

0 ran35366 over 7 years ago in reply to Dan Skehan7

TI__Genius 12805 points

I am really not a LINUX guy so I will ask someone else to look at your issue, meanwhile I suggest you read (if you had not done it before) the following Wiki:

http://processors.wiki.ti.com/index.php/CMEM_Overview

Ran

0 Rex Chang over 7 years ago in reply to ran35366

TI__Guru 50170 points

Hi, Dan,

It seems that the CMEM isn't loaded. Are you using Linux from MCSDK or ProcSDK? The document pointed out by Ran describes how to load the cmem kernel module using insmod. In the ProcSDK, it is enabled in the dts file.

Rex

0 Dan Skehan7 over 7 years ago in reply to Rex Chang

Intellectual 560 points

Hi,

I loaded CMEM using a variation of the CMEM instructions on the wiki:

/sbin/insmod /lib/modules/3.10.72-gca334de/extra/cmemk.ko pools=1x1000000 phys_start=0x85000000 phys_end=0x86000000

It has now progressed me to the next error:

./qmInfraDmaMC.out
No command line given. Forking off 4 tasks
coreNum: 1; pid=1984
coreNum: 2; pid=1985
coreNum: 0; pid=1983
coreNum: 3; pid=1986

Initialized RM_Client2

Initialized RM_Client1

Initialized RM_Client0

Initialized RM_Client3

fw_initMemMapSizeFds: Failed to open "/dev/qmss" err=No such file or directory
ERROR: "Top Level Test" fw_memAllocInit failed

Do I have to modify/configure something for QMSS beforehand?

0 Dan Skehan7 over 7 years ago in reply to Dan Skehan7

Intellectual 560 points

When looking in the code I see the following in fw_test.h:

/* User supplied list of possible memory mapping devices in /dev, and
* /proc/device-tree entry which lists the memory regions allowed */
typedef struct
{
const char devPath[FW_MAX_PATH];
const char procPath[FW_MAX_PATH];
} fwTestMemMapDevProc_t;

Where does the user pass this information? Is this what the code is complaining about?
Are there any instructions on how to use this example? I can not find anything other than a very short readme.

0 Rex Chang over 7 years ago in reply to Dan Skehan7

TI__Guru 50170 points

Hi, Dan,

in src/armv7/linux/fw_mem_allocator.c

/* Create device list */
static fwTestMemMapDevProc_t fwDevProcMap[] = MEM_DEV_PROC;

in k2h/armv7/linux/qmssPlatCfg.h

/* Define memory mapping devices */
#define MEM_DEV_PROC { \
{ "/dev/qmss", "/proc/device-tree/soc/qmss/mem" }, \
}

Did you start rmServer before you run the example?

Rex

0 Dan Skehan7 over 7 years ago in reply to Rex Chang

Intellectual 560 points

Hi Rex,

Thank you for your reply.

I am starting RM server on startup. Just to be sure though I have attempted to run it again with the files specified in the wiki:

http://processors.wiki.ti.com/index.php/MCSDK_UG_Chapter_Developing_System_Mgmt#DSP_.2B_ARM_Linux_Test_Project

I see the following output leading me to believe the rm is working:

Starting ./rmServer.out
Daemon already running on PID file 1829

I have done this after loading cmem:

/sbin/insmod /lib/modules/3.10.72-gca334de/extra/cmemk.ko pools=1x1000000 phys_start=0x85000000 phys_end=0x86000000

[ 184.960709] CMEMK module: built on Sep 30 2015 at 00:40:48
[ 184.966257] Reference Linux version 3.10.72
[ 184.970825] allocated heap buffer 0x85000000 of size 0xf0b000
[ 184.976634] cmemk initialized

I still have the same output as before:

./qmInfraDmaMC.out
No command line given. Forking off 4 tasks
coreNum: 1; pid=1984
coreNum: 2; pid=1985
coreNum: 0; pid=1983
coreNum: 3; pid=1986

Initialized RM_Client2

Initialized RM_Client1

Initialized RM_Client0

Initialized RM_Client3

fw_initMemMapSizeFds: Failed to open "/dev/qmss" err=No such file or directory
ERROR: "Top Level Test" fw_memAllocInit failed

Is there further configuration I need to perform?

0 Dan Skehan7 over 7 years ago in reply to Dan Skehan7

Intellectual 560 points

Is there a QMSS kernel module that needs to be loaded? I can not find anything of the sort.

Processors

Processors forum

Keystone II 66ak2h qmss LLD used by both DSP and ARM