is qmssIpcBenchmark really a Monolithic qpend example?

Hi,

    w.r.t to qmssIpcBenchmark, the other question i have is how the default descriptor type is taken as HOST type

is it so?, when no specification is done as part of

1.Qmss_init(),

2. Cppi_init().

3,Qmss_insertMemoryRegion()

4. .cfg file

please clarify on above points [current post and previous post]

Thanks

RC Reddy

RC,

The qmssIpcBenchmark example benchmarks the IPC QMSS Transport.  The transport instance initialization code configures CPPI to use monolithic descriptors.  It is not done in the benchmark example code.

To see the transport initialization code which configures the CPPI for monolithic descriptors look here:

<your pdk install dir>/packages/ti/transport/ipc/qmss/transports/TransportQmss.c

In the latter source file look at the TransportQmss_Instance_init function.  You'll see the CPPI set to initialize descriptors as monolithic.

      /* Setup the descriptors for the receive free queue */
      descCfg.memRegion           = (Qmss_MemRegion) params->descMemRegion;
      descCfg.descNum             = TransportQmss_numDescriptors;
      descCfg.destQueueNum        = QMSS_PARAM_NOT_SPECIFIED;
      descCfg.queueType           = Qmss_QueueType_STARVATION_COUNTER_QUEUE;
      descCfg.initDesc            = Cppi_InitDesc_INIT_DESCRIPTOR;
      descCfg.descType            = Cppi_DescType_MONOLITHIC;
      descCfg.epibPresent         = Cppi_EPIB_NO_EPIB_PRESENT;
      descCfg.cfg.mono.dataOffset = MONOLITHIC_DESC_DATA_OFFSET;            
      descCfg.returnQueue.qMgr    = QMSS_PARAM_NOT_SPECIFIED;
      descCfg.returnQueue.qNum    = QMSS_PARAM_NOT_SPECIFIED;
      
      /* Initialize the descriptors and push to free Queue */
      freeQueueHdlr = Cppi_initDescriptor (&descCfg, &numAllocated);

The benchmark example code just provides the memory area where the monolithic descriptors will live.  It does not initialize them.  That's why you don't see any monolithic descriptor-specific configuration code there.

Justin

"The qmssIpcBenchmark example benchmarks the IPC QMSS Transport.  The transport instance initialization code configures CPPI to use monolithic descriptors.  It is not done in the benchmark example code." ---> so in simple statement, can i Confirm that the qmssIpcBenchmark is NOT MONOLITHIC one (Though it is using all variable names as MONOLITHIC ) ?

"The benchmark example code just provides the memory area where the monolithic descriptors will live" ---> I ran the complete project on EVM board and i never saw any Monolithic descriptor in that memory region [i observed this at many stages of the code run].can you please comment on this [as you told even the transport instance initalization is missing, how it is monolithic then ?]

I put another question, how descriptor is defaulted to Host type [can you point me to the line of code which makes it Host type] ? and also please comment on how to convert the example code to monlithic type ? [is it through cppi_InitDescriptor(), as i have put above] ? 

Thanks

RC Reddy

RC,

The qmssIpcBenchmark uses monolithic descriptors or at least it should.  The IPC QMSS transport initialization code DOES get run.  It is run when Ipc_attach() is called.

I'll run the example and double check the descriptor types.  I'll also look to see if the descriptors are for some reason being defaulted to type Host.

Justin

Hi Justin, 

                  when i say Monolithic, i am standing by Ti statement from its documents

==================latest multicore navigator user guide [acrobat pdf page number24/151]======================

Monolithic Packet
Monolithic packet descriptors differ from host packet descriptors in that the descriptor area also contains the payload data, whereas the host packet contains a pointer to buffer located elsewhere. Monolithic packets are simpler to deal with, but are not as flexible as host packets.

so no question of Pointer logic/stuff there, Monolithic packet has to CARRY THE DATA PACKET AS A WHOLE.

Thanks

RC Reddy

qmssIpcBenchmark does IPC_attach() in the code, so i assume that IPC QMSS transport initialization is done during IPC_attach() ["The IPC QMSS transport initialization code DOES get run.  It is run when Ipc_attach() is called"]

and

also you told "The transport instance initialization code configures CPPI to use monolithic descriptors.  It is not done in the benchmark example code." 

i am confused by your statements, please clarify on what is happening

Thanks

RC Reddy

RC,

The documentation may say a monolithic descriptor has to carry the data packet as a whole but what's the definition of a "data packet"?  There's no way the QM/CPPI hardware subsystem can enforce monolithic descriptors carrying only entire data packets if there's no way to define a "data packet".  In some cases a data packet may be thousands of bytes, in other cases, only a few.  It just so happens in this case it's only a few and those few bytes make up a pointer.  What does the QM/CPPI hardware care?  It doesn't.

You're questioning of why host packets aren't used is warranted given they're designed to carry pointers to data buffers.  Host descriptors are designed for use with the PKTDMA engines which we aren't using in this case.  What I'm pretty sure happened was we designed the transport to use the monolithic descriptor, copying the entire MessageQ message into the descriptor.  When it came time to optimize the transport we realized that the memcpy's into the descriptor on the send side and out of the descriptor on the receive side were useless and needlessly consuming a lot of cycles.  The MessageQ message was already in MSMCSRAM, making it visible to all cores already.  So we cut out the memcpy's and just copied a pointer to the MessageQ message into the monolithic descriptor.  We could have switched the descriptor type to Host but the infrastructure was already there for monolithic.

In the end what the transport implements is a transport using Monolithic descriptors in a Host descriptor fashion.  We're not trying to push the monolithic descriptor through the PKTDMAs so QM/CPPI doesn't care and isn't complaining.

Justin

At the start of the example QMSS and CPPI are initialized (via qmss_init and cppi_init, respectively).  Then a chunk of memory is provided to the Qmss_insertMemoryRegion function for the descriptors that will be initialized and used by the IPC QMSS transport.  When the init code is complete Ipc_attach() is called.  Ipc_attach() calls the IPC QMSS transport initialization code.  The IPC QMSS transport initialization code that gets called is located in the TransportQmssSetup.c and TransportQmss.c files.

Ipc_attach() will invoke TransportQmssSetup_attach() which calls TransportQmss_create().

TransportQmss_create() invokes TransportQmss_Instance_init().

TransportQmss_Instance_init() is the "transport instance initialization code" i referred to before.  This code allocates queues, initializes the descriptors as monolithic, and sets up the interrupt mechanisms between the cores.  The monolithic descriptors that were initialized will be located in the memory block provided to Qmss_insertMemoryRegion prior to calling Ipc_attach().

Justin

Hi Justin,

"The documentation may say a monolithic descriptor has to carry the data packet as a
whole but what's the definition of a "data packet"?
There's no way the QM/CPPI hardware subsystem can enforce monolithic
descriptors carrying only entire data packets if there's no way to define a "data packet".
In some cases a data packet may be thousands of bytes, in other cases, only a few." 

please refer these below pages for definition of data packet

1. please google for "KeyStone-MC-Navigator.pdf" and in that see page number 14/21 [Figure which is self explanatory]

and/or view the slides at 

http://learningmedia.ti.com/public/hpmp/KeyStone/04_Navigator/index.html

bottom line, Ti DOES mention about PACKET being part of descriptor Area [in MONOLITHIC]

2. For your question "In some cases a data packet may be thousands of bytes,"

From Keystone Architecture multicore navigator

[From UserGuide]Question: What happens when no configured RX packet queue has a packet that is large
enough to hold the data?
It depends. For monolithic packet mode, the Rx DMA assumes the descriptor is big
enough, and will overwrite adjacent memory if it isn't. 

[From UserGuide]The maximum size of a monolithic packet descriptor is 65535 bytes. Of this, monolithic
packet descriptors always contain 12 bytes of required information and may also
contain 16 bytes of software-specific tagging information and up to 128 bytes
(indicated in 4-byte increments) of protocol-specific information

3. "When it came time to optimize the transport we realized that the memcpy's into the descriptor on the send side and out of the descriptor on the receive side were useless and needlessly consuming a lot of cycles.  The MessageQ message was already in MSMCSRAM, making it visible to all cores already.  So we cut out the memcpy's and just copied a pointer to the MessageQ message into the monolithic descriptor.  We could have switched the descriptor type to Host but the infrastructure was already there for monolithic."

I understand and at the same time this contradicts the very definition of Monolithic [if you are not copying the data and just copying the pointer to the data buffer] and also can you please point me to the code where you are bypassing copying of data and instead just copying the pointer . [please point me to both code sections].

4. "In the end what the transport implements is a transport using Monolithic descriptors in a Host descriptor fashion."

so I suggest please specify this clearly in all Ti documents [particularly in benchmarking example http://processors.wiki.ti.com/index.php/BIOS_MCSDK_2.0_User_Guide#IPC_Benchmarks] and I can conclude that it is NEITHER FULLY HOST descriptor NOR FULLY MONOLITHIC descriptor.

Thanks

RC Reddy

Thanks and now i understood the process.

1. If i call Ipc_attach(), by default all the descriptors will be MONOLITHIC and will reside in the memory region inserted. [by the way, why dont Ti provide this information in documents or as part of code comments].

2. Is there anyway to make them Host Type [i mean not tweaking into TransportQmss_Instance_init code, but later through cppi_initdescriptor() or any other API].

Thanks

RC Reddy

So from your post

" What does the QM/CPPI hardware care?"

and

"We're not trying to push the monolithic descriptor through the PKTDMAs so QM/CPPI doesn't care and isn't complaining"

So, There is no hardware [QMSS/PKTDMA] involvement here at all and also this transport is also Kind/Blend of Shared memory sort of concept ?? can you please clarify ??

Thanks

RC Reddy

RC Reddy,

QMSS and PKTDMA are still used, on MessageQ_Put we are still pushing that monolithic descriptor (which has pointer as a payload) to the queue. It is just we are making use of knowledge of payload which is there in shared memory and avoiding copying that payload.

About using Host descriptors, for MessageQ. i will check and get back to you soon.

Regards,

Bhavin

Hi Bhavin,

                 Thanks for reply and clearing my doubt ["QMSS and PKTDMA are still used, on MessageQ_Put we are still pushing that monolithic descriptor (which has pointer as a payload) to the queue"]. Can you please depict each step/stage/sequence in the code flow..when gatemp is used,listmp is used,semphore post/pend,queue push, queue pop, memory alloc, nameserver propogation [This will help me very much in understanding the flow].

Thanks

RC Reddy

RC,

The PKTDMA is not used.  The QMSS IPC transport configures QM in such a manner that hardware does not touch anything in the descriptor.  As a result, the only content within the descriptor that matters is the data length and the data pointer.  It does not matter whether the remaining descriptor content is Host or Monolithic because the interpreter of the descriptor content is not the PKTDMA hardware but instead the QMSS IPC transport software, TransportQmss_put and TransportQmss_xxx_isr, to be exact.  All fields in the Monolithic data structure could have been wiped and replaced with a length and pointer field only interpretable by the IPC transport code since the PKTDMA hardware is not used and does not touch the descriptor . 

Since the PKTDMA is not used the cppi_initDescriptor API does not really need to be invoked.  However, the API was used as a simple way to carve a memory region into descriptors and pushed them into a queue. 

Similarly, monolithic descriptors are configured and their fields used but this was only for convenience.  Instead of inventing new code which inserted a data packet length and data pointer into the first two words of the descriptor the monolithic descriptor set/get APIs provided by CPPI were reused.

As for IPC setup and transport code flow please refer to the MCSDK 2.0 User Guide section on IPC control flow

http://processors.wiki.ti.com/index.php/BIOS_MCSDK_2.0_User_Guide#IPC_Flow

Anything past that is documented by the source code itself.  If you want to gain a better understanding of BIOS, IPC, and QMSS transport operation try stepping through the transport examples.  Prior to stepping you'll need to rebuild the projects with the BIOS and IPC debug libraries.  It is fairly easy to do this.

1. Import an IpcBenchmark example into CCS.

2. Open the example's .cfg for editing.

3. After the BIOS module is loaded add the highlighted line below

var BIOS = xdc.useModule('ti.sysbios.BIOS');

BIOS.libType = BIOS.LibType_Debug;
BIOS.clockEnabled = false;
BIOS.heapSize = 0x8000;

*Please note that setting the BIOS library to debug in the .cfg file will also set the IPC library to debug.

4. Delete the Debug and Release directories in the project

5. Rebuild

Justin

Hi Justin,

                Thanks for answer/reply. I still am doubtfull regarding the statement you put "PKTDMA is not being used"

According to my understanding, in the TransportQmss.c, in the function TransportQmss_put, it contains Qmss_queuePushDescSize and this would be surely pushing the data out through hardware and the data would be PKTDMA'ed or loopedBack (if it is infrastructure queue). please confirm/correct my understanding here ?

/*
* ======== TransportQmss_put ========
* Routine used to send packets via QMSS driver
*/
Bool TransportQmss_put(TransportQmss_Object *obj, Ptr msg)

/* Push descriptor to Rx queue of remote core */
Qmss_queuePushDescSize (rxQueueHdlr, (UInt32 *) monoDescPtr, TransportQmss_descriptorSize);

I went through your mail once, let me understand in depth [about what you are telling] and will do the experiment/debug you suggested.

Thanks

RC Reddy

RCReddy:

The packet DMA is only used when descriptors are pushed onto "special" transmit queues that are associated with hardware that has a packet DMA.  You can identify these special queues based on the call to Qmss_queueOpen's Qmss_QueueType queType argument.  The Qmss_QueueType (defined in csl_qm_queue.h) that do NOT associate to packet DMAs are:

• Qmss_QueueType_LOW_PRIORITY_QUEUE
• Qmss_QueueType_INTC_QUEUE
• Qmss_QueueType_HIGH_PRIORITY_QUEUE
• Qmss_QueueType_STARVATION_COUNTER_QUEUE
• Qmss_QueueType_TRAFFIC_SHAPING_QUEUE
• Qmss_QueueType_GENERAL_PURPOSE_QUEUE. 

In the IPC QMSS transport (ipc/qmss/transports/TransportQmss.c), only Qmss_QueueType_INTC_QUEUE and Qmss_QueueType_HIGH_PRIORITY_QUEUE are used which are NOT associated with packet DMAs, so therefore the IPC QMSS transport does NOT use packet DMAs.