I am debugging our C6678 code running on our DSP board. I have narrowed down the problem which looks like that GateMP protection of shared resource failed. I could not understand what my code has been doing wrong, so would be very grateful for any advises.
The gateMP is created by core0 (calling CGateMP::Create()) and opened (calling CGateMP::Open()) by the rest of cores. The code doing this is shown at the end of this posting. IPC is started with Ipc.ProcSync_ALL. All gateMP calls are done in task contexts. Core0 has two tasks that use gateMP, while each of the other cores has only one task that uses the gateMP. The code timestamps just after entering the gateMp and before leaving gateMP and it generated the following trace messages:
[29006][00000161] core0 - CJobQueue::DoneSlcJob() seqNum(177), fmt(1d98), jobPtr(0c1035c0), size(1, 1, 1), ts(683114b4, 683115e2)
[29006][00000162] core0 - ::OnLocalFrameJobDone() seqNum(177, 175), jobPtr(0c1035c0, 0c1035bc), size(0), ts(68312612, 6831269b)
[29006][00000163] core5 - CJobQueue::DoneSlcJob() seqNum(176), fmt(1792), jobPtr(0c1035c4), size(2, 0, 0), ts(683125f8, 6831271e)
The number in 1st square bracket is timestamp (in millisecond) of the trace, while that of the 2nd is the sequence number of trace of the C6678. The pair of values at the end of trace messages show the timestamps said above. The gateMP protects the job completion queue. The 1st line and 3rd lines are generated by the following code:
void CJobQueue::DoneSlcJob(CFrameJob* job, int sliceId, char* dstAddr, int dataSize, unsigned procMs)
{
// some code omitted
....
if(0 == numUncompSlices)
{
m_completedJobQueueLockable.Lock();
unsigned ts0 = GetTimestamp32();
CacheInv(m_completedJobQueue);
CFrameJob** jobPtr = m_completedJobQueue->alloc_back();
int queueSize0 = m_completedJobQueue->size();
CacheWbInv(m_completedJobQueue);
int queueSize1 = m_completedJobQueue->size();
CacheInv(jobPtr);
job->SetDstAddr(dstAddr);
*jobPtr = job;
CacheWbInv(jobPtr);
int queueSize2 = m_completedJobQueue->size();
unsigned ts1 = GetTimestamp32();
m_completedJobQueueLockable.Unlock();
if(queueSize0 != queueSize1 || queueSize0 != queueSize2)
{
TraceErr(TU_GENERAL, "CJobQueue::DoneSlcJob() seqNum(%d), fmt(%04x), jobPtr(%08x), size(%d, %d, %d), ts(%08x, %08x)\n",
job->GetSeqNum(), job->GetFmt(), jobPtr, queueSize0, queueSize1, queueSize2, ts0, ts1);
}
else
{
TraceInfo(TU_GENERAL, "CJobQueue::DoneSlcJob() seqNum(%d), fmt(%04x), jobPtr(%08x), size(%d, %d, %d), ts(%08x, %08x)\n",
job->GetSeqNum(), job->GetFmt(), jobPtr, queueSize0, queueSize1, queueSize2, ts0, ts1);
}
// some code omitted
....
CpIntc_postSysInt(0, SYS_INT_LOCAL_FRAME_JOB_DONE);
}
}
The above code (which can be called by any cores) adds a completed job to the completion job queue and then posts a system interrupt for the core0 to serialize the transfer of encoded/decoded video from DDR3 to PCIe as below:
void CPCIeTaskScheduler::OnLocalFrameJobDone()
{
while(Semaphore_pend(m_semaJobDone, BIOS_WAIT_FOREVER))
{
CFrameJob* frameJob;
m_completedJobQueueLockable.Lock();
unsigned ts0 = GetTimestamp32();
CacheInv(m_completedJobQueue);
CFrameJob** jobPtr = m_completedJobQueue->remove_front();
int queueSize = m_completedJobQueue->size();
CacheWbInv(m_completedJobQueue);
unsigned ts1 = GetTimestamp32();
m_completedJobQueueLockable.Unlock();
CacheInv<CFrameJob*>(jobPtr);
frameJob = *jobPtr;
CacheInv<CFrameJob>(frameJob);
unsigned seqNum = frameJob->GetSeqNum();
TraceInfo(TU_GENERAL, "::OnLocalFrameJobDone() seqNum(%d, %d), jobPtr(%08x, %08x), size(%d), ts(%08x, %08x)\n",
seqNum, lastSeqNumLocal[frameJob->GetSessionId() != 0], jobPtr, lastLocalJobPtr[frameJob->GetSessionId() != 0], queueSize, ts0, ts1);
lastSeqNumLocal[frameJob->GetSessionId() != 0] = seqNum;
lastLocalJob[frameJob->GetSessionId() != 0] = frameJob;
lastLocalJobPtr[frameJob->GetSessionId() != 0] = jobPtr;
// some code omitted
....
}
}
The 3 line trace messages show that the core5 already entered gateMP but before leveing it core0 entered the same gateMP and left it before core5. I could not understand how this could happen. Our code does not use any hardware semaphore directly nor via CSL. The completion queue aligned to 128 bytes is created in shared memory by core0 and cacheWbInv its content before creating the gateMp, while the other cores CacheInv the completion queue after opening the gateMP. The gateMP's name "complJobQueueL" is also unique in our code.
I would very grateful for being pointed out possible reasons of this gateMp protection failure.
Dongning
--------- Code create and open gateMP ------
class CGateMP
{
enum eOwnerShip
{
OS_INVALID,
OS_CREATED,
OS_OPENED,
OS_RECEIVED
};
public:
CGateMP() : m_owner(OS_INVALID), m_handle(NULL) {}
CGateMP(GateMP_Handle handle) : m_owner(OS_RECEIVED), m_handle(handle) {}
CGateMP(const String& name, UInt16 regionId) : m_owner(OS_CREATED), m_handle(NULL) {
GateMP_Params params;
GateMP_Params_init(¶ms);
params.localProtect = GateMP_LocalProtect_THREAD;
params.remoteProtect = GateMP_RemoteProtect_SYSTEM;
params.name = name;
params.regionId = regionId;
m_handle = GateMP_create(¶ms);
}
CGateMP(const String& name) : m_owner(OS_OPENED), m_handle(NULL) {GateMP_open(name, &m_handle); }
~CGateMP() {
if(m_handle)
{
if(OS_CREATED == m_owner) GateMP_delete(&m_handle);
if(OS_OPENED == m_owner) GateMP_close(&m_handle);
}
}
void Lock() { m_key = GateMP_enter(m_handle); }
void Unlock() { GateMP_leave(m_handle, m_key); }
void Receive(GateMP_Handle handle) {m_handle = handle; m_owner = OS_RECEIVED; }
int Create(const String& name, UInt16 regionId)
{
m_owner = OS_CREATED;
GateMP_Params params;
GateMP_Params_init(¶ms);
params.localProtect = GateMP_LocalProtect_THREAD;
params.remoteProtect = GateMP_RemoteProtect_SYSTEM;
params.name = name;
params.regionId = regionId;
m_handle = GateMP_create(¶ms);
return m_handle ? CSL_SOK : CSL_ESYS_FAIL;
}
int Open(const String& name)
{
m_owner = OS_OPENED;
int rc = -1;
do {
Task_sleep(1);
rc = GateMP_open(name, &m_handle);
} while(rc < 0);
return rc;
}
private:
CGateMP(const CGateMP&);
CGateMP& operator = (const CGateMP&);
eOwnerShip m_owner;
GateMP_Handle m_handle;
IArg m_key;
};
