I have been investigating a situation in the Appro software where one of my video tasks (analytics) does not do a transform, it just reads the video buffer pointer and then passes it along the chain.
In the usual operation:
- The video tasks use the API in
avserver.cto ask the previous video task for a full buffer pointer and asks the next task for an empty buffer pointer. - It then processes the full buffer pointer data into the empty buffer pointer.
- It then passes the emptied full buffer pointer back to the previous video task as an empty buffer pointer and passes the filled empty buffer pointer to the next task as a full buffer pointer.
I now have a video task that:
- Asks the previous task for a full buffer pointer and asks the next task for an empty buffer pointer.
- Processes the data in the full buffer pointer
- Swaps the full buffer pointer with the empty buffer pointer
- It then passes the empty buffer pointer from the next task back to the previous task as an empty buffer pointer and passes the full buffer pointer from the previous task to the next task as a full buffer pointer.
I have implemented this functionality, however, had very curious streaming problems when the pass through video task was after the capture task.
After much debugging I found out that the capture task does not feed buffer pointers into the driver to fill. The way the capture task works is like so:
- Ask driver for a full buffer pointer which is basically an index into an array of pointers.
- Passes this full buffer on to the next task
- Asks the next task for an empty buffer pointer.
- Doesn't care what this empty buffer pointer is just say that the full buffer pointer that was given to me is now empty.
This is fine when the task after the capture task is a transpose task. The next task will just 'copy' the data from the full buffer pointer provided by the driver into another empty buffer provided by the next task.
The problem occurs when the task after capture is returning a different empty buffer pointer to the one that the driver gave to capture. For example in normal operation with a transpose task after capture:
- Encode asks Capture for a full buffer pointer and Stream for an empty buffer pointer
- Capture asks the IPIPE resize driver for a filled buffer pointer and it returns
0x8B7C6000this is the pointer that is given to Encode. - Stream returns a pointer (
0x8BA98000) to one of it's allocated CMEM buffers that is empty. - Encode then processes the video information in
0x8B7C6000->0x8BA98000 - It then tells Stream that
0x8BA98000is full and Stream starts it's processing. - It then tells Capture that
0x8B7C6000is empty - Capture tells the IPIPE resize driver that the buffer pointer it returned (
0x8B7C6000) is free.
In the situation where there isn't a transpose task after capture:
- Analytics asks Capture for a full buffer pointer and Encode for an empty buffer pointer
- Capture asks the IPIPE resize driver for a filled buffer pointer and it returns
0x8B7C6000this is the pointer that is given to Analytics. - Stream returns a pointer (
0x8BA98000) to one of it's allocated CMEM buffers that is empty. - Analytics then processes the video information in 0x8B7C6000
- Analytics then swaps the pointers (
0x8B7C6000<->0x8BA98000) so that it can pass on the video information unmodified. - It then tells Encode that
0x8B7C6000is full and Encode starts it's processing. - It then tells Capture that
0x8BA98000is empty - Capture tells the IPIPE resize driver that the buffer pointer it returned
(
0x8B7C6000) is free. - The driver is now filling a buffer that has been passed down the chain.
I have been looking at the driver code in the Appro framework. I have noticed that the drv_ipipe.c allocates some contiguous memory:
gDRV_ipipeObj.bufInfoRsz[rszId][i].virtAddr = OSA_cmemAlloc(bufSize, 32);
gDRV_ipipeObj.bufInfoRsz[rszId][i].physAddr = OSA_cmemGetPhysAddr(gDRV_ipipeObj.bufInfoRsz[rszId][i].virtAddr);
These pointers are then passed to the CSL layer in the same file:
for(k=0; k<bufInit[i].numBuf; k++) {
bufInit[i].bufAddr[k] = (Uint32)gDRV_ipipeObj.bufInfoRsz[i][k].physAddr + gDRV_ipipeObj.config.rszValidDataStartOffset;
}
[...snip...]
status = CSL_rszBufInit(&gCSL_rszHndl, i, &bufInit[i], &bufConfig[i]);
The CSL then takes these pointers and works out where to dump video information in:
CSL_Status CSL_rszBufInit(CSL_RszHandle hndl, Uint8 rszMod, CSL_BufInit * bufInit, CSL_RszBufConfig *bufConfig)
{
CSL_Status status;
Uint8 *pAddrY, *pAddrC;
if (rszMod >= CSL_RSZ_CH_MAX)
return CSL_EFAIL;;
status = CSL_bufInit(&hndl->outBuf[rszMod], bufInit);
hndl->yuv420BufCoffset[rszMod] = bufConfig->offsetH*bufConfig->offsetV;
if(bufConfig->format==CSL_RSZ_CHROMA_FORMAT_422) {
hndl->flipVOffsetY[rszMod] = bufConfig->offsetH*2*(bufConfig->height-1);
hndl->flipHOffsetYC[rszMod] = bufConfig->width*2-1;
} else {
hndl->flipVOffsetY[rszMod] = bufConfig->offsetH*(bufConfig->height-1);
hndl->flipHOffsetYC[rszMod] = bufConfig->width-1;
}
hndl->flipVOffsetC[rszMod] = bufConfig->offsetH * (bufConfig->height/2-1);
hndl->curBufInfo[rszMod].id = CSL_BUF_ID_INVALID;
pAddrY = (Uint8 *) bufInit->bufAddr[0];
pAddrC = (Uint8 *)( bufInit->bufAddr[0] + hndl->yuv420BufCoffset[rszMod]);
if(hndl->flipV[rszMod]) {
pAddrY += hndl->flipVOffsetY[rszMod];
pAddrC += hndl->flipVOffsetC[rszMod];
}
if(hndl->flipH[rszMod]) {
pAddrY += hndl->flipHOffsetYC[rszMod];
pAddrC += hndl->flipHOffsetYC[rszMod];
}
CSL_rszSetOutAddr(hndl, rszMod, pAddrY, pAddrC);
return status;
}
These configured pointers are changed in CSL_rszSetOutAddr on an interupt in csl_ccdcInt.c.
It seems the drivers were written with no runtime support to change the output buffers, which is exactly what I need. I feel it has been a huge oversight that the driver doesn't care what free buffer pointer is being passed back in videoCaptureThr.c.
So what I need is to able reconfigure the output pointers of the IPIPE at runtime. I want to add an API call so that the pointer offsets calculated above are done for each frame.
I was not there when the drivers were designed and also have never written a linux kernel driver before. I cannot comment on design decisions made when these were written.
Could anyone from TI or Appro guide me in modifing the driver in a sensible way so that it has the ability to support runtime buffer changes. There are so many layers in the driver I do not know what I will be breaking when I modify the code.
Thanks,
Matt
