Other Parts Discussed in Thread: TVP7002
I have been trying for a frustratingly long time to get simple H.264 encoding of a D1 composite capture working on the DM365EVM. I was expecting this to essentially work out of the box, but I'm clearly doing something wrong.
I'm using the Arago 2.6.37 git kernel, linux-davinci branch, updated just a few days ago, with the default configuration for the DM365EVM. It compiles and boots fine.
I'm using TI's H.264 2.30.00.04 production codec. That part works fine; I can encode the sample file.
Originally, I tried to reuse my VPFE capture code that I had for DM6446... which didn't work because that code was compiled against a 2.6.28 kernel that didn't use the Media Controller API. I updated the code to use the MC API, only to find out the encoder only accepts YUV420SP frames, not the UYVY frames from the CCDC capture node that I had been using.
After searching around, it seemed I needed to use the DaVinci Resizer hardware to convert the UYVY frames to the NV12 that the encoder expects. So, I change my device chain to look like this:
tvp514x ---> DAVINCI CCDC ---> DAVINCI PREVIEWER ---> DAVINCI RESIZER ---> DAVINCI VIDEO RSZ output
All of the pads are formatted as 720x480, V4L2_MBUS_FMT_YUYV8_2x8, although I attempted to set the pixel format to V4L2_MBUS_FMT_NV12_1x10 only to have the
driver return V4L2_MBUS_FMT_YUYV8_2x8, anyways. VIDIOC_S_FMT on the resizer output successfully set the format to NV12, though, so I thought it might still work.
Firing up the application, however, saw my capture thread going into the VIDIOC_DQBUF ioctl for the resizer output and never returning...
Digging around, I found the mt9p031_ccdc_prv_rsz_loopback_mmap.c example app in the PSP. Not exactly the same chain, but I was mostly concerned about the "prv, rsz, and mmap" elements, since those were the parts I couldn't get working. This example program set up a few more IOCTLs (for which the only documentation I could find was related to the 2.6.10 kernel version of the drivers... a bit out of date), so I figured I was missing yet more configuration. I attempted to adapt the code to configure my situation, but still my capture thread hangs in DQBUF.
Here is the code to set the whole thing up (note: it's a C++ app, the mediaDevice_ object has an overloaded [] operator to give me its entities by name and the >> operator handles linking magic... trying to help out readability for whoever inherits this code after I jump off a cliff from frustration :) ):
mediaDevice_["DAVINCI VIDEO CCDC output"];
mediaDevice_["tvp514x"] >>mediaDevice_["DAVINCI CCDC"] >> mediaDevice_["DAVINCI PREVIEWER"] >> mediaDevice_["DAVINCI RESIZER"] >> mediaDevice_["DAVINCI VIDEO RSZ output"];
char subdevice0Name[] = "/dev/v4l-subdev0";
AO_MESSAGE << "Attempting to open " << subdevice0Name << "..." << endl;
mediaDevice_["tvp514x"].SetSubDevice(subdevice0Name);
v4l2_mbus_framefmt frameFormat;
memset(&frameFormat, 0, sizeof(frameFormat));
frameFormat.width = 720;
frameFormat.height = 480;
frameFormat.code = V4L2_MBUS_FMT_YUYV8_2X8;
frameFormat.field = V4L2_FIELD_INTERLACED;
frameFormat.colorspace = V4L2_COLORSPACE_SMPTE170M;
mediaDevice_["tvp514x"].SetMediaBusFormat(frameFormat);
char subdevice1Name[] = "/dev/v4l-subdev2";
AO_MESSAGE << "Attempting to open " << subdevice1Name << "..." << endl;
mediaDevice_["DAVINCI CCDC"].SetSubDevice(subdevice1Name);
mediaDevice_["DAVINCI CCDC"].SetMediaBusFormat(frameFormat);
char subdevice2Name[] = "/dev/v4l-subdev3";
AO_MESSAGE << "Attempting to open " << subdevice2Name << "..." << endl;
mediaDevice_["DAVINCI PREVIEWER"].SetSubDevice(subdevice2Name);
//set default previewer config... I think this might only be neccessary to get the default continuous configs in the next step
prev_channel_config previewerChannelConfig;
previewerChannelConfig.len = 0;
previewerChannelConfig.config = 0;
AO_MESSAGE << "Attempting to set previewer default configuration..." << endl;
if(0 > mediaDevice_["DAVINCI PREVIEWER"].SubDeviceIOControl(PREV_S_CONFIG, &previewerChannelConfig))
{
AO_ERROR << "Can't set previewer default configuration:" << strerror(errno);
//throw std::runtime_error("Can't set previewer default configuration");
abort();
}
//get default continuous config
prev_continuous_config previewerContinuousConfig;
bzero(&previewerContinuousConfig, sizeof(prev_continuous_config));
previewerChannelConfig.len = sizeof(prev_continuous_config);
previewerChannelConfig.config = &previewerContinuousConfig;
AO_MESSAGE << "Attempting to get previewer default configuration for continuous mode..." << endl;
if(0 > mediaDevice_["DAVINCI PREVIEWER"].SubDeviceIOControl(PREV_G_CONFIG, &previewerChannelConfig))
{
AO_ERROR << "Can't get previewer default configuration for continuous mode:" << strerror(errno);
//throw std::runtime_error("Can't set previewer default configuration");
abort();
}
//now set the continuous mode parameters we need...
//for some reason we need to configure these fields again?!
previewerChannelConfig.len = sizeof(prev_continuous_config);
previewerContinuousConfig.bypass = IPIPE_BYPASS_ON;
previewerChannelConfig.config = &previewerContinuousConfig;
AO_MESSAGE << "Attempting to set previewer final configuration..." << endl;
if(0 > mediaDevice_["DAVINCI PREVIEWER"].SubDeviceIOControl(PREV_S_CONFIG, &previewerChannelConfig))
{
AO_ERROR << "Can't set previewer final configuration:" << strerror(errno);
//throw std::runtime_error("Can't set previewer default configuration");
abort();
}
//FIXME I have no idea what the hell the point of this next section is
prev_cap previewerCapabilities;
previewerCapabilities.index = 0;
prev_module_param previewerModuleParameters;
while(1)
{
if( 0 > mediaDevice_["DAVINCI PREVIEWER"].SubDeviceIOControl(PREV_ENUM_CAP, &previewerCapabilities) )
{
break;
}
strcpy(previewerModuleParameters.version, previewerCapabilities.version);
previewerModuleParameters.module_id = previewerCapabilities.module_id;
AO_MESSAGE << "Setting default for previewer module " << previewerCapabilities.module_name << endl;
previewerModuleParameters.param = 0;
if( 0 > mediaDevice_["DAVINCI PREVIEWER"].SubDeviceIOControl(PREV_S_PARAM, &previewerModuleParameters) )
{
AO_ERROR << "Can't set default parameters for module " << previewerCapabilities.module_name << endl;
abort();
}
++previewerCapabilities.index;
}
mediaDevice_["DAVINCI PREVIEWER"].SetMediaBusFormat(frameFormat);
char subdevice3Name[] = "/dev/v4l-subdev4";
AO_MESSAGE << "Attempting to open " << subdevice3Name << "..." << endl;
mediaDevice_["DAVINCI RESIZER"].SetSubDevice(subdevice3Name);
//set default resizer config... I think this might only be neccessary to get the default continuous configs in the next step
rsz_channel_config resizerChannelConfig;
resizerChannelConfig.chain = 1; //take input from previewer
resizerChannelConfig.len = 0;
resizerChannelConfig.config = 0;
AO_MESSAGE << "Attempting to set resizer default configuration..." << endl;
if(0 > mediaDevice_["DAVINCI RESIZER"].SubDeviceIOControl(RSZ_S_CONFIG, &resizerChannelConfig))
{
AO_ERROR << "Can't set resizer default configuration:" << strerror(errno);
//throw std::runtime_error("Can't set resizer default configuration");
abort();
}
//get default continuous config
rsz_continuous_config resizerContinuousConfig;
bzero(&resizerContinuousConfig, sizeof(rsz_continuous_config));
resizerChannelConfig.chain = 1; //take input from previewer
resizerChannelConfig.len = sizeof(rsz_continuous_config);
resizerChannelConfig.config = &resizerContinuousConfig;
AO_MESSAGE << "Attempting to get resizer default configuration for continuous mode..." << endl;
if(0 > mediaDevice_["DAVINCI RESIZER"].SubDeviceIOControl(RSZ_G_CONFIG, &resizerChannelConfig))
{
AO_ERROR << "Can't get resizer default configuration for continuous mode:" << strerror(errno);
//throw std::runtime_error("Can't set resizer default configuration");
abort();
}
//now set the continuous mode parameters we need...
resizerContinuousConfig.output1.enable = 1; //output1 doesn't support changing the pixel format... WTF
resizerContinuousConfig.output2.enable = 1;
resizerContinuousConfig.output2.width = captureWidth_;
resizerContinuousConfig.output2.height = captureHeight_;
resizerContinuousConfig.output2.pix_fmt = IPIPE_YUV420SP;
//resizerContinuousConfig.output1.pix_fmt = IPIPE_YUV420SP;
//for some reason we need to configure these fields again?!
resizerChannelConfig.chain = 1; //take input from previewer
resizerChannelConfig.len = sizeof(rsz_continuous_config);
resizerChannelConfig.config = &resizerContinuousConfig;
AO_MESSAGE << "Attempting to set resizer final configuration..." << endl;
if(0 > mediaDevice_["DAVINCI RESIZER"].SubDeviceIOControl(RSZ_S_CONFIG, &resizerChannelConfig))
{
AO_ERROR << "Can't set resizer final configuration:" << strerror(errno);
//throw std::runtime_error("Can't set resizer default configuration");
abort();
}
frameFormat.code = V4L2_MBUS_FMT_NV12_1X20;
//N.B. this is only setting the output pad format... trying to set the input pad format results in "Operation Not Permitted" and general badness...
mediaDevice_["DAVINCI RESIZER"].SetMediaBusOutputFormat(frameFormat);
//deviceName is set outside this function to /dev/video4, the resizer output
if(-1 == (fd_ = open(deviceName, O_RDWR, 0)))
{
AO_ERROR << "cannot open " << deviceName << ": " << strerror(errno) << endl;
return;
}
//input is 0
if(-1 == ioctl(fd_, VIDIOC_S_INPUT, &input_))
{
AO_ERROR << "VIDIOC_S_INPUT failed for input " << input_ << ": " << strerror(errno) << endl;
return;
}
//sanity checks on device
if(-1 == ioctl(fd_, VIDIOC_QUERYCAP, &v4l2Capability_))
{
AO_ERROR << "VIDIOC_QUERYCAP failed: " << strerror(errno) << endl;
return;
}
if(!(v4l2Capability_.capabilities & V4L2_CAP_VIDEO_CAPTURE))
{
AO_ERROR << "doesn't support capture" << endl;
return;
}
if(!(v4l2Capability_.capabilities & V4L2_CAP_STREAMING))
{
AO_ERROR << "doesn't support streaming" << endl;
return;
}
v4l2VideoStandard_ = V4L2_STD_NTSC;
if(-1 == ioctl(fd_, VIDIOC_S_STD, &v4l2VideoStandard_))
{
AO_ERROR << "VIDIOC_S_STD to " << v4l2VideoStandard_ << " failed: " << strerror(errno) << endl;
return;
}
Zero(v4l2Format_);
v4l2Format_.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
v4l2Format_.fmt.pix.width = captureWidth_;
v4l2Format_.fmt.pix.height = captureHeight_;
v4l2Format_.fmt.pix.pixelformat = V4L2_PIX_FMT_NV12;
v4l2Format_.fmt.pix.field = V4L2_FIELD_INTERLACED; //should this be NONE?
char fourcc[5] = {0, 0, 0, 0, 0};
AO_MESSAGE << "Attempting to set capture format:" << endl;
AO_MESSAGE << "\ttype:" << v4l2Format_.type << endl;
AO_MESSAGE << "\twidth:" << v4l2Format_.fmt.pix.width << endl;
AO_MESSAGE << "\theight:" << v4l2Format_.fmt.pix.height << endl;
memcpy(fourcc, &v4l2Format_.fmt.pix.pixelformat, 4);
AO_MESSAGE << "\tpixel format:" << fourcc << endl;
AO_MESSAGE << "\tfield:" << v4l2Format_.fmt.pix.field << endl;
if(-1 == ioctl(fd_, VIDIOC_S_FMT, &v4l2Format_))
{
AO_ERROR << "VIDIOC_S_FMT failed: " << strerror(errno) << endl;
return;
}
AO_MESSAGE << "Reading back capture format:" << endl;
AO_MESSAGE << "\ttype:" << v4l2Format_.type << endl;
AO_MESSAGE << "\twidth:" << v4l2Format_.fmt.pix.width << endl;
AO_MESSAGE << "\theight:" << v4l2Format_.fmt.pix.height << endl;
AO_MESSAGE << "\tbytesperline:" << v4l2Format_.fmt.pix.bytesperline << endl;
memcpy(fourcc, &v4l2Format_.fmt.pix.pixelformat, 4);
AO_MESSAGE << "\tpixel format:" << fourcc << endl;
AO_MESSAGE << "\tfield:" << v4l2Format_.fmt.pix.field << endl;
//numBuffersRequested is 3
AO_MESSAGE << "Requesting " << numBuffersRequested_ << " capture buffers..." << endl;
Zero(v4l2RequestBuffers_);
v4l2RequestBuffers_.count = numBuffersRequested_;
v4l2RequestBuffers_.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
v4l2RequestBuffers_.memory = V4L2_MEMORY_MMAP;
if(-1 == ioctl(fd_, VIDIOC_REQBUFS, &v4l2RequestBuffers_))
{
AO_ERROR << "VIDIOC_REQBUFS failed: " << strerror(errno) << endl;
return;
}
AO_MESSAGE << "Number of buffers granted:" << v4l2RequestBuffers_.count << endl;
if(v4l2RequestBuffers_.count < numBuffersRequested_)
{
AO_ERROR << "too few buffers available: " << v4l2RequestBuffers_.count << "<" << numBuffersRequested_ << endl;
return;
}
videoBuffers_.resize(v4l2RequestBuffers_.count);
v4l2_buffer* v4l2BuffersTemp = new v4l2_buffer[v4l2RequestBuffers_.count];
for(unsigned int bufferIndex = 0; bufferIndex < v4l2RequestBuffers_.count; bufferIndex++)
{
v4l2_buffer& buffer = v4l2BuffersTemp[bufferIndex];
Zero(buffer);
buffer.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buffer.memory = V4L2_MEMORY_MMAP;
buffer.index = bufferIndex;
if(-1 == ioctl(fd_, VIDIOC_QUERYBUF, &buffer))
{
AO_ERROR << "VIDIOC_QUERYBUF failed: " << strerror(errno) << endl;
return;
}
AO_MESSAGE << "VIDIOC_QUERYBUF for buffer " << buffer.index << endl;
AO_MESSAGE << "\ttype:" << buffer.type << endl;
AO_MESSAGE << "\tbytesused:" << buffer.bytesused << endl;
AO_MESSAGE << "\tflags:" << buffer.flags << endl;
AO_MESSAGE << "\tfield:" << buffer.field << endl;
AO_MESSAGE << "\tmemory:" << buffer.memory << endl;
AO_MESSAGE << "\tlength:" << buffer.length << endl;
AO_MESSAGE << "\toffset:" << buffer.m.offset << endl;
videoBuffers_[bufferIndex].SetSize(buffer.length);
videoBuffers_[bufferIndex].SetStart( (int)mmap((void *)0,
buffer.length,
PROT_READ | PROT_WRITE,
MAP_SHARED,
fd_,
buffer.m.offset) );
if((int)MAP_FAILED == videoBuffers_[bufferIndex].GetStart())
{
AO_ERROR << "memory map failed for buffer " << bufferIndex << ": " << strerror(errno) << endl;
return;
}
if(-1 == ioctl(fd_, VIDIOC_QBUF, &buffer))
{
AO_ERROR << "VIDIOC_QBUF failed for buffer: " << bufferIndex << ":" << strerror(errno) << endl;
return;
}
videoBuffers_[bufferIndex].SetRelease(boost::bind(&VPFECapture::ReleaseCaptureBuffer,this,buffer));
}
Another function calls the STREAMON ioctl, but when I go to DQBUF, that call never returns. I don't have this problem with the "DAVINCI VIDEO CCDC output", but that doesn't give me the right format for the encoder.
What am I doing wrong? Is this generally the correct method to do D1 composite capture and H.264 encode using TI's encoder on the DM365, or am I off the mark?
If the resizer has two outputs associated with the RSZ_S_CONFIG, which one ends up at /dev/video4? And, if it's output1, how do I configure it to be YUV420SP (there's no pix_fmt field for output1...)?
Thanks in advance for any help...