This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Compiler/OPT8241-CDK-EVM: steram multiple OPT8241 cameras synchrously.

Part Number: OPT8241-CDK-EVM

Tool/software: TI C/C++ Compiler

Dear All:

We would like to steram multiple OPT8241 cameras synchrously.

The flow is designed as follows.
1. Connect Camera 1 to the computer with USB cable.
2. Start Voxel Viewer and start streaming.
3. Connect Camera 2 to the computer with USB cable.
4. Start another instance of Voxel Viewer and start streaming.

We will need to register call back twice, one for each camera. Each camera should have a different buffer.  (refer: github.com/.../Qt)

First we try to integrate grabber.cpp and grabber.h(in github.com/.../Qt) to my main code.We found that callback function execute only once and can't continue to run again.Why?

Which setting is ignored in main code? What can we do next step? 

Best Regards,

ShengHua

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

The partial code is as follows.


main code:

CameraSystem sys;

// Get all valid detected devices

  • const Vector<DevicePtr> &devices = sys.scan();
  • DevicePtr toConnect;
  •  DepthCameraPtr depthCamera = sys.connect(toConnect);
  • ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  • ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  • Grabber Grabber1(depthCamera, Grabber::FRAMEFLAG_DEPTH_FRAME, sys);
  • Grabber1.start();
  • ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In grabber.cpp

void Grabber::_callback(DepthCamera &depthCamera, const Frame &frame,DepthCamera::FrameType type)
{
Lock<Mutex> _(_mtx); // run only once 

~~~~

else if (type == DepthCamera::FRAME_XYZI_POINT_CLOUD_FRAME)
{
if (_qXYZIFrame.size() >= FIFO_SIZE)
{
XYZIPointCloudFrame *f = _qXYZIFrame.front();
_qXYZIFrame.pop_front();
if (f != NULL)
delete f;
}

XYZIPointCloudFrame *nf = new XYZIPointCloudFrame;
*nf = *dynamic_cast<const XYZIPointCloudFrame *>(&frame);
_qXYZIFrame.push_back(nf);

_frameCount++;
}

}

////////////////////////////////////////////////////////////////////////////////////////////

  • Hello Sheng,

    We have received your inquiry about the OPT8241-CDK-EVM and the device applications team will get back to you shortly.
  • ShengHua

    As you indicated, each Grabber has its own buffer and loop. You will have to register two separate callbacks, one for each. It's hard to see from you code snippet if that is done. If you can share the full code (send to ti-3dtof@list.ti.com) we can better help you.

    -Larry
  • Dear Praveen:

    Thanks your kind support.

    Best Regards,

    ShengHua

  • Dear Larry:

    OK.
    I will pack my codes of Multi-Camera Synchronous later, then send to the email address of " ti-3dtof@list.ti.com" a.s.a.p.
    Thanks your kind support.

    Best Regards,
    ShengHua
  • Dear Larry:

    We modify the CamerasSaveStreamTest Project from TI Voxel SDK ,then registered three separate callbacks for each Grabber and each callback has its own buffer and loop.
    After finding some bugs, we can roughly run 3 different T.O.F Camera streaming together. Thanks your help.

    Best Regards,
    ShengHua

    /*****************************************************************/
    The main program is as follows.
    /////////////////////////////////////////////////////////////////////////////
    The program flow :
    1. Connect Camera 1/2/3 to the computer with USB cable.
    2. Initial grabber1/2/3 with specific callback function
    3. Start grabber1/2/3 streaming.
    ////////////////////////////////////////////////////////////////////////////

    /*
    * TI Voxel Lib component.
    *
    * Copyright (c) 2014 Texas Instruments Inc.
    */


    #include "CameraSystem.h"

    #include "SimpleOpt.h"
    #include "Common.h"
    #include "Logger.h"
    #include "UVCStreamer.h"
    #include <iomanip>
    #include <fstream>
    //2017.0928 by ysh .....st ......
    #include "grabber.h"
    ///// .....end ......
    using namespace Voxel;

    enum Options
    {
    VENDOR_ID = 0,
    PRODUCT_ID = 1,
    SERIAL_NUMBER = 2,
    DUMP_FILE = 3,
    NUM_OF_FRAMES = 4
    };

    Vector<CSimpleOpt::SOption> argumentSpecifications =
    {
    { VENDOR_ID, "-v", SO_REQ_SEP, "Vendor ID of the USB device (hexadecimal)"}, // Only worker count is needed here
    { PRODUCT_ID, "-p", SO_REQ_SEP, "Comma separated list of Product IDs of the USB devices (hexadecimal)"},
    { SERIAL_NUMBER,"-s", SO_REQ_SEP, "Serial number of the USB device (string)"},
    { DUMP_FILE, "-f", SO_REQ_SEP, "Name of the file to dump extracted frames"},
    { NUM_OF_FRAMES,"-n", SO_REQ_SEP, "Number of frames to dump [default = 1]"},
    SO_END_OF_OPTIONS
    };

    void help()
    {
    std::cout << "CameraSystemSaveStreamTest v1.0" << std::endl;

    CSimpleOpt::SOption *option = argumentSpecifications.data();

    while(option->nId >= 0)
    {
    std::cout << option->pszArg << " " << option->helpInfo << std::endl;
    option++;
    }
    }

    #if 1
    typedef enum {
    FRAMEFLAG_RAW_UNPROCESSED = (1 << DepthCamera::FRAME_RAW_FRAME_UNPROCESSED),
    FRAMEFLAG_RAW_PROCESSED = (1 << DepthCamera::FRAME_RAW_FRAME_PROCESSED),
    FRAMEFLAG_DEPTH_FRAME = (1 << DepthCamera::FRAME_DEPTH_FRAME),
    FRAMEFLAG_XYZI_POINT_CLOUD_FRAME = (1 << DepthCamera::FRAME_XYZI_POINT_CLOUD_FRAME),
    FRAMEFLAG_ALL = (FRAMEFLAG_RAW_UNPROCESSED
    | FRAMEFLAG_RAW_PROCESSED
    | FRAMEFLAG_DEPTH_FRAME
    | FRAMEFLAG_XYZI_POINT_CLOUD_FRAME)
    } FrameFlag;
    #endif


    int main(int argc, char *argv[])
    {

    CSimpleOpt s(argc, argv, argumentSpecifications);

    logger.setDefaultLogLevel(LOG_INFO);

    uint16_t vid = 0;

    Vector<uint16_t> pids;
    String serialNumber;
    //2017.0929 by ysh .....
    String serialNumber1 = "10408201039135";
    String serialNumber2 = "10410711038992";
    String serialNumber3 = "10408671038987";

    String dumpFileName;

    int32_t frameCount = 1;

    //2017.0929 by ysh
    bool test_flag = 0;

    char *endptr;

    while (s.Next())
    {
    if (s.LastError() != SO_SUCCESS)
    {
    std::cout << s.GetLastErrorText(s.LastError()) << ": '" << s.OptionText() << "' (use -h to get command line help)" << std::endl;
    help();
    return -1;
    }

    //std::cout << s.OptionId() << ": " << s.OptionArg() << std::endl;

    Vector<String> splits;
    switch (s.OptionId())
    {
    case VENDOR_ID:
    vid = (uint16_t)strtol(s.OptionArg(), &endptr, 16);
    break;

    case PRODUCT_ID:
    split(s.OptionArg(), ',', splits);

    for(auto &s1: splits)
    pids.push_back((uint16_t)strtol(s1.c_str(), &endptr, 16));

    break;

    case SERIAL_NUMBER:
    serialNumber = s.OptionArg();
    break;

    case DUMP_FILE:
    dumpFileName = s.OptionArg();
    break;

    case NUM_OF_FRAMES:
    frameCount = (int32_t)strtol(s.OptionArg(), &endptr, 10);
    break;

    default:
    help();
    break;
    };
    }

    if(vid == 0 || pids.size() == 0 || pids[0] == 0 || dumpFileName.size() == 0)
    {
    logger(LOG_ERROR) << "Required argument missing." << std::endl;
    help();
    return -1;
    }

    CameraSystem sys;

    // Get all valid detected devices
    const Vector<DevicePtr> &devices = sys.scan();

    DevicePtr toConnect;
    DevicePtr toConnect1;
    DevicePtr toConnect2;
    DevicePtr toConnect3;

    std::cout << "Detected devices: " << std::endl;
    for(auto &d: devices)
    {
    std::cout << d->id() << std::endl;

    if(d->interfaceID() == Device::USB)
    {
    USBDevice &usb = (USBDevice &)*d;

    //if (usb.vendorID() == vid && (serialNumber.size() == 0 || usb.serialNumber() == serialNumber)){
    //if (usb.vendorID() == vid && (serialNumber1.size() == 0 || serialNumber2.size() == 0 || usb.serialNumber() == serialNumber1 || usb.serialNumber() == serialNumber2)){
    if (usb.vendorID() == vid){
    for(auto pid: pids)
    if (usb.productID() == pid)
    {
    if (usb.serialNumber() == serialNumber1)
    { toConnect1 = d;}
    if (usb.serialNumber() == serialNumber2)
    { toConnect2 = d; }
    if (usb.serialNumber() == serialNumber3)
    {toConnect3 = d;}
    }
    //toConnect = d;
    }
    }
    }

    //if(!toConnect)
    //{
    // logger(LOG_ERROR) << "No valid device found for the specified VID:PID:serialnumber" << std::endl;
    // return -1;
    //}

    if (!toConnect1 && !toConnect2)
    {
    logger(LOG_ERROR) << "No valid device found for the specified VID:PID:serialnumber" << std::endl;
    return -1;
    }

    //DepthCameraPtr depthCamera = sys.connect(toConnect);
    DepthCameraPtr depthCamera1 = sys.connect(toConnect1);
    DepthCameraPtr depthCamera2 = sys.connect(toConnect2);
    DepthCameraPtr depthCamera3 = sys.connect(toConnect3);

    //2017.0928 by ysh ......
    CameraSystem sys1, sys2,sys3;

    //2017.0928 by ysh......It is OK ........
    //Grabber Grabber1(depthCamera,Grabber::FRAMEFLAG_XYZI_POINT_CLOUD_FRAME, sys);

    Grabber Grabber1(depthCamera1, Grabber::FRAMEFLAG_DEPTH_FRAME, sys1,1);
    Grabber Grabber2(depthCamera2, Grabber::FRAMEFLAG_XYZI_POINT_CLOUD_FRAME, sys2,2);
    Grabber Grabber3(depthCamera3, Grabber::FRAMEFLAG_XYZI_POINT_CLOUD_FRAME, sys2,3);
    // test_flag=Grabber1.isInitialized();

    //std::cout << "Successfully loaded depth camera for device " << toConnect->id() << std::endl;
    Grabber1.start();
    Grabber2.start();
    Grabber3.start();

    while (1){};

    return 0;
    }
  • Dear All:

           We would like to stream TOF Cameras by register callback function. First we try to integrate grabber.cpp and grabber.h(in github.com/.../Qt)

           to my main code. By setting frame rate to 30 fps and Camera profile then run Grabber1.start().

           We found the frame rate is nearly 6 fps due to drop a frame in a slow forward pipeline.  Why? How to do next step?

    The program code is as follows.

    ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

    DepthCameraPtr depthCamera1 = sys1.connect(toConnect1);

    Grabber Grabber1(depthCamera1, Grabber::FRAMEFLAG_RAW_PROCESSED, sys1, 1);

    Grabber1.setFrameRate(30.0);

    Grabber1.setProfile(_profile2_SR);

    if (Grabber1.isInitialized() && !Grabber1.isRunning())

    Grabber1.start();

    while (1){ };

    return 0;

    //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

    Best Regards,

    Sheng Hua

  • Dear All:

    Attached is my program code.

    Best Regards,

    ShengHua

    CameraSystemSaveStreamTest_3.cppgrabber_sh.cppgrabber_sh.h

  • Hi Shenghua,

    Is your multi-camera streaming working normally now?

    -Larry

  • Dear Larry:

     

    Thanks your kind support. We try to rewrite a callback function then test the PCL callback function. After connect multiple TOF cameras, the frame rate of grabber is nearly 30 fps. Now we would like to integrate algorithms for multiple camera callback and found the frame rate is reduced to 12 fps due to slow forward pipelines. In our application, we hope to keep the frame rate to 30 fps with image process algorithms. How can we do next step? Using multiple thread for each callback function or other methods?

     

    Best Regards,

    ShengHua

  • Using multiple thread is one possibility. Also make sure you're using a FIFO and not doing too much processing inside your callback.
  • Dear Larry:

    OK. How can we use a FIFO in the PCL callback?
    Is there an example for this case in muliple cameras synchronization?
    Thanks your support.

    Best Regards,
    ShengHua