TDA2E: Input to lane detection algorithm sample

suganthi velusamy

Part Number: TDA2E

Hi,

I am trying to understand and make use of the Lane Detection sample application provided in vision sdk.

As per the use guide, 8-bit binary images are given as input to the Lane Detection algorithm.

For a real case, if the video stream is coming from the camera, what input we need to provide to the LD sample application?

It is mentioned as C66x DSP core can be used for LD. Any other cores can be used for this purpose, or DSP core is dedicated to the LD?

Thanks,

Suganthi

over 8 years ago

0 Yordan Kamenov over 8 years ago

TI__Mastermind 42515 points

Hi Suganthi,

the Lane detection supports as input only 8 bit luma data. In case a camera is used as an input,
then only the Y (the luma) component from it's YUV output can be used as input for the LD algorithm.

You can check
<VisionSDK_install_dir>\vision_sdk\examples\tda2xx\src\usecases\vip_single_cam_lane_detection
usecase for Camera -> LD example.

I am not sure if running the LD on other cores is possible, but usually object detection algorithms require
doing a lot of same calculation on different data, for which the DSP architecture is optimal.

Regards,
Yordan

0 suganthi velusamy over 8 years ago in reply to Yordan Kamenov

Intellectual 530 points

Hi Yordan,

Currently we are trying to use the following.

As per our requirement, we are trying to use the IP camera. So the video input will be coming through Ethernet port.

But currently, for testing purpose, we are sending a recorded video (Video of road with Lane markings and vehicles moving) from PC to EVM board via Ethernet port using network_tx tool and using the network rx/tx use case.

The video input from Ethernet port has to go to the Lane Detection Module.

Query 1:
Whether the Lane Detection module will be able to use this video input?

Query 2:
What kind of configuration we need to do on the EVM board, to make use of the video from network port and use the Lane detection module?

Query 3:
What will be the output of the Lane detection module?
How can we make use of this output further?

Thanks,
Suganthi

0 Yordan Kamenov over 8 years ago in reply to suganthi velusamy

TI__Mastermind 42515 points

Hi Suganthi,

On Query 3:
the Lane detection outputs (X, Y) coordinates of detected lanes and also departure warning as one of no lane crossed/left lane crossed/right lane crossed.
You can refer to the document:
<VisionSDK_install_dir>\ti_components\algorithms_codecs\REL.200.V.LD.C66X.00.02.03.00\200.V.LD.C66X.00.02\modules\ti_lane_detection\docs\LaneDetection_DSP_UserGuide.pdf
(in particular chapter 4.6.3. and 5.1.9.4.). You can also go through the code in the test app:
<VisionSDK_install_dir>\ti_components\algorithms_codecs\REL.200.V.LD.C66X.00.02.03.00\200.V.LD.C66X.00.02\modules\ti_lane_detection\test

On Query 1 and 2 I will ping an expert to comment.

Regards,
Yordan

0 Prasad Jondhale over 8 years ago in reply to Yordan Kamenov

TI__Mastermind 27743 points

Hello Suganthi,

Query 1:
Whether the Lane Detection module will be able to use this video input?
>> Yes, the VSDK links and chains framework allows easily plug any input for your application. Network input can be taken for your LD module using null source link (vsdk/src/links_common/nullsrc). I believe you have already run some examples using this link.

Query 2:
What kind of configuration we need to do on the EVM board, to make use of the video from network port and use the Lane detection module?
>> No explicit configuration is needed for getting network data. Just make sure you are able ping to EVM IP address from your PC before starting network application.

0 suganthi velusamy over 8 years ago in reply to Prasad Jondhale

Intellectual 530 points

Thanks Prasad.

As per the user guide, the input to LD module is 8-bit binary images.

And I am able to see *.y files (image files) inside the test/input folder of LD module.

But how can I use the video from camera/network port as an input to the Lane Detection Module.

Currently we are trying to run the sample test application of the Lane Detection Module.

My doubt is, input to Lane Detection sample test application should be an 8-bit image files or it can be video from camera also?

Regards,
Suganthi

0 suganthi velusamy over 8 years ago in reply to Prasad Jondhale

Intellectual 530 points

Hi Prasad,

By default, the LD module is taking input from HDMI Src. This is not working in our case, so we tried with the OMNI_VISION camera as source. We modified the source code for getting omnivision camera as source and it is working fine.

As per your reply to Query 1, we can feed the network input to the LD module. We are trying to achieve this.

Whether we need to create the server, and we need to port the related code of network rx usecase in the Lane Detection usecase?

Thanks,
Suganthi

0 Kedar Chitnis over 8 years ago in reply to suganthi velusamy

TI__Genius 9101 points

Yes, LD algorithm APIs need 8b grayscale image as input.

Since camera gives YUV data format, while capturing we convert the YUV to YUV420SP.

Here there is one plane of 8b Y data and one plance of 8b UV data.

The lane detect algplugin takes such a YUV420SP data as input and when invoking the LD algorithm it gives only the 8b Y plane as input to the algorithm

regards

Kedar

0 suganthi velusamy over 8 years ago in reply to Kedar Chitnis

Intellectual 530 points

Hi Kedar,

We have two different sample programs for Lane Detection under vision sdk.

1. One is under the following directory.
C:\VISION_SDK_02_12_00_00\ti_components\algorithms_codecs\REL.200.V.LD.C66X.00.02.03.00
We tried running this and we are getting an excel file with (x,y) co-ordinates as output.
How can we convert video to YUV420SP and send as input to the sample application?

2. Another is under the following directoy.
C:\VISION_SDK_02_12_00_00\vision_sdk\examples\tda2xx\src\usecases\vip_single_cam_lane_detection
We tested this by sending a sample road video through Ethernet port.
How it will mark the Lane detection in the display video? We are able to see some line markings on only the right side lane.

Query 1 :
Can you please help us to understand which example application we can use for lane detection, what is the difference between the two sample application.

Query 2:
Our use-case is,
1. To detect both the lanes from the input video and display clear line markings on lane.
2. Get the X,Y co-ordinates or difference between the center of the and lane marking, for providing departure warning when crossing lanes.

For the above requirement, which sample application will be suitable, and how to proceed further on this?

Please advice.

Thanks,
Suganthi

0 Pramod Kumar Swami over 8 years ago in reply to suganthi velusamy

TI__Genius 14050 points

Hi Suganthi,

The core algorithm used by both the test applications is provided by same algorithm library located at ti_components\algorithms_codecs\REL.200.V.LD.C66X.00.02.03.00\modules\ti_lane_detection\lib

First test application is for simple test purpose on the particular core (C66x) without involving entire SOC and real time IO devices. It can only accept file input and output coordinates of the detected lane marking. whereas

Second test application is to test with real input and to also display on a display device - so it involves more SOC resources like capture, display, controller etc..

Since your requirement is to display lane marking - i would suggest you to use the second application and also tap out the output from core library in form of X,Y cordinates of lane marking and use in the way you want to use for your additional purpose.

Thanks,

with Regards,

Pramod

0 suganthi velusamy over 8 years ago in reply to Pramod Kumar Swami

Intellectual 530 points

Hi Promod,

Thank you for the details.

We tried to run the lane detection use-case in two scenarios.

One with camera input and another via network rx tool.

When using camera as input, and currently we are trying to play sample video in PC. We are able to see the lane region with markings clearly.

When using network_rx, we are sending the sample video from PC in MJPEG format. In use case, we are decoding the video. In this case, the lane region markings are not coming.

What are the basic parameters we need to modify, for different handling different video inputs?
Is there any ratio, that we can calculate and modify the values of roi(width, height, startX, startY)?
What other parameters are there that I need to update if the video input is in a different resolution?

Your suggestion will be helpful.

Thanks,
Suganthi

0 Prashanth Viswanath over 8 years ago in reply to suganthi velusamy

TI__Prodigy 410 points

Hi Suganthi,

The values of the roi can be set based on your input resolution and the scene itself (for example where is your lane region/road region). Based on that, you can determine your startX, startY, roi width and height. Note that the roi width set assumes that it includes padding of 6 (required for canny edge).Eg: If you want effective roi of 576, please set the width to 582. Internally in the algorithm, we construct a trapezoidal roi using the parameters set by user. Note that the effective roi computed can be lesser than the user defined roi dimensions. Also, we assume that the roi width is a multiple of 8.

Based on the input resolution, you must also change the maxRho parameter used in hough transform. This is calculated as sqrt(roi_w*roi_w + roi_h*roi_h).

0 suganthi velusamy over 8 years ago in reply to Prashanth Viswanath

Intellectual 530 points

Hi Prashanth,

Thanks for the details.

I have one more query. In the lane detection sample use case, we are giving the LD_ALG_WIDTH and LD_ALG_HEIGHT as 640x360.

For object detection, it is 1280x720.

Is there any specific reason that the width and height of Lane detection algorithm as 640x4360?

Regards,
Suganthi

0 Prashanth Viswanath over 8 years ago in reply to suganthi velusamy

TI__Prodigy 410 points

Hi Suganthi,

For lane detection, we use 640x360 since that resolution is sufficient for detecting lanes. However, for object detection, larger resolution would help since you do multi-scale based object detect in order to detect variable sized objects.

You can always use higher resolution for lane detection as well, but the compute is going to increase.

0 suganthi velusamy over 8 years ago in reply to suganthi velusamy

Intellectual 530 points

Hi,

Query 1:
Consider there is a video frame with 4 lanes, the lane detection algorithm will give the co-ordinates of all the lanes in the frame or it will give the left and right co-ordinates of current track only?

Query 2:
How what basis, we need to configure the canny minimum and maximum threshold parameters?

It will be really helpful if we get clarification on the above queries.

Thanks,
Suganthi

0 suganthi velusamy over 8 years ago in reply to suganthi velusamy

Intellectual 530 points

Hi Prashanth,

Thank you for the details.
Is there any suggestion on the above queries?

Thanks,
Suganthi

0 Prashanth Viswanath over 8 years ago in reply to suganthi velusamy

TI__Prodigy 410 points

Hi Suganthi,

1. It doesnt do multi-lane detection. So, depending on the ROI, it will detect the line with more number of votes in hough transform and output those lines. We use it to do only current lane detection only by restricting the ROI.

2. Canny edge can be tuned as per your needs. If you expect clear lane markings, then you can set these to high values (around 80 to 100). This will reduce the number of edges produced and mostly related to lanes in case of clear lane markings and also would reduce the compute post edge detection since you would operate on fewer edges. However, if the markings are smudged and not clear, then you may have to set the thresholds lower.

0 suganthi velusamy over 8 years ago in reply to Prashanth Viswanath

Intellectual 530 points

Prashanth,

Thank you for the clarity.

We like to detect all the lanes in the captured frame, which may be needed during lane crossing.
Can we modify the algorithm for achieving this?
In that case, where can we start modifying the source code?
Please suggest.

Thanks,
Suganthi

0 Prashanth Viswanath over 8 years ago in reply to suganthi velusamy

TI__Prodigy 410 points

Hi Suganthi,
If you want to modify to detect multiple lanes, you would need to add few extra blocks after Hough Transform. The hough transform for lines would provide multiple maxima. The maxima could be related to the same lane or different lanes. So, you would need some logic to determine that. I think that should be the start, and then some post-processing like tracking multiple lanes to smoothen your detection. You can add your specific kernels in a file and place it in the ti_lane_detection/algo/src folder. Then, you can call your kernels in the algo/ld_alg.c file.

0 suganthi velusamy over 8 years ago in reply to Prashanth Viswanath

Intellectual 530 points

Prashanth,

Thank you for the explanation.
I have only the "test, include, lib" folders under "ti_lane_detection" in my "ti_components".
I don't have the "algo" directory with me.
How can I get the source code?

Thanks,
Suganthi

0 Pramod Kumar Swami over 8 years ago in reply to suganthi velusamy

TI__Genius 14050 points

HI Suganthi,

Source code related infomration are confidential and you shall use private forum as mentioned below for such discussions.
e2eprivate.ti.com/.../309.aspx

If your company has NDA with TI, you should be able to get access to the private forum and we can discuss the remaining topics in that forum. If you don't have access to the above forum please work with your TI point of contact to get access to this forum

Thanks,
With Regards,
Pramod

0 suganthi velusamy over 8 years ago in reply to Pramod Kumar Swami

Intellectual 530 points

Hi Pramod,

Thanks for the info.

Thanks,
Suganthi

Processors

Processors forum

TDA2E: Input to lane detection algorithm sample