• State Resolved
  • Locked Locked
  • Replies 8 replies
  • Answers 2 answers
  • Subscribers 101 subscribers
  • Views 1671 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM5718: Low-latency video

Hitoshi Sugawara
Mastermind 18650 points
Part Number: AM5718

Hi Experts,

I am looking for the lowest latency for the video function solution with AM5718.
The ISP is done at the external HW.
CSI2 -> AM5718 IVA H.264 compression -> transferred the data by any IF.
The input is/are 720P60 single or dual.


According to the white paper called "Low-latency design considerations for video-enabled drones" 
It introduces " Figure 4: Slice-based impact on processing timeline” and shows that the End-to-End latency is 39.9ms.
www.ti.com/lit/spry301

Could you please let me know if AM5718 could support the same ultra-low latency video compression and transfer?
I would like to know the latency with the conditon.

Thank you very much for you kind investigation.
Best regards,
Hitoshi

  • 0
    TI__Mastermind 22386 points

    Hi Hitoshi,

    H.264 encoder on AM5718 takes roughly 8 ms to encode one 720p resolution frame. What is your latency requirement?

    The IVA-HD encoder library on AM5718 supports slice level encoding feature that the white paper is referring but there is no example application available which demonstrates this feature.

    You may want to  read H.264 encoder user guide Appendix G section on "Low Latency / Sub Frame Level Synchronization".

    You can try the slice level encoding functionality by modifying this test application which encodes at frame level.

  • 0 in reply to Manisha Agrawal
    TI__Mastermind 18650 points

    Hi Manisha-san,

    Thank you for prompt reply.

    This is the required flows as follows:

      Camera capturing at 720p60, encoding by H.264 (slice level  N=30), Network BW 2Mbps, receiving the data, Decode ( slice level N=30), and then display 720p60.  ( Single and Dual inputs are  planned. )

    According to the white paper, it takes 39.9ms including ISP.

    I believe that AM5718 does not support ISP HW.

    So, could you please let me know you estimated time from the encode stage  to the display stage following the procedures in " Figure 4: Slice-based impact on processing timeline” in .

    Best regards,
    Hitoshi

  • 0 in reply to Hitoshi Sugawara
    TI__Mastermind 22386 points
    Hi Hitoshi,

    capture @720p60 will take 16 ms, encode can take 1 to 2 ms when there are 10 + slices in a 720p resolution frame, transmit will take less then 0.5 ms, network latency will depend upon network speed. If the streamed video are received on AM57x SoC, then same timings will apply on the receiver side. So you can expect overall system latency to be around 36 ms + network latency.
  • 0 in reply to Manisha Agrawal
    TI__Mastermind 18650 points
    Hi Manisha-san,
    Thank you so much for your estimation.
    It helps a lot.
    Best regards,
    Hitoshi
  • 0 in reply to Hitoshi Sugawara
    TI__Mastermind 22386 points
    Hi Hitoshi,

    I would like to correct the estimate above. On the receiver side, AM57x display driver doesn't support slice slice level display. Once a frame is produced for display, it expects entire frame data is present for display. Due to this, slice level low latency decoding won't help. Entire frame needs to be decoded before submitting it for display. With this constrain in mind, the receiver side latency on AM57x is 24ms. With that, you can expect overall network latency as 42 ms + network latency.
  • 0 in reply to Manisha Agrawal
    TI__Mastermind 18650 points
    Hi Manisha,
    Thank you for your correction.
    I understood.

    May I ask one more question?
    If two video streams comes from CSI2 and then they are encoded and decoded, could you please estimate the latency?

    Thank you very much for your kind support.
    Best regards,
    Hitoshi
  • 0 in reply to Hitoshi Sugawara
    TI__Mastermind 22386 points
    The camera capture will depend upon capture rate. If you are capturing the videos at 60 fps, then image capture will take 16 ms.

    I am assuming that encoding and decoding in your use case happens on same AM57x SoC. In this case, for 720p resolution, encoding and decoding will take 8 ms each. This is because there is single IVA-HD on AM57x device, so encoding and decoding will need to happen sequentially.
  • 0 in reply to Manisha Agrawal
    TI__Mastermind 18650 points
    Hi Manisha,
    Thank you very much for your estimation.
    Best regards,
    Hitoshi