This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

SK-AM62A-LP: Frame drop often happens on Am62ax platform?

Part Number: SK-AM62A-LP

  We are porting our dToF sensor from rockchip rk3568 to TI Am62ax SoC recently. Our dToF sensor output 4 streams (2 VCs and 2 data type: raw12 and raw8), and the first 2 bytes of the frame buffer for last streams(stream2, stream3) is frame id (16 bits value from 0 to 65535). Because from our test, it looks some frames are dropped, so I added the following code the check the continuity of the frame id,from the attached log, some frame-drop are happening.

bool V4l2_Dev::usrExtFrameIdMatchCheck(const char *FrameBuff)
{
    bool ret = true;
    u16 curFrameId = (FrameBuff[1] << 8) | FrameBuff[0];

    if ((UNINITIED_FRAME_ID != last_frameId_in_raw) && ((last_frameId_in_raw + 1) != curFrameId))
    {
        ret = false;
        DBG_ERROR("--last_frameId_in_raw: %d, curFrameId: %d for stream%d--...", 
            last_frameId_in_raw, curFrameId, video_device_idx);
    }
    else {
        DBG_INFO("--last_frameId_in_raw: %d, curFrameId: %d for stream%d--...", 
            last_frameId_in_raw, curFrameId, video_device_idx);
    }

    last_frameId_in_raw = curFrameId;
    return ret;
}

Would you please help to check the reason and figure out a solution?

I have some ideas/questions, please help to check whether it help too?

1. Currently, the mipi link frequency is 500M, since the mipi frequency of the TX side of our dToF sensor is 1G. Can I increase the mipi link frequency of RX side from 500M to 600 or higher? Will this help the frame-drop issue?

2. Can we change the DMA buffer speed? If yes, how to do?

3. I will try to add the buffer count when the application call VIDIOC_REQBUFS ioctl commands.

------------------log-------------------------------

[ 513.722678] Failed DMA transfer retval=3, residue=0
[2022-05-04 00:01:44.305.475.100] ERROR: <v4l2_dev.cpp-usrExtFrameIdMatchCheck() 447> --last_frameId_in_raw: 2181, curFrameId: 2206 for stream2--...
[ 513.782685] j721e-csi2rx 30102000.ticsi2rx: DMA transfer timed out for drain buffer
[2022-05-04 00:01:44.367.935.735] ERROR: <v4l2_dev.cpp-usrExtFrameIdMatchCheck() 447> --last_frameId_in_raw: 2181, curFrameId: 2206 for stream3--...
[ 513.846682] j721e-csi2rx 30102000.ticsi2rx: DMA transfer timed out for drain buffer
[ 513.854670] Failed DMA transfer retval=3, residue=0
[ 513.859802] Failed DMA transfer retval=3, residue=0
[2022-05-04 00:01:44.442.392.015] ERROR: <v4l2_dev.cpp-usrExtFrameIdMatchCheck() 447> --last_frameId_in_raw: 2206, curFrameId: 2233 for stream2--...
[2022-05-04 00:01:44.442.670.220] ERROR: <v4l2_dev.cpp-usrExtFrameIdMatchCheck() 447> --last_frameId_in_raw: 2206, curFrameId: 2233 for stream3--...

MobaXterm_TI_SoC_Uart_20240327_191905.txt

  • Hi David,

    Please give me some time to analyze and get back. 

    Thanks

    Yogesh

  • Here is intterrupts BEFORE and AFTER running our application, please see the attched log.

    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new# insmod ads6311.ko__
    [   55.644820] <DRV_ADS6311> [2022-05-04 09:34:15.090.132.115] <N> 9333 ----- Probe Start -------
    [   55.653712] ads6311 virtio1.ti.ipc4.ping-pong.-1.13: mcuctrl_rpmsg_probe() Line:8156
    [   55.661539] <DRV_ADS6311> [2022-05-04 09:34:15.106.854.390] <N> 8163 new channel: 0x400 -> 0xd, rpmsg_device_id: ti.ipc4.ping-pong, endpoint address: 0x400!
    [   55.661539]
    [   55.761712] <DRV_ADS6311> [2022-05-04 09:34:15.207.023.285] <N> 8175 done, ret:0!
    [   55.761712]
    [   55.771189] ads6311 virtio0.ti.ipc4.ping-pong.-1.14: mcuctrl_rpmsg_probe() Line:8156
    [   55.779063] <DRV_ADS6311> [2022-05-04 09:34:15.224.375.545] <E> 8159 This is NOT MCU core, rpdev->dst: 14
    [   55.788681] ads6311 virtio0.ti.ipc4.ping-pong.-1.14: rpmsg_dev_probe: failed: -22
    [   55.796196] ads6311: probe of virtio0.ti.ipc4.ping-pong.-1.14 failed with error -22
    [   55.803959] ads6311 virtio2.ti.ipc4.ping-pong.-1.14: mcuctrl_rpmsg_probe() Line:8156
    [   55.811726] <DRV_ADS6311> [2022-05-04 09:34:15.257.040.080] <E> 8159 This is NOT MCU core, rpdev->dst: 14
    [   55.821329] ads6311 virtio2.ti.ipc4.ping-pong.-1.14: rpmsg_dev_probe: failed: -22
    [   55.828823] ads6311: probe of virtio2.ti.ipc4.ping-pong.-1.14 failed with error -22
    [   55.836587] SPI driver adaps,ads6311spi has no spi_device_id for adaps,ads6311spi
    [   55.844271] adaps,ads6311spi spi1.0: cam_sensor_spi_driver_probe() Line:6581
    [   55.851520] adaps,ads6311spi spi1.0: Consider updating driver adaps,ads6311spi to match on endpoints
    [   55.862808] <DRV_ADS6311> [2022-05-04 09:34:15.308.119.925] <N> 6603 done, ret:0!
    [   55.862808]
    [   55.872269] <DRV_ADS6311> [2022-05-04 09:34:15.317.581.300] <N> 9381 ----- Start calib_flash_spi_driver Probe-------
    [   55.886267] SPI driver adaps,calib_flash has no spi_device_id for adaps,calib_flash
    [   55.894139] <DRV_ADS6311> [2022-05-04 09:34:15.339.452.350] <N> 9383 ----- Start calib_flash_spi_driver Probe--rc:0-----
    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new# echo 0x8 > /sys/kernel/debug/adaps/dbg_ctrl
    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new# cat ~/hawk/info
    Adaps Ads6311 dToF Sensor Driver
    Version: 2.0.0_LM20240327a
    Build UTC Time: Mar 27 2024,07:56:01
    Sensor ChipId: 0x0
    Platform: TI Am62ax
    MultipleStream Support: Yes
    ONE_DATATYPE_ENABLE: No
    Dbg_ctrl: 0x8
    MIPI Link Frequency:
        260000000
        500000000
    Bus Type: SPI
    Bus Num: 1
    SPI mode: 3
    SPI max speed: 15000000Hz (15MHz)
    SPI Read Mode: Linux default (2-frames)
    
    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new# media-ctl -d /dev/media0 -R "'cdns_csi2rx.30101000.csi-bridge' [0/0 -> 1/0 [1], 0/1 -> 1/1 [1], 0/2 -> 1/2 [1], 0/3 -> 1/3 [1]]"
    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new# media-ctl -d /dev/media0 -R "'30102000.ticsi2rx' [0/0 -> 1/0 [1], 0/1 -> 2/0 [1], 0/2 -> 3/0 [1], 0/3 -> 4/0 [1]]"
    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new# export rawchk_frame_id_enable=true
    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new# export qchk_frame_data_enable=true
    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new# export display_frame_rx_progress=true
    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new# ./hawk_concurrency_test --setworkmode 6  --dumpframe 1000 &
    [1] 1284
    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new# Hawk concurrency test util
    Version: 1.0.0_LM20240327a
    Build Time: Mar 28 2024,03:01:52
    ONE_DATATYPE_ENABLE: No
    
    [2022-05-04 09:34:54.685.277.395] NOTICE: <hawk_concurrency_test.cpp-Hawk_Concurrency_Test() 112> ----input test_case: 0----
    [2022-05-04 09:34:54.685.392.995] NOTICE: <hawk_concurrency_test.cpp-Hawk_Concurrency_Test() 113> ----input work_mode: 6----
    [2022-05-04 09:34:54.685.431.510] NOTICE: <hawk_concurrency_test.cpp-Hawk_Concurrency_Test() 114> ----input dumpframe: 1000----
    [2022-05-04 09:34:54.685.468.335] NOTICE: <hawk_concurrency_test.cpp-Hawk_Concurrency_Test() 115> ----test_stream_cnt: 4----
    [2022-05-04 09:34:54.685.517.795] NOTICE: <v4l2_dev.cpp-V4l2_Dev() 60> Request frame_buffer_count: 6
    [2022-05-04 09:34:54.687.037.070] NOTICE: <v4l2_dev.cpp-V4l2_Dev() 60> Request frame_buffer_count: 6
    [2022-05-04 09:34:54.688.181.285] NOTICE: <v4l2_dev.cpp-V4l2_Dev() 60> Request frame_buffer_count: 6
    [2022-05-04 09:34:54.688.450.645] NOTICE: <v4l2_dev.cpp-V4l2_Dev() 60> Request frame_buffer_count: 6
    ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................[2022-05-04 09:34:58.896.657.850] NOTICE: <v4l2_dev.cpp-Stop_streaming() 687>
    ------Test Statistics------fps: 0, Qcheck invalid cnt: 0, md5_mismatch_count: 0 for /dev/video0, work_mode: 6 raw size: 32 X 192-----------
    [   99.506873] j721e-csi2rx 30102000.ticsi2rx: DMA transfer timed out for drain buffer
    [2022-05-04 09:34:58.960.301.780] NOTICE: <v4l2_dev.cpp-Stop_streaming() 687>
    ------Test Statistics------fps: 0, Qcheck invalid cnt: 0, md5_mismatch_count: 0 for /dev/video1, work_mode: 6 raw size: 32 X 192-----------
    [   99.570718] j721e-csi2rx 30102000.ticsi2rx: DMA transfer timed out for drain buffer
    [   99.578859] cdns-csi2rx 30101000.csi-bridge: csi2rx_stop: err_irqs: 0x20100, info_irqs: 0x2, err_bypass_cfg: 0x0, monitor_irqs: 0x0, monitor_irqs_mask_cfg: 0x0, info_irqs_mask_cfg: 0x0, err_irqs_mask_cfg: 0x0,
    [   99.578859] dphy_status: 0x333306, dphy_errirq: 0x0, dphy_errirq_mask_cfg: 0x0, err_dbg: 0x0, integration_dbg: 0x10000000.
    [   99.682723] j721e-csi2rx 30102000.ticsi2rx: DMA transfer timed out for drain buffer
    [2022-05-04 09:34:59.143.181.770] NOTICE: <hawk_concurrency_test.cpp-Testcase0() 208> ------Testcase 0 PASSED--------
    
    [2022-05-04 09:34:59.143.437.645] NOTICE: <hawk_concurrency_test.cpp-main() 618> ------Test End--------
    
    
    [1]+  Done                    ./hawk_concurrency_test --setworkmode 6 --dumpframe 1000
    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new#
    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new#
    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new#
    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new# cat /proc/interrupts
               CPU0       CPU1       CPU2       CPU3
     11:       4729       5676       4506       5536     GICv3  30 Level     arch_timer
     14:          6          0          0          0     GICv3 108 Level     mbox-r5-0
     15:          6          0          0          0     GICv3 109 Level     mbox-c7x-0
     16:         31          0          0          0     GICv3 140 Level     mbox-mcu_r5-0
     17:          0          0          0          0     GICv3  23 Level     arm-pmu
     18:          0          0          0          0     GICv3 131 Level     pinctrl
     19:       3704          0          0          0     GICv3  66 Level     4d000000.mailbox thr_012
     28:       3459          0          0          0     GICv3 193 Level     20000000.i2c
     29:         10          0          0          0     GICv3 194 Level     20010000.i2c
     30:          0          0          0          0     GICv3 195 Level     20020000.i2c
    119:          1          0          0          0  MSI-INTA 1970707 Level     8000000.ethernet-tx0
    209:          1          0          0          0  MSI-INTA 1971731 Level     8000000.ethernet
    247:       1002          0          0          0  MSI-INTA 13045760 Level
    248:       1009          0          0          0  MSI-INTA 13045761 Level
    249:       1008          0          0          0  MSI-INTA 13045762 Level
    250:       1008          0          0          0  MSI-INTA 13045763 Level
    253:       3773          0          0          0     GICv3 210 Level     2800000.serial
    254:          0          0          0          0   pinctrl 456 Edge      2800000.serial:wakeup
    256:          0          0          0          0     GICv3 134 Level     8000000.ethernet
    260:        245          0          0          0     GICv3 165 Level     mmc0
    262:          0          0          0          0      GPIO   0 Edge    -davinci_gpio  tps6594-1-0x48
    440:          0          0          0          0  tps6594-1-0x48   0 Edge      buck1_ov
    441:          0          0          0          0  tps6594-1-0x48   1 Edge      buck1_uv
    442:          0          0          0          0  tps6594-1-0x48   2 Edge      buck1_sc
    443:          0          0          0          0  tps6594-1-0x48   3 Edge      buck1_ilim
    444:          0          0          0          0  tps6594-1-0x48   4 Edge      buck2_ov
    445:          0          0          0          0  tps6594-1-0x48   5 Edge      buck2_uv
    446:          0          0          0          0  tps6594-1-0x48   6 Edge      buck2_sc
    447:          0          0          0          0  tps6594-1-0x48   7 Edge      buck2_ilim
    448:          0          0          0          0  tps6594-1-0x48   8 Edge      buck3_ov
    449:          0          0          0          0  tps6594-1-0x48   9 Edge      buck3_uv
    450:          0          0          0          0  tps6594-1-0x48  10 Edge      buck3_sc
    451:          0          0          0          0  tps6594-1-0x48  11 Edge      buck3_ilim
    452:          0          0          0          0  tps6594-1-0x48  12 Edge      buck4_ov
    453:          0          0          0          0  tps6594-1-0x48  13 Edge      buck4_uv
    454:          0          0          0          0  tps6594-1-0x48  14 Edge      buck4_sc
    455:          0          0          0          0  tps6594-1-0x48  15 Edge      buck4_ilim
    456:          0          0          0          0  tps6594-1-0x48  16 Edge      buck5_ov
    457:          0          0          0          0  tps6594-1-0x48  17 Edge      buck5_uv
    458:          0          0          0          0  tps6594-1-0x48  18 Edge      buck5_sc
    459:          0          0          0          0  tps6594-1-0x48  19 Edge      buck5_ilim
    460:          0          0          0          0  tps6594-1-0x48  20 Edge      ldo1_ov
    461:          0          0          0          0  tps6594-1-0x48  21 Edge      ldo1_uv
    462:          0          0          0          0  tps6594-1-0x48  22 Edge      ldo1_sc
    463:          0          0          0          0  tps6594-1-0x48  23 Edge      ldo1_ilim
    464:          0          0          0          0  tps6594-1-0x48  24 Edge      ldo2_ov
    465:          0          0          0          0  tps6594-1-0x48  25 Edge      ldo2_uv
    466:          0          0          0          0  tps6594-1-0x48  26 Edge      ldo2_sc
    467:          0          0          0          0  tps6594-1-0x48  27 Edge      ldo2_ilim
    468:          0          0          0          0  tps6594-1-0x48  28 Edge      ldo3_ov
    469:          0          0          0          0  tps6594-1-0x48  29 Edge      ldo3_uv
    470:          0          0          0          0  tps6594-1-0x48  30 Edge      ldo3_sc
    471:          0          0          0          0  tps6594-1-0x48  31 Edge      ldo3_ilim
    472:          0          0          0          0  tps6594-1-0x48  32 Edge      ldo4_ov
    473:          0          0          0          0  tps6594-1-0x48  33 Edge      ldo4_uv
    474:          0          0          0          0  tps6594-1-0x48  34 Edge      ldo4_sc
    475:          0          0          0          0  tps6594-1-0x48  35 Edge      ldo4_ilim
    476:          0          0          0          0  tps6594-1-0x48  36 Edge      vcca_ov
    477:          0          0          0          0  tps6594-1-0x48  37 Edge      vcca_uv
    495:          0          0          0          0  tps6594-1-0x48  55 Edge      npwron_start
    496:          0          0          0          0  tps6594-1-0x48  56 Edge      enable
    497:          0          0          0          0  tps6594-1-0x48  57 Edge      fsd
    498:          0          0          0          0  tps6594-1-0x48  58 Edge      soft_reboot
    499:          0          0          0          0  tps6594-1-0x48  59 Edge      bist_pass
    500:          0          0          0          0  tps6594-1-0x48  60 Edge      ext_clk
    501:          0          0          0          0  tps6594-1-0x48  61 Edge      twarn
    502:          0          0          0          0  tps6594-1-0x48  62 Edge      tsd_ord
    503:          0          0          0          0  tps6594-1-0x48  63 Edge      bist_fail
    504:          0          0          0          0  tps6594-1-0x48  64 Edge      reg_crc_err
    505:          0          0          0          0  tps6594-1-0x48  65 Edge      recov_cnt
    506:          0          0          0          0  tps6594-1-0x48  66 Edge      spmi_err
    507:          0          0          0          0  tps6594-1-0x48  67 Edge      npwron_long
    508:          0          0          0          0  tps6594-1-0x48  68 Edge      nint_readback
    509:          0          0          0          0  tps6594-1-0x48  69 Edge      nrstout_readback
    510:          0          0          0          0  tps6594-1-0x48  70 Edge      tsd_imm
    511:          0          0          0          0  tps6594-1-0x48  71 Edge      vcca_ovp
    512:          0          0          0          0  tps6594-1-0x48  72 Edge      pfsm_err
    513:          0          0          0          0  tps6594-1-0x48  73 Edge      imm_shutdown
    514:          0          0          0          0  tps6594-1-0x48  74 Edge      ord_shutdown
    515:          0          0          0          0  tps6594-1-0x48  75 Edge      mcu_pwr_err
    516:          0          0          0          0  tps6594-1-0x48  76 Edge      soc_pwr_err
    517:          0          0          0          0  tps6594-1-0x48  77 Edge      comm_frm_err
    518:          0          0          0          0  tps6594-1-0x48  78 Edge      comm_crc_err
    519:          0          0          0          0  tps6594-1-0x48  79 Edge      comm_adr_err
    520:          0          0          0          0  tps6594-1-0x48  82 Edge      en_drv_readback
    521:          0          0          0          0  tps6594-1-0x48  83 Edge      nrstout_soc_readback
    522:          0          0          0          0  tps6594-1-0x48  84 Edge      esm_soc_pin
    523:          0          0          0          0  tps6594-1-0x48  85 Edge      esm_soc_fail
    524:          0          0          0          0  tps6594-1-0x48  86 Edge      esm_soc_rst
    528:      14740          0          0          0     GICv3 115 Level     mmc1
    529:          0          0          0          0     GICv3 204 Level     20100000.spi
    IPI0:       492       1217       2017        744       Rescheduling interrupts
    IPI1:      3852       8443       6855       4961       Function call interrupts
    IPI2:         0          0          0          0       CPU stop interrupts
    IPI3:         0          0          0          0       CPU stop (for crash dump) interrupts
    IPI4:         0          0          0          0       Timer broadcast interrupts
    IPI5:       259        273        166        241       IRQ work interrupts
    IPI6:         0          0          0          0       CPU wake-up interrupts
    Err:          0
    root@am62axx-evm:/run/media/BOOT-mmcblk1p1/4G_new#
    

  • Hi ALL,

    1. Currently, the mipi link frequency is 500M, since the mipi frequency of the TX side of our dToF sensor is 1G. Can I increase the mipi link frequency of RX side from 500M to 600 or higher? Will this help the frame-drop issue?

    [Jianzhong] it will not related mipi frequency, csi driver have transferred the data to app side, because the app side can't consuming the data, so cause the drop frame.


    2. Can we change the DMA buffer speed? If yes, how to do?

    [Yogesh] disable CMA to see any improve, Yogesh will help post the detail for disable CMA. Yogesh want to see how much interrupts running in the CPU0. 

    3. I will try to add the buffer count when the application call VIDIOC_REQBUFS ioctl commands.

    BR,

    Biao 

  • David,

    Is it possible to configure the sensor to only send stream 4 and see if there are frame drops?

    Thank you.

    Jianzhong

  • Hi Jianzhong,

       No, it is impossbile.

    Thanks

    David

  • Hi Yogesh,

       I viewed the /proc/intterupts and /proc/meminfo before I run my application and during running my application. It looks all DMA intterupt happened on CPU0, please check the details as blow:

    Thanks and Best Regards,

    David

    ---------------------DURING running my application-------------------------

    root@am62axx-evm:/opt/edgeai-gst-apps# cat /proc/interrupts
               CPU0       CPU1       CPU2       CPU3
     11:      34437      60972      66400      67256     GICv3  30 Level     arch_timer
     14:          8          0          0          0     GICv3 108 Level     mbox-r5-0
     15:          7          0          0          0     GICv3 109 Level     mbox-c7x-0
     16:         35          0          0          0     GICv3 140 Level     mbox-mcu_r5-0
     17:          0          0          0          0     GICv3  23 Level     arm-pmu
     18:          0          0          0          0     GICv3 131 Level     pinctrl
     19:       7052          0          0          0     GICv3  66 Level     4d000000.mailbox thr_012
     28:       3461          0          0          0     GICv3 193 Level     20000000.i2c
     29:         10          0          0          0     GICv3 194 Level     20010000.i2c
     30:          0          0          0          0     GICv3 195 Level     20020000.i2c
    119:          0          0       3439          0  MSI-INTA 1970707 Level     8000000.ethernet-tx0
    209:          0        513          0          0  MSI-INTA 1971731 Level     8000000.ethernet
    241:          0          0          0          0  MSI-INTA 13045248 Edge      4e230000.dma-controller chan0
    242:          0          0          0          0  MSI-INTA 13045249 Edge      4e230000.dma-controller chan1
    243:          0          0          0          0  MSI-INTA 13045250 Edge      4e230000.dma-controller chan2
    244:          0          0          0          0  MSI-INTA 13045251 Edge      4e230000.dma-controller chan3
    245:          0          0          0          0  MSI-INTA 13045252 Edge      4e230000.dma-controller chan4
    246:          0          0          0          0  MSI-INTA 13045253 Edge      4e230000.dma-controller chan5
    247:       6915          0          0          0  MSI-INTA 13045760 Level     4e230000.dma-controller chan0
    248:       6922          0          0          0  MSI-INTA 13045761 Level     4e230000.dma-controller chan1
    249:       6921          0          0          0  MSI-INTA 13045762 Level     4e230000.dma-controller chan2
    250:       6926          0          0          0  MSI-INTA 13045763 Level     4e230000.dma-controller chan3
    251:          0          0          0          0  MSI-INTA 13045764 Level     4e230000.dma-controller chan4
    252:          0          0          0          0  MSI-INTA 13045765 Level     4e230000.dma-controller chan5
    253:      21908          0          0          0     GICv3 210 Level     2800000.serial
    254:          0          0          0          0   pinctrl 456 Edge      2800000.serial:wakeup
    256:          0          0          0          0     GICv3 134 Level     8000000.ethernet
    260:        277          0          0          0     GICv3 165 Level     mmc0
    262:          0          0          0          0      GPIO   0 Edge    -davinci_gpio  tps6594-1-0x48
    440:          0          0          0          0  tps6594-1-0x48   0 Edge      buck1_ov
    441:          0          0          0          0  tps6594-1-0x48   1 Edge      buck1_uv
    442:          0          0          0          0  tps6594-1-0x48   2 Edge      buck1_sc
    443:          0          0          0          0  tps6594-1-0x48   3 Edge      buck1_ilim
    444:          0          0          0          0  tps6594-1-0x48   4 Edge      buck2_ov
    445:          0          0          0          0  tps6594-1-0x48   5 Edge      buck2_uv
    446:          0          0          0          0  tps6594-1-0x48   6 Edge      buck2_sc
    447:          0          0          0          0  tps6594-1-0x48   7 Edge      buck2_ilim
    448:          0          0          0          0  tps6594-1-0x48   8 Edge      buck3_ov
    449:          0          0          0          0  tps6594-1-0x48   9 Edge      buck3_uv
    450:          0          0          0          0  tps6594-1-0x48  10 Edge      buck3_sc
    451:          0          0          0          0  tps6594-1-0x48  11 Edge      buck3_ilim
    452:          0          0          0          0  tps6594-1-0x48  12 Edge      buck4_ov
    453:          0          0          0          0  tps6594-1-0x48  13 Edge      buck4_uv
    454:          0          0          0          0  tps6594-1-0x48  14 Edge      buck4_sc
    455:          0          0          0          0  tps6594-1-0x48  15 Edge      buck4_ilim
    456:          0          0          0          0  tps6594-1-0x48  16 Edge      buck5_ov
    457:          0          0          0          0  tps6594-1-0x48  17 Edge      buck5_uv
    458:          0          0          0          0  tps6594-1-0x48  18 Edge      buck5_sc
    459:          0          0          0          0  tps6594-1-0x48  19 Edge      buck5_ilim
    460:          0          0          0          0  tps6594-1-0x48  20 Edge      ldo1_ov
    461:          0          0          0          0  tps6594-1-0x48  21 Edge      ldo1_uv
    462:          0          0          0          0  tps6594-1-0x48  22 Edge      ldo1_sc
    463:          0          0          0          0  tps6594-1-0x48  23 Edge      ldo1_ilim
    464:          0          0          0          0  tps6594-1-0x48  24 Edge      ldo2_ov
    465:          0          0          0          0  tps6594-1-0x48  25 Edge      ldo2_uv
    466:          0          0          0          0  tps6594-1-0x48  26 Edge      ldo2_sc
    467:          0          0          0          0  tps6594-1-0x48  27 Edge      ldo2_ilim
    468:          0          0          0          0  tps6594-1-0x48  28 Edge      ldo3_ov
    469:          0          0          0          0  tps6594-1-0x48  29 Edge      ldo3_uv
    470:          0          0          0          0  tps6594-1-0x48  30 Edge      ldo3_sc
    471:          0          0          0          0  tps6594-1-0x48  31 Edge      ldo3_ilim
    472:          0          0          0          0  tps6594-1-0x48  32 Edge      ldo4_ov
    473:          0          0          0          0  tps6594-1-0x48  33 Edge      ldo4_uv
    474:          0          0          0          0  tps6594-1-0x48  34 Edge      ldo4_sc
    475:          0          0          0          0  tps6594-1-0x48  35 Edge      ldo4_ilim
    476:          0          0          0          0  tps6594-1-0x48  36 Edge      vcca_ov
    477:          0          0          0          0  tps6594-1-0x48  37 Edge      vcca_uv
    495:          0          0          0          0  tps6594-1-0x48  55 Edge      npwron_start
    496:          0          0          0          0  tps6594-1-0x48  56 Edge      enable
    497:          0          0          0          0  tps6594-1-0x48  57 Edge      fsd
    498:          0          0          0          0  tps6594-1-0x48  58 Edge      soft_reboot
    499:          0          0          0          0  tps6594-1-0x48  59 Edge      bist_pass
    500:          0          0          0          0  tps6594-1-0x48  60 Edge      ext_clk
    501:          0          0          0          0  tps6594-1-0x48  61 Edge      twarn
    502:          0          0          0          0  tps6594-1-0x48  62 Edge      tsd_ord
    503:          0          0          0          0  tps6594-1-0x48  63 Edge      bist_fail
    504:          0          0          0          0  tps6594-1-0x48  64 Edge      reg_crc_err
    505:          0          0          0          0  tps6594-1-0x48  65 Edge      recov_cnt
    506:          0          0          0          0  tps6594-1-0x48  66 Edge      spmi_err
    507:          0          0          0          0  tps6594-1-0x48  67 Edge      npwron_long
    508:          0          0          0          0  tps6594-1-0x48  68 Edge      nint_readback
    509:          0          0          0          0  tps6594-1-0x48  69 Edge      nrstout_readback
    510:          0          0          0          0  tps6594-1-0x48  70 Edge      tsd_imm
    511:          0          0          0          0  tps6594-1-0x48  71 Edge      vcca_ovp
    512:          0          0          0          0  tps6594-1-0x48  72 Edge      pfsm_err
    513:          0          0          0          0  tps6594-1-0x48  73 Edge      imm_shutdown
    514:          0          0          0          0  tps6594-1-0x48  74 Edge      ord_shutdown
    515:          0          0          0          0  tps6594-1-0x48  75 Edge      mcu_pwr_err
    516:          0          0          0          0  tps6594-1-0x48  76 Edge      soc_pwr_err
    517:          0          0          0          0  tps6594-1-0x48  77 Edge      comm_frm_err
    518:          0          0          0          0  tps6594-1-0x48  78 Edge      comm_crc_err
    519:          0          0          0          0  tps6594-1-0x48  79 Edge      comm_adr_err
    520:          0          0          0          0  tps6594-1-0x48  82 Edge      en_drv_readback
    521:          0          0          0          0  tps6594-1-0x48  83 Edge      nrstout_soc_readback
    522:          0          0          0          0  tps6594-1-0x48  84 Edge      esm_soc_pin
    523:          0          0          0          0  tps6594-1-0x48  85 Edge      esm_soc_fail
    524:          0          0          0          0  tps6594-1-0x48  86 Edge      esm_soc_rst
    528:      15878          0          0          0     GICv3 115 Level     mmc1
    529:          0          0          0          0     GICv3 204 Level     20100000.spi
    IPI0:      1000       3020       1603       1102       Rescheduling interrupts
    IPI1:     16948      20656      20935      15707       Function call interrupts
    IPI2:         0          0          0          0       CPU stop interrupts
    IPI3:         0          0          0          0       CPU stop (for crash dump) interrupts
    IPI4:         0          0          0          0       Timer broadcast interrupts
    IPI5:       561        437        541        492       IRQ work interrupts
    IPI6:         0          0          0          0       CPU wake-up interrupts
    Err:          0
    
    root@am62axx-evm:/opt/edgeai-gst-apps# cat /proc/meminfo
    MemTotal:        3399916 kB
    MemFree:         2990636 kB
    MemAvailable:    3118536 kB
    Buffers:           11528 kB
    Cached:           247032 kB
    SwapCached:            0 kB
    Active:            47236 kB
    Inactive:         266064 kB
    Active(anon):        552 kB
    Inactive(anon):    63268 kB
    Active(file):      46684 kB
    Inactive(file):   202796 kB
    Unevictable:           0 kB
    Mlocked:               0 kB
    SwapTotal:             0 kB
    SwapFree:              0 kB
    Dirty:                 0 kB
    Writeback:             0 kB
    AnonPages:         54204 kB
    Mapped:            61000 kB
    Shmem:              9080 kB
    KReclaimable:      20360 kB
    Slab:              45844 kB
    SReclaimable:      20360 kB
    SUnreclaim:        25484 kB
    KernelStack:        2448 kB
    PageTables:         1908 kB
    SecPageTables:         0 kB
    NFS_Unstable:          0 kB
    Bounce:                0 kB
    WritebackTmp:          0 kB
    CommitLimit:     1699956 kB
    Committed_AS:     383580 kB
    VmallocTotal:   133143592960 kB
    VmallocUsed:        7120 kB
    VmallocChunk:          0 kB
    Percpu:             1216 kB
    HardwareCorrupted:     0 kB
    AnonHugePages:     16384 kB
    ShmemHugePages:        0 kB
    ShmemPmdMapped:        0 kB
    FileHugePages:         0 kB
    FilePmdMapped:         0 kB
    CmaTotal:         589824 kB
    CmaFree:          587412 kB
    HugePages_Total:       0
    HugePages_Free:        0
    HugePages_Rsvd:        0
    HugePages_Surp:        0
    Hugepagesize:       2048 kB
    Hugetlb:               0 kB
    

    ---------------------BEFORE running my application-------------------------

    root@am62axx-evm:/opt/edgeai-gst-apps# cat /proc/interrupts
               CPU0       CPU1       CPU2       CPU3
     11:      32127      57813      61625      63771     GICv3  30 Level     arch_timer
     14:          8          0          0          0     GICv3 108 Level     mbox-r5-0
     15:          7          0          0          0     GICv3 109 Level     mbox-c7x-0
     16:          4          0          0          0     GICv3 140 Level     mbox-mcu_r5-0
     17:          0          0          0          0     GICv3  23 Level     arm-pmu
     18:          0          0          0          0     GICv3 131 Level     pinctrl
     19:       6441          0          0          0     GICv3  66 Level     4d000000.mailbox thr_012
     28:       3461          0          0          0     GICv3 193 Level     20000000.i2c
     29:         10          0          0          0     GICv3 194 Level     20010000.i2c
     30:          0          0          0          0     GICv3 195 Level     20020000.i2c
    119:          0          0       3275          0  MSI-INTA 1970707 Level     8000000.ethernet-tx0
    209:          0        251          0          0  MSI-INTA 1971731 Level     8000000.ethernet
    253:       1020          0          0          0     GICv3 210 Level     2800000.serial
    254:          0          0          0          0   pinctrl 456 Edge      2800000.serial:wakeup
    256:          0          0          0          0     GICv3 134 Level     8000000.ethernet
    260:        277          0          0          0     GICv3 165 Level     mmc0
    262:          0          0          0          0      GPIO   0 Edge    -davinci_gpio  tps6594-1-0x48
    440:          0          0          0          0  tps6594-1-0x48   0 Edge      buck1_ov
    441:          0          0          0          0  tps6594-1-0x48   1 Edge      buck1_uv
    442:          0          0          0          0  tps6594-1-0x48   2 Edge      buck1_sc
    443:          0          0          0          0  tps6594-1-0x48   3 Edge      buck1_ilim
    444:          0          0          0          0  tps6594-1-0x48   4 Edge      buck2_ov
    445:          0          0          0          0  tps6594-1-0x48   5 Edge      buck2_uv
    446:          0          0          0          0  tps6594-1-0x48   6 Edge      buck2_sc
    447:          0          0          0          0  tps6594-1-0x48   7 Edge      buck2_ilim
    448:          0          0          0          0  tps6594-1-0x48   8 Edge      buck3_ov
    449:          0          0          0          0  tps6594-1-0x48   9 Edge      buck3_uv
    450:          0          0          0          0  tps6594-1-0x48  10 Edge      buck3_sc
    451:          0          0          0          0  tps6594-1-0x48  11 Edge      buck3_ilim
    452:          0          0          0          0  tps6594-1-0x48  12 Edge      buck4_ov
    453:          0          0          0          0  tps6594-1-0x48  13 Edge      buck4_uv
    454:          0          0          0          0  tps6594-1-0x48  14 Edge      buck4_sc
    455:          0          0          0          0  tps6594-1-0x48  15 Edge      buck4_ilim
    456:          0          0          0          0  tps6594-1-0x48  16 Edge      buck5_ov
    457:          0          0          0          0  tps6594-1-0x48  17 Edge      buck5_uv
    458:          0          0          0          0  tps6594-1-0x48  18 Edge      buck5_sc
    459:          0          0          0          0  tps6594-1-0x48  19 Edge      buck5_ilim
    460:          0          0          0          0  tps6594-1-0x48  20 Edge      ldo1_ov
    461:          0          0          0          0  tps6594-1-0x48  21 Edge      ldo1_uv
    462:          0          0          0          0  tps6594-1-0x48  22 Edge      ldo1_sc
    463:          0          0          0          0  tps6594-1-0x48  23 Edge      ldo1_ilim
    464:          0          0          0          0  tps6594-1-0x48  24 Edge      ldo2_ov
    465:          0          0          0          0  tps6594-1-0x48  25 Edge      ldo2_uv
    466:          0          0          0          0  tps6594-1-0x48  26 Edge      ldo2_sc
    467:          0          0          0          0  tps6594-1-0x48  27 Edge      ldo2_ilim
    468:          0          0          0          0  tps6594-1-0x48  28 Edge      ldo3_ov
    469:          0          0          0          0  tps6594-1-0x48  29 Edge      ldo3_uv
    470:          0          0          0          0  tps6594-1-0x48  30 Edge      ldo3_sc
    471:          0          0          0          0  tps6594-1-0x48  31 Edge      ldo3_ilim
    472:          0          0          0          0  tps6594-1-0x48  32 Edge      ldo4_ov
    473:          0          0          0          0  tps6594-1-0x48  33 Edge      ldo4_uv
    474:          0          0          0          0  tps6594-1-0x48  34 Edge      ldo4_sc
    475:          0          0          0          0  tps6594-1-0x48  35 Edge      ldo4_ilim
    476:          0          0          0          0  tps6594-1-0x48  36 Edge      vcca_ov
    477:          0          0          0          0  tps6594-1-0x48  37 Edge      vcca_uv
    495:          0          0          0          0  tps6594-1-0x48  55 Edge      npwron_start
    496:          0          0          0          0  tps6594-1-0x48  56 Edge      enable
    497:          0          0          0          0  tps6594-1-0x48  57 Edge      fsd
    498:          0          0          0          0  tps6594-1-0x48  58 Edge      soft_reboot
    499:          0          0          0          0  tps6594-1-0x48  59 Edge      bist_pass
    500:          0          0          0          0  tps6594-1-0x48  60 Edge      ext_clk
    501:          0          0          0          0  tps6594-1-0x48  61 Edge      twarn
    502:          0          0          0          0  tps6594-1-0x48  62 Edge      tsd_ord
    503:          0          0          0          0  tps6594-1-0x48  63 Edge      bist_fail
    504:          0          0          0          0  tps6594-1-0x48  64 Edge      reg_crc_err
    505:          0          0          0          0  tps6594-1-0x48  65 Edge      recov_cnt
    506:          0          0          0          0  tps6594-1-0x48  66 Edge      spmi_err
    507:          0          0          0          0  tps6594-1-0x48  67 Edge      npwron_long
    508:          0          0          0          0  tps6594-1-0x48  68 Edge      nint_readback
    509:          0          0          0          0  tps6594-1-0x48  69 Edge      nrstout_readback
    510:          0          0          0          0  tps6594-1-0x48  70 Edge      tsd_imm
    511:          0          0          0          0  tps6594-1-0x48  71 Edge      vcca_ovp
    512:          0          0          0          0  tps6594-1-0x48  72 Edge      pfsm_err
    513:          0          0          0          0  tps6594-1-0x48  73 Edge      imm_shutdown
    514:          0          0          0          0  tps6594-1-0x48  74 Edge      ord_shutdown
    515:          0          0          0          0  tps6594-1-0x48  75 Edge      mcu_pwr_err
    516:          0          0          0          0  tps6594-1-0x48  76 Edge      soc_pwr_err
    517:          0          0          0          0  tps6594-1-0x48  77 Edge      comm_frm_err
    518:          0          0          0          0  tps6594-1-0x48  78 Edge      comm_crc_err
    519:          0          0          0          0  tps6594-1-0x48  79 Edge      comm_adr_err
    520:          0          0          0          0  tps6594-1-0x48  82 Edge      en_drv_readback
    521:          0          0          0          0  tps6594-1-0x48  83 Edge      nrstout_soc_readback
    522:          0          0          0          0  tps6594-1-0x48  84 Edge      esm_soc_pin
    523:          0          0          0          0  tps6594-1-0x48  85 Edge      esm_soc_fail
    524:          0          0          0          0  tps6594-1-0x48  86 Edge      esm_soc_rst
    528:      15780          0          0          0     GICv3 115 Level     mmc1
    529:          0          0          0          0     GICv3 204 Level     20100000.spi
    IPI0:       945       2888       1560       1053       Rescheduling interrupts
    IPI1:     15508      18889      16006      12400       Function call interrupts
    IPI2:         0          0          0          0       CPU stop interrupts
    IPI3:         0          0          0          0       CPU stop (for crash dump) interrupts
    IPI4:         0          0          0          0       Timer broadcast interrupts
    IPI5:       521        415        497        450       IRQ work interrupts
    IPI6:         0          0          0          0       CPU wake-up interrupts
    Err:          0
    
    root@am62axx-evm:/opt/edgeai-gst-apps# cat /proc/meminfo
    MemTotal:        3399916 kB
    MemFree:         2995832 kB
    MemAvailable:    3123008 kB
    Buffers:           11452 kB
    Cached:           246364 kB
    SwapCached:            0 kB
    Active:            45932 kB
    Inactive:         266112 kB
    Active(anon):        540 kB
    Inactive(anon):    62732 kB
    Active(file):      45392 kB
    Inactive(file):   203380 kB
    Unevictable:           0 kB
    Mlocked:               0 kB
    SwapTotal:             0 kB
    SwapFree:              0 kB
    Dirty:                 0 kB
    Writeback:             0 kB
    AnonPages:         53716 kB
    Mapped:            60048 kB
    Shmem:              9044 kB
    KReclaimable:      20324 kB
    Slab:              45064 kB
    SReclaimable:      20324 kB
    SUnreclaim:        24740 kB
    KernelStack:        2512 kB
    PageTables:         1896 kB
    SecPageTables:         0 kB
    NFS_Unstable:          0 kB
    Bounce:                0 kB
    WritebackTmp:          0 kB
    CommitLimit:     1699956 kB
    Committed_AS:     382704 kB
    VmallocTotal:   133143592960 kB
    VmallocUsed:        7060 kB
    VmallocChunk:          0 kB
    Percpu:             1216 kB
    HardwareCorrupted:     0 kB
    AnonHugePages:     16384 kB
    ShmemHugePages:        0 kB
    ShmemPmdMapped:        0 kB
    FileHugePages:         0 kB
    FilePmdMapped:         0 kB
    CmaTotal:         589824 kB
    CmaFree:          587556 kB
    HugePages_Total:       0
    HugePages_Free:        0
    HugePages_Rsvd:        0
    HugePages_Surp:        0
    Hugepagesize:       2048 kB
    Hugetlb:               0 kB
    

  • Hi Yogesh,

       I noticed there are some comments in drivers\media\platform\ti\j721e-csi2rx\j721e-csi2rx.c as below, would you please help to check whether you have implement the function to get frame id from mipi csi packet?

    Best Regards,

    David

  • David,

    I noticed there are some comments in drivers\media\platform\ti\j721e-csi2rx\j721e-csi2rx.c as below, would you please help to check whether you have implement the function to get frame id from mipi csi packet?

    No, getting frame id from mipi csi packets has not been implememted.

    Regards,

    Jianzhong

  • Hi All,

    Yogesh will help post the script on e2e in hours. 

    Hi Jai,

    can you help check can we provide the frame id in our CSI Driver? 

    BR,

    Biao 

  • Biao,

    I am not able to access my AM62A board remotely. So, I will just share the document that you can refer to now:
    cs.uwaterloo.ca/~brecht/servers/apic/SMP-affinity.txt

    Morning I will test the instructions and send.

    Thanks

    Yogesh 

  • Hi David,

    > would you please help to check whether you have implement the function to get frame id from mipi csi packet?

    This is not implemented in the driver, as this is done in cadence csi2rx IP for a single virtual channel at a time, so the multi-channel usecase of driver cannot use the counter for each virtual channel. Same was discussed in the upstream linux mailing lists when adding that TODO.

    You can follow Section 12.6.4.8.10 Stream Monitor Frame Capture Control in AM62A TRM, to program and read the frame monitor register in the cdns-csi2rx.c driver for Stream0 (which is routed to TI DMA)

    Optionally software can read back the frame count value from register
    (CSI_RX_IF_VBUS2APB_STREAM0_FCC_CTRL - CSI_RX_IF_VBUS2APB_STREAM3_FCC_CTRL).
    In order to do that, software will first need to enable the monitor control register.
    1. Define the same Virtual Channel as above and enable the
    monitor control register (CSI_RX_IF_VBUS2APB_STREAM0_MONITOR_CTRL -
    CSI_RX_IF_VBUS2APB_STREAM3_MONITOR_CTRL).
    a. [15] FRAME_MON_EN -> set to ‘1’
    b. [14:11] FRAME_MON_VC -> define VC used
    2. Read frame_counter containing the current frame number being processed from
    CSI_RX_IF_VBUS2APB_STREAM0_FCC_CTRL - CSI_RX_IF_VBUS2APB_STREAM3_FCC_CTRL (only
    when frame numbers are coming from the protocol, not when internal counter)
    a. [31:16] FRAME_COUNTER -> read frame number value

    Thanks,
    Jai

  • For IRQ balancing:

    1) Stop IRQ Balance

         systemctl stop irqbalanced

    2)  Change CPU affinity
        echo <bit mask of allowed cores> > /proc/irq/<irq number>/smp_affinity 

        Bit Mask
    Binary Hex CPU 0 0001 1 CPU 1 0010 2 CPU 2 0100 4 CPU 3 1000 8

    For eg. changing the below interrupt from CPU0 to CPU1

    247:       6915          0          0          0  MSI-INTA 13045760 Level     4e230000.dma-controller chan0

      echo 2  >  /proc/irq/247/smp_affinity 

    Thanks

    Yogesh

  • Hi Jianzhong, Yogesh,

    pls help review attach source code from ADAPS team, can you try to give some optimize suggestion to them? 

    hawk_concurrency_test_403B.tar.gz

    Hi Jai,

    Thanks for your response here, but can you help implement this feature Frame ID in CSI driver side? with this, we can clearly prove there is frame drop or not. can you help have a test on SDK9.1, and share some patch to enable this feature?

    BR,

    Biao 

  • Hi Jai,

       I noticed this status is 0x33 or 0x05 when a DMA error happens, would you please tell me which register I should refer to for this error, what they mean when it is 0x33 or 0x05.

    Thanks

    David

  • Hi Yogesh,

      I tried your suggested command, but I met the error as below, please help to check the failure reason.

    Thanks

    David

  • David,

    Seems this IRQ is tied to CPU0 and cannot be changed.

    For task priority please refer:

    Linux commands: How to manipulate process priority  | Enable Sysadmin (redhat.com)

    For CPU Affinity for task please refer to: 

    Set Processor Affinity on Linux: A Guide to Optimizing CPU Performance - Byte Bite Bit

    Thanks

    Yogesh

  • Seems this IRQ is tied to CPU0 and cannot be changed.

    Hi Yogesh,

        Really?  But I used the following command, it seems work? Please help to double confirm? And tell us how to implement IRQ balance with code or script instead of manual input commands.

    systemctl stop irqbalanced
    echo 2 > /proc/irq/248/smp_affinity
    echo 4 > /proc/irq/249/smp_affinity
    echo 8 > /proc/irq/250/smp_affinity

    then I cat /proc/intterupts, I can see some intterupts are handled by CPU1-CPU3.

    Thanks

    David

  • Hi David,

    > I noticed this status is 0x33 or 0x05 when a DMA error happens, would you please tell me which register I should refer to for this error, what they mean when it is 0x33 or 0x05.

    These are DMA TR response errors. Please note the errors are in decimal, i.e. 33 and 5 which in hex is 0x21 (CBA error) and 0x5 (Short packet)
    You can refer to the TRM section "11.1.3.4 Transfer Response Record" for the detailed meaning.

    These errors are seen when sensor is streaming over CSI but the DMA was starved of empty buffers. When finally an empty buffer is pushed to DMA for filling, we see short packet or CBA bus error as due to starvation the StartOfFrame/EndOfFrame signals were missed so only partial data is copied over.

    My suspicion is the application is taking too much time to process frame data, and thus not queuing empty buffers (using VIDIOC_QBUF/VIDIOC_DQBUF) within the sensor framerate limit. This will lead to buffer pool becoming empty, stalling DMA and causing starvation.

    This will also explain why you were seeing multiple DMA drain/restart_dma() logs before the patch was shared to disable the logs, as CSI driver will try to drain its internal FIFOs when starvation happens to clear out old stale data.

    Can you profile how long the application is taking to process each frame - i.e. time between dqbuf and qbuf ioctls?


  • David,

    You can use the function irq_set_affinity_hint() to set the IRQ CPU affinity.

    For setting irq cpu affinity to CPU1

    #define CPU_AFFINITY_MASK 0x1

      /* setting cpu affinity to cpu 1 */
     irq_set_affinity_hint(irq, cpumask_of(CPU_AFFINITY_MASK));

    where irq is handler for interrupt 248

    Thanks

    Yogesh

  • Hi Yogesh,

       Thanks for your suggestion.  I tried to implement a function as below, then I call it in my .s_stream callback function. 

    static void set_dma_irq_balance(void)
    {
    int dma_irq = 248;
    unsigned int cpu = 1;

    irq_set_affinity_hint(dma_irq,get_cpu_mask(cpu));
    irq_set_affinity_hint(dma_irq+1,get_cpu_mask(cpu+1));
    irq_set_affinity_hint(dma_irq+2,get_cpu_mask(cpu+2));
    }

    I have 2 new questions:

    1. As you see, I hard-code the dma_irq in set_dma_irq_balance(), it may be not so good. Is there any API function to get the DMA's IRQ number? This IRQ number must be 247,248,249,250? 

    2. I tested it with my modified driver. It seems work at the begin, there are some IRQ handled by CPU1-CPU3 as below photo.

      But after a while, when I cat /proc/interrupts once more, it seems all IRQ are changed back to CPU0, please see the following photo.

    Please help check them.

    Thanks

    David

  • Hi Yogesh,

       One more question: Is it possible for Am62ax SoC to configure the DMA IRQ balance in start-up script or DTS file. If yes, it may be better than in my driver code.

    Thanks

    David

  • Hi Yogesh,

        Since DMA driver (ti_udma.c) is the owner of these DMA interrupt. I think it is better if you can add some code in ti_udma.c to handle the IRQ balance, then send us a patch for this.

    Thanks

    David

  • I tried to the following steps, but it doesn't work.

    // step 1

    root@am62axx-evm:/etc/init.d# cat dma_irq_balance.sh

    #!/bin/sh

     

    systemctl stop irqbalanced

    echo 2  >  /proc/irq/248/smp_affinity

    echo 4  >  /proc/irq/249/smp_affinity

    echo 8  >  /proc/irq/250/smp_affinity

    echo "irqbalance service is stopped, DMA irq is configured to 4 CPUs..."

    // step 2

    root@am62axx-evm:/lib/systemd/system# cat dma_irq_balance.service

    [Unit]

    Description=dma irqbalance daemon

    ConditionCPUs=>1

     

    [Service]

    ExecStart=/etc/init.d/dma_irq_balance.sh

     

    [Install]

    WantedBy=multi-user.target

    // step 3

    ln -s /lib/systemd/system/dma_irq_balance.service dma_irq_balance.service

    chmod +x /etc/init.d/dma_irq_balance.sh

    // manually execute the script

    /etc/init.d/dma_irq_balance.sh 

    system will output the following error

  • Hi Yogesh,

       It seems irqbalance is configured correctly, so that it bring some reverse action (DMA irq is focused on CPU0), would you please help to modify its configure , or add some code in ti_udma.c to balance the DMA irq?

    Thanks

    David

  • David,

    Please give me some time to experiment and get back.

    Thanks

    Yogesh

  • Hi Yogesh,

       Sure, thanks for your support and help.

    Thanks

    David

  • Hi Yogesh,

        Do you have any update for this issue?

    Thanks

    David

  • Hi David,

    Today and tomorrow is still weekend in US, pls expect some delay. Thanks for your patience.

    BR,

    Biao 

  • Hi David,

        But after a while, when I cat /proc/interrupts once more, it seems all IRQ are changed back to CPU0, please see the following photo.

    Could you also help check that is there any other irq_set_affinity_hint functions might be called in your applications that may change the CPU back to CPU0 please?

    Thanks,

    Kevin

  • Hi Kevin,

       No, we didn't do so, in fact, we even didn't know the irq balance before, we don't change code for this, but I think if IRQ balance service is configured correctly, it should not focus on the first CPU.

    Thanks

    David

  • Hi David,

    Thanks for your reply!

    May I confirm that if you manually input the following commands, it will work? Also will the CPU still be changed back to CPU0 after running some time in this case?

    systemctl stop irqbalanced
    echo 2 > /proc/irq/248/smp_affinity
    echo 4 > /proc/irq/249/smp_affinity
    echo 8 > /proc/irq/250/smp_affinity

    However, if you make them into a script, and when you run the script it will have the below error right?

    There is also a guide to manually set smp irq affinity below.

    https://www.kernel.org/doc/html/next/core-api/irq/irq-affinity.html#:~:text=%2Fproc%2Firq%2Fdefault_smp_affinity%20specifies,Default%20mask%20is%200xffffffff

    Kind Regards,

    Kevin

  • Yes, It is.  But it is best if you can help to tell us the correct configure for IRQ balance. From the name, it should be designed to balance the IRQ load, but the current result is oppsite, so I guess the current configure maybe have some issue(We didn't change it). Please help to correct it.

    Best Regards,

    David

  • Hi David,

    Thanks for the clarification!

    For the internal IRQ balance configuration I need to check with Yogesh for the latest feedback.

    As for the problem you meet when run the script, may I know have you tried to run the script after board boots up? Do you see the same problem after board boot up?

    Thanks,

    Kevin