RE: AM62L: Testing Codesys

xi he

This thread is a followup to thread
AM62L: AM62L Real-time Performance Issue

Hi Nick，

Thank you very much for your support.

I have applied the above DDR QoS configuration in the actual operating scenario, where Codesys runs with a 1ms cycle and controls four motors. After a 6-hour test, the jitter value displayed on the Codesys host ranged from -132 to 139. Unfortunately, I did not observe any performance improvement. Our project will halt motor control and stop data collection once the jitter exceeds the threshold of [-100, 100].

Based on the cyclictest data you provided, the configuration with -i 1000 shows significant optimization. The jitter has dropped from over 200 microseconds to above 100 microseconds.

Regards，

5 days ago

0 Kevin Peng 5 days ago

TI__Expert 4768 points

Hi Xi,

Our internal Ethercat expert has the following 3 points may need align with you, thanks.

For the A53 core Codesys is running, are there any other peripheral interrupts (apart from ethernet IRQ) on this core also?
Are Ethernet IRQ and Codesys running on the same A53 core?
Are Ethernet IRQ threads priority the same as Codesys EtherCAT_Task?

Thanks,

Kevin

0 xi he 5 days ago in reply to Kevin Peng

Prodigy 20 points

Hi Kevin,

There are eight Ethernet TX interrupts in the CODESYS application scenario. Four of them are assigned to CPU0, which shares the same core as CODESYS, while the other four are on CPU1, running alongside the EtherCAT task.

All interrupt threads have a priority of -51, and the EtherCAT task thread runs at priority -57. Apart from these, only the arch_timer interrupt is observed, with no other peripheral interrupts present.

We also tried pinning all Ethernet interrupts to CPU0, but tests showed this brought no improvement to real-time performance.

0 Daolin Qiu 4 days ago in reply to xi he

TI__Mastermind 19665 points

Hi Xi,

1. How did youverify the 4 TX interrupts are truly mapped to port connected to EtherCAT network and other 4 TX interrupts mapped to the other port?
2. Can you try setting all RX and TX interrupts to same priority as EtherCAT_Task
3. How are you mapping the RX interrupt(s)?

-Daolin

0 xi he 1 day ago in reply to Daolin Qiu

Prodigy 20 points

Hi Daolin,

The previous description regarding interrupt relationships was incorrect. I have reorganized my interrupt mapping situation. Currently, under my CPU, there are 8 ethernet-tx interrupts and 1 ethernet-rx interrupt (corresponding to two hardware network ports: ethernet & ethercat). I confirmed this by connecting only ethercat or only ethernet. Under ethercat business functions, only the tx6 interrupt is frequently generated, so I bound it to cpu1 together with the ethercat task and adjusted it to the same priority of -57.

As for the ethernet-rx interrupt, it is generated when either the ethernet port or the ethercat port is plugged in, so I have bound it to cpu0 with a priority of -51.

I bound the other ethernet interrupts to cpu0 with unadjusted priority, all remaining at -51. I will continue running until tomorrow to see the results of this modification.

By the way, regarding the reason for changing my CODESYS application from 1ms 4-axis to 8-axis: our product's ultimate goal is 1ms 8-axis control. Previously, when verifying jitter performance with 4-axis, the results were not satisfactory, so we didn't increase the load to the final target. Now, after configuring core-level DDR QoS and the kernel configuration CONFIG_VIRT_CPU_ACCOUNTING_GEN, the CODESYS jitter has improved to [-89, 90] (tested over 24 hours), so I switched to the final target of 8-axis. However, the above configuration doesn't show significant optimization for 8-axis; the jitter situation is similar to that under cyclictest simulated load.

Regards,

0 Daolin Qiu 20 hours ago in reply to xi he

TI__Mastermind 19665 points

Hello Xi,

xi he said:
By the way, regarding the reason for changing my CODESYS application from 1ms 4-axis to 8-axis: our product's ultimate goal is 1ms 8-axis control. Previously, when verifying jitter performance with 4-axis, the results were not satisfactory, so we didn't increase the load to the final target. Now, after configuring core-level DDR QoS and the kernel configuration CONFIG_VIRT_CPU_ACCOUNTING_GEN, the CODESYS jitter has improved to [-89, 90] (tested over 24 hours), so I switched to the final target of 8-axis.

Thanks for this clarification. Let's see the results of your IRQ mapping to see if that helps improve jitter.

xi he said:
I confirmed this by connecting only ethercat or only ethernet. Under ethercat business functions, only the tx6 interrupt is frequently generated, so I bound it to cpu1 together with the ethercat task and adjusted it to the same priority of -57.

Please ensure that you have disabled irqbalance so that the irq mapping doesn't get changed by the OS during runtime. If irqbalance is still enabled, even if you mapped irqs to a specific cpu, the OS could still change the core the irq is run on.

xi he said:
As for the ethernet-rx interrupt, it is generated when either the ethernet port or the ethercat port is plugged in, so I have bound it to cpu0 with a priority of -51.

One note about your IRQ mapping, since ethercat port is also using the same ethernet-rx interrupt, mapping to non-ethercat cpu core (cpu1) might not help with jitter. There is a way to create two separate RX flows/queues to introduce two separate ethernet-rx interrupts (see below). Once you enable two ethernet-rx interrupts, you can then bound the rx interrupt for ethercat port to cpu1 (with same priority as EtherCAT_Task -57) and give it a test.

ethtool -l eth0
ethtool -l eth1
ifconfig eth0 down
ifconfig eth1 down
sleep 2
ethtool -L eth1 rx 2
ifconfig eth0 up
ifconfig eth1 up
ethtool -l eth0
ethtool -l eth1
sleep 2
cat /proc/interrupts | grep rx

-Daolin

0 xi he 13 hours ago in reply to Daolin Qiu

Prodigy 20 points

Hi Daolin,

Yesterday's test results: 1ms 8-axis, test duration 16h, jitter range [-136, 136].

I have confirmed that irqbalance is not configured;

Regarding the creation of ethernet-rx queues, I encountered a problem. When attempting to generate two rx queues, the kernel reports an error, and the log situation is as follows:

root@am62xx-evm:/# ethtool -l eth1
Channel parameters for eth1:
Pre-set maximums:
RX: 8
TX: 8
Other: n/a
Combined: n/a
Current hardware settings:
RX: 1
TX: 8
Other: n/a
Combined: n/a
root@am62xx-evm:/# ethtool -l eth0
Channel parameters for eth0:
Pre-set maximums:
RX: 8
TX: 8
Other: n/a
Combined: n/a
Current hardware settings:
RX: 1
TX: 8
Other: n/a
Combined: n/a

root@am62xx-evm:/# ifconfig eth0 down
[63581.120218] am65-cpsw-nuss 8000000.ethernet eth0: Link is Down
root@am62xx-evm:/# ifconfig eth1 down
[63584.617452] am65-cpsw-nuss 8000000.ethernet eth1: Link is Down
root@am62xx-evm:/# 2026-01-13T20:10:25.990Z: Cmp=CmpRouter, Class=1, Error=0, Info=5, pszInfo= Network interface <interface>ether 2</interface> unregistered
root@am62xx-evm:/# ethtool -L eth1 rx 2
[63604.110907] __cma_alloc: 11 callbacks suppressed
[63604.110925] cma: __cma_alloc: reserved: alloc failed, req-size: 2 pages, ret: -38
[63604.110931] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.137523] cma: __cma_alloc: reserved: alloc failed, req-size: 17 pages, ret: -38
[63604.137547] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.157844] cma: __cma_alloc: reserved: alloc failed, req-size: 2 pages, ret: -38
[63604.157867] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.173800] cma: __cma_alloc: reserved: alloc failed, req-size: 17 pages, ret: -38
[63604.173818] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.195817] cma: __cma_alloc: reserved: alloc failed, req-size: 2 pages, ret: -38
[63604.195837] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.212817] cma: __cma_alloc: reserved: alloc failed, req-size: 17 pages, ret: -38
[63604.212836] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.232819] cma: __cma_alloc: reserved: alloc failed, req-size: 2 pages, ret: -38
[63604.232843] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.252837] cma: __cma_alloc: reserved: alloc failed, req-size: 17 pages, ret: -38
[63604.252856] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.274813] cma: __cma_alloc: reserved: alloc failed, req-size: 2 pages, ret: -38
[63604.274832] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.293814] cma: __cma_alloc: reserved: alloc failed, req-size: 17 pages, ret: -38
[63604.293834] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.315821] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.326816] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.341814] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.353867] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.366822] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.377814] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.393820] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.394190] am65-cpsw-nuss 8000000.ethernet: set new flow-id-base 96
[63604.412087] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.424816] cma: number of available pages: 8192@0=> 8192 free of 8192 total pages
[63604.424916] am65-cpsw-nuss 8000000.ethernet: Failed to get rx dma irq -22
netlink error: Invalid argument

Processors

Processors forum

RE: AM62L: Testing Codesys