• State TI Thinks Resolved
  • Locked Locked
  • Replies 7 replies
  • Answers 1 answer
  • Subscribers 31 subscribers
  • Views 392 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Original question:

AM2634: Does AM263 support self interrupt nesting?

AM263P4: enet_lwip_cpsw example: I2C ISR not triggering during initialization when CPU utilization is high

Huey Duong
Intellectual 290 points
Part Number: AM263P4
Other Parts Discussed in Thread: SYSCONFIG, DP83869, CCSTUDIO

Tool/software:

I have a TI sample project that mimics my company project scenario.

This may be an unlikely situation in the real world, but the fact that interrupts are not triggering may be indicating something is not properly set up with the interrupts that may cause errors down the road.

Basically I start of with TI's enet_lwip_cpsw project that sets up ethernet.   I added PWM0 to run @ 40kHz and generate and interrupt every 2 cycles (@20kHz). The ISR executes for 50% of the CPU execution time.  (This code will be moved to a nortos core that runs control code into an ISR.  But for now it is running on core 0 for testing).

Inside PWMISR a pin is toggled and a long delay is done.

Also note, the PWM is configured up/down count mode during initialization, so PWM and ISR will execute very early.

When I run this code, the enet initialization fails while trying to read the I2C EEPROM for MAC information.    

EnetBoard_getMacAddrList calls EEPROM_read() 2 times.  I2C_EEPROM_MAC_CTRL_OFFSET is successfull, but I2C_EEPROM_MAC_DATA_OFFSET fails during SemaphoreP_pend in I2C_transfer().

Here are some captures of registers from both calls to EEPROM_read():

  1. EEPROM_read(I2C_EEPROM_MAC_CTRL_OFFSET) before SemaphoreP_pend.
  2. EEPROM_read(I2C_EEPROM_MAC_CTRL_OFFSET) after SemaphoreP_pend.
  3. EEPROM_read(I2C_EEPROM_MAC_DATA_OFFSET) before  SemaphoreP_pend.   !!! never gets past SemaphoreP_pend
  4. Halted CPU to capture I2C registers.

I am surprised that I2C is not able to trigger the second time.  Curious if it has something to do with I2C ISRs overflowing.

Any help would be appreciated to determine the exact cause and how to fix this situation.    I am concerned it has to do with interrupt setup, that will become an intermittent issue later down the road.

Regards.

==================== How to reproduce the issue =======================

  1. Load project: mcu_plus_sdk_am263px_10_01_00_31\source\networking\enet\core\examples\lwip\enet_lwip_cpsw\am263px-cc\r5fss0-0_freertos
  2. overwrite the .sysconfig and main.c file.
  3. Run the project
  4. See non working output below

1106.test.zip

===================================LOGS=====================

### working output if PWM ISR is not using 50% CPU utilization when enet reads EEPROM

Starting NULL Bootloader ...
KPI_DATA: [BOOTLOADER_PROFILE] Boot Media : undefined
KPI_DATA: [BOOTLOADER_PROFILE] Boot Image Size : 0 KB
KPI_DATA: [BOOTLOADER_PROFILE] Cores present :
KPI_DATA: [BOOTLOADER PROFILE] System_init : 378us
KPI_DATA: [BOOTLOADER PROFILE] Drivers_open : 50us
KPI_DATA: [BOOTLOADER PROFILE] LoadHsmRtFw : 8584us
KPI_DATA: [BOOTLOADER PROFILE] SBL End : 3043us
KPI_DATA: [BOOTLOADER_PROFILE] SBL Total Time Taken : 12057us

NULL Bootloader Execution Complete...
==========================
ENET LWIP App
==========================
EnetAppUtils_reduceCoreMacAllocation: Reduced Mac Address Allocation for CoreId:0 From 4 To 2
Link Status Changed. PHY: 0x0, state: up
Open MAC port 2
EnetPhy_bindDriver: PHY 0: OUI:080028 Model:0f Ver:03 <-> 'DP83869' : OK
PHY 0 is alive
Starting lwIP, local interface IP is dhcp-enabled
[LWIPIF_LWIP] NETIF INIT SUCCESS
Host MAC address-0 : 70:ff:76:1f:61:88
[LWIPIF_LWIP] Enet has been started successfully
[0]status_callback==UP, local interface IP is 0.0.0.0
UDP server listening on port 5001
Cpsw_handleLinkUp: Port 2: Link up: 1-Gbps Full-Duplex
MAC Port 2: link up
[0] link_callback==UP
5. 69s : CPU load = 3.63 %

### NON WORKING output if PWM ISR IS using 50% CPU utilization when enet reads EEPROM

### LWIP init halted trying to read I2C EEPROM

Starting NULL Bootloader ...
KPI_DATA: [BOOTLOADER_PROFILE] Boot Media : undefined
KPI_DATA: [BOOTLOADER_PROFILE] Boot Image Size : 0 KB
KPI_DATA: [BOOTLOADER_PROFILE] Cores present :
KPI_DATA: [BOOTLOADER PROFILE] System_init : 378us
KPI_DATA: [BOOTLOADER PROFILE] Drivers_open : 50us
KPI_DATA: [BOOTLOADER PROFILE] LoadHsmRtFw : 8584us
KPI_DATA: [BOOTLOADER PROFILE] SBL End : 3042us
KPI_DATA: [BOOTLOADER_PROFILE] SBL Total Time Taken : 12054us

NULL Bootloader Execution Complete...
==========================
ENET LWIP App
==========================

  • 0
    TI__Genius 16412 points

    Hi Huey,

    I've started looking into this. I'll share an update soon.

    Regards,
    Shaunak

  • 0
    TI__Genius 16412 points

    Hi Huey,

    While im still working to root-case this (probably some issue with interrupt bits not clearing before high priority PWM interrupt comes in), I do have a workaround in mind.

    Incase you don't have a hard requirement of reading MAC addresses from EEPROM, you can also manually assign MAC address from Syscfg Enet(CPSW). This way the EEPROM reads can be avoided and this issue can be bypassed. Will only work if don't have a hard requirement of reading MAC address from EEPROM.

    Regards,
    Shaunak

  • 0 in reply to Shaunak Deshpande
    TI__Genius 16412 points

    Also from the 4th image, the Value of ICSTR register shows that there is an I2C overrun on the receiving end (0x1C18). If possible can you once try to reduce the frequency of PWM ISRs so they CPU has more free time to handle I2C interrupts. My suspicion here is that the CPU does not have enough time to service the interrupts and is hitting overrun. Just a speculation, might be incorrect.

    Regards,
    Shaunak

  • 0
    TI__Genius 16412 points

    Hi Huey,

    I tried to reproduce the issue on my AM263PX-CC. I wasn't able to observe the enet initialization failing.

    Experiment-1 (40KHz, time base period = 2500) :

    Experiment-2 (4KHz, time base period = 25000): 

    - I did profile and make sure if the ISR was executing 25us similar to the screenshot shared above, even then I did not run into issues.

    - I also spoke with some experts from the controlSS team, they have successfully been able to run ePWM at very high frequencies as well (500KHz - 800KHz as part of OBC reference designs).I do not see any issue in running ePWM at higher speeds

    Just to make sure, there are no hardware modifications made to the board right? Asking since I haven't been able to reproduce this. 

    Regards,
    Shaunak

  • 0 in reply to Shaunak Deshpande
    Intellectual 290 points

    Hi Shaunak, please update the images, they look like broken links.

    I have resurrected the sample project and it looks like the release build works, but the debug still does not work.  Here are the updated instructions to recreate the failed project.

    Using CCS 20.1.1.8 and MCU SDK 10.2.0.15

    1. Import enet_lwip_cpsw example
    2. Right click on the following files and "exclude from build"
      1. main.c and example.syscfg
    3. Extract the following files into the project
      1. se_main.c and se_example.syscfg
      2. se_files.zip
    4. Change build configuration to Debug
    5. Build and run project
    6. Console should be stuck at "ENET LWIP APP" header
    7. GPIO65 / HSEC 86 / Breakout J21_18 should show waveform of when code enters PwmISR().  This should be 50%  (Release is smaller, hence the timing issue is not there)

    LMK if you still have issues reproducing the I2C overflow issue.  Thanks.

  • 0 in reply to Huey Duong
    TI__Genius 16412 points

    Hi Huey,

    I was able to reproduce using the above steps, thanks. I am looking into it now,

    Regards,
    Shaunak

  • 0 in reply to Shaunak Deshpande
    TI__Genius 16412 points

    Hi Huey,

    I performed some profiling experiments, and sharing the results below.

    1. I was able to reproduce your issue yesterday, running into the same enet init failing. I hit the I2C_Transfer function once, but wasn't able to reach that breakpoint later. I tried profiling around PwmISR function in your main.c file which does GPIO Toggle. On average i see the breakpoint being hit every 50us for PwmISR (This aligns with the waveforms shared above by you).

    2. With the above observation, it is clear that all the other I2C transactions need to be completed within the 25us window we have. I also tried profiling the I2C_Transfer function. In total, during Ethernet initialization there were 33 times the I2C_transfer breakpoint was hit, some transactions taking longer, others not so long. For ethernet initialization to complete, all these I2C transactions have to complete successfully. These transactions are for MAC address fetching from EEPROM as well as configuring the MAC Port-2 IO Expander (function call stacks attached below) on the AM263Px-CC.

    Now, the time between 2 subsequent I2C Transfers was as follows (profiled from CCSTUDIO internal clocks)

    From the above profiling for I2C_Transfers, the issue highlighted. Before the I2C Transaction completes, a PWM High priority interrupt is received and the I2C transactions are left incomplete, hitting an overflow on the I2C side. 

    Huey Duong said:
    I have resurrected the sample project and it looks like the release build works, but the debug still does not work.

    All the profiling shared above was done over a debug build. The release build optimizes code for speed and size. You do see release build working (both Ethernet and PWM ISR). This aligns with our speculation of I2C transactions not getting enough time to complete, and High priority PWM ISR interrupting it, and causing the ethernet init to fail at a system level. In case of release build, code will be optimized and I assume the I2C transactions don't take a long time (as seen in case of debug builds). Due to this release build works OR if you make sure there is enough time free (current free time is 25us), the application works.

    Now from a PWM point of view, this does not mean the AM263Px cannot run at higher frequencies. As mentioned previously, in some reference designs and tests we have tested in the range of 500KHz-800KHz (making sure the ISR is small and deterministic). The issue is only when you have some long initialization process (like Ethernet in Debug builds). Such are the only instances where you would observe the combined (Ethernet + PWM) system failing.

    What can be done further to reduce I2C overhead?

    1. When using Ethernet, you can assign the MAC address through Syscfg/ or override the ethernet code to not read MAC addr from EEPROM, reducing I2C Calls further.

    2. When designing a custom board, not having a complex IO Expander will reduce some I2C overhead

    Let me know if you have any more questions

    Regards,
    Shaunak