TMS570LS0332: FMC - uncorrectable error: bus1 and bus2 interfaces

Hello Sreekanth,

Can you please post the value of FEDACSTATUS register, and FUNC_ERR_ADD register? Do you access ECC and OTP memory region in your code?

Dear QJ Wang,

After testing for a while, I found the FUNC_ERR_ADD value. FUNC_ERR_ADD = 0xFFF8

I think this is the same problem specified in the following link. 

https://e2e.ti.com/support/microcontrollers/hercules/f/312/t/779080

As specified in the above link, we habve two executables in micro controller memory.

1. Boot loader (From 0x0000 to 0xFFFF)

2. Application (From 0x10000 to 0x2FFFF)

As you can see, FUNC_ERR_ADD = 0xFFF8 is in the boot loader memory.  The bootlooader sys_intvecs.asm fiile is as follows. 

resetEntry
b _c_int00
undefEntry
b #0xFFF8 ;undefEntry
svcEntry
b #0xFFF8 ;svcEntry
prefetchEntry
b #0xFFF8 ;prefetchEntry
dataEntry
b #0xFFF8 ;dataEntry
b #0xFFF8 ;phantomInterrupt
ldr pc,[pc,#-0x1b0] ;Hardware control of interrupts. no effect due to this (IRQ)
ldr pc,[pc,#-0x1b0] ;Need to be handled in application code. FIQ

The Application starts at 0x10000 and "sys_intvecs.asm" file contents are below.

resetEntry
b _c_int00
b _excpt_vec_udef_instr
b _svc
b _excpt_vec_abort_pref
b _excpt_vec_abort_data
b phantomInterrupt
ldr pc,[pc,#-0x1b0]
ldr pc,[pc,#-0x1b0]

If you see 0xFFF8 is used for all the exceptions in the bootloader. This allows the software to jump to application. Highlighted in yellow color.

We found this approach of jumping in the TI CAN bootloader example.  

Additional information -> Some files in the applications have different optimizations than the bootloader. I also believe that is causing the problem.

Thanks & Best regards,

Sreekanth challa

Hi,

Can you please comment on this? We are short of ideas from our side regarding this issue.

Thanks & best regards,

Sreekanth challa 

Hi QJ,

a while ago you have stated that you could reproduce the nERROR issues by the compiler optimization settings. Did you ever investigate and hear back from the compiler team? We see a strong correlation between the issue described and compiler optimizations. Thanks for your feeback.

http://e2e.ti.com/support/microcontrollers/hercules/f/312/t/705563?CCS-TMS570LS0332-Optimizations-in-CCS-Vs-Controller-ESM-interrupt-Abort-related-to-Flash-memory

-Stefan

Hello,

1. Is the application programmed to 0x10000? If the application is not located starting at 0x10000, the jumping instruction of bootloader will cause Abort, or any exception in bootloader will cause Abort too. To load the application to 0x10000, the liner command file for application has to be modified:

VECTORS (X) : origin=0x00010000 length=0x00000020
FLASH_CODE (RX) : origin=0x00010020 length=0x8000 - 0x20 fill=0xFFFFFFFF /*sector 8/9/10/11 are used for application */

2. The ECC values for the whole Flash banks must be programmed into the Flash before SECDED is enabled. This can be done by generating the correct values of the ECC in Linker command file of bootloader. Attached is the example.

2626.bl_link.cmd

3. The Cortex-R4/5 processor has a system control coprocessor implemented, the CP15. The CP15 offers the possibility to readout additional information about an abort. Registers in the CP15 that hold information about the cause of an abort are:

• Data Fault Status Register

• Auxiliary Fault Status Registers

• Data Fault Address Register

• Instruction Fault Address Register

Please check those registers in CCS register window:

4. Prefetch abort and Data abort

The CPU takes prefetch abort if CPU tries to execute an instruction from a protected or faulty memory location.

• The memory location is not implemented in the system,
• The memory region is protected by the MPU,
• An error is detected in the data by the ECC checking logic.

The CPU takes the data abort if data is read from or written to a protected or faulty memory location. This could be because of:

• The memory location is not implemented
• The memory location is read or write only in privileged mode (when processor is in User mode)
• The memory location is read or write protected by the MPU
• If an error is detected in the data by the ECC checking logic

Dear QJ,

Thanks for the response. Please see my comments on each of the points you specified.

1. Is the application programmed to 0x10000? If the application is not located starting at 0x10000, the jumping instruction of bootloader will cause Abort, or any exception in bootloader will cause Abort too. To load the application to 0x10000, the linker command file for application has to be modified:

VECTORS (X) : origin=0x00010000 length=0x00000020
FLASH_CODE (RX) : origin=0x00010020 length=0x8000 - 0x20 fill=0xFFFFFFFF /*sector 8/9/10/11 are used for application */

Yes. This is true. The application linker file is correctly specified. 

2. The ECC values for the whole Flash banks must be programmed into the Flash before SECDED is enabled. This can be done by generating the correct values of the ECC in Linker command file of bootloader. Attached is the example.

We are using a boot loader to flash the entire memory using FlashAPI. So, there is no problem there. 

What do you mean by the entire flash memory? For example, if the application image is only of 195KB, the boot loader only erases the required memory and re-programs it. While programming the image, the programming of flash memory is done 16 bytes at a time using Flash API with Fapi_AutoEccGeneration.

3. The Cortex-R4/5 processor has a system control coprocessor implemented, the CP15. The CP15 offers the possibility to read out additional information about an abort. Registers in the CP15 that hold information about the cause of an abort are:

We did not look into these registers. Since the problem occurs once a while when we let the controller run for hours. I will update you with these values once i get the issue again.  I found the FUNC_ERR_ADD value. FUNC_ERR_ADD = 0xFFF8.

4. Prefetch abort and Data abort

 We do understand them. 

I have one more point from my side. We do have a safety library included in our application. (V2.2.0). 

CCS Version       : 6.1.0.00104 

Compiler version: TI v5.2.9

If you see the following ticket this also has the flash error, with error address, 0xFFF8. I strongly believe this is exact same problem.

Thanks & Best regards,

Sreekanth challa

Dear QJ,

Adding a few more points to my last response.

1. We do not have the optimizations for the entire project. Only for selected files optimizations enabled as below.

2. The safety library itself does not work if I enable the optimizations for the entire project. This creates a prefetch abort. 

3. When I looked int the code of  "_excpt_vec_abort_data" function in "exception_handlers.c" file, the jump back to USER/SYSTEM mode (At the end of the abort handler function) is implemented using assembly code as shown in the attached picture. 

This may not be the case for all the optimizations. This may only work for no optimizations. 

Thanks & best regards,

Sreekanth challa

Hello,

The linker command file attached in my previous post can be used to calculate the ECC of the whole flash, and the ECC will programmed at loading time. 

Dear QJ,

Do you have any comments on this issue? We are very sure that the flash is programmed correctly. A complete CRC check is performed on the entire flash memory once on the start-up.  There seems to be no problem at all.

Thanks & best regards,

Sreekanth challa

Hello,

FUNC_ERR_ADD = 0xFFF8

Does your application read data from this address or write data to this address?

As I mentioned last time, please program the ECC for the whole flash using linker CMD file.

1. Using the linker cmd file attached here for your bootloader

4341.bl_link.txt

2. When loading your bootloader in CCS, please uncheck "Auto ECC Generation" and check "None" for Flash Verification settings.

Dear QJ,

Thanks for your response. The following are the answers to your questions in a blue color.

Does your application read data from this address or write data to this address?

  -> No. It does not and should not. As I mentioned in my one of the post above, 

The bootlooader sys_intvecs.asm fiile is as follows. 

resetEntry
b _c_int00
undefEntry
b #0xFFF8 ;undefEntry
svcEntry
b #0xFFF8 ;svcEntry
prefetchEntry
b #0xFFF8 ;prefetchEntry
dataEntry
b #0xFFF8 ;dataEntry
b #0xFFF8 ;phantomInterrupt
ldr pc,[pc,#-0x1b0] ;Hardware control of interrupts. no effect due to this (IRQ)
ldr pc,[pc,#-0x1b0] ;Need to be handled in application code. FIQ

So, it should jump to application start address. (i.e., 0x10000) in the case if exceptions or IRQ or FIQ.

I have one more question -> Do you think stack overflow can cause this issue? I do have a stack overflow check implemented in SW. There seems to be no problem there. 

Thanks & best regards,

Sreekanth challa

Sreekanth,

This double-bit ECC error in Flash memory should have also resulted in a prefetch / data abort. The best way to debug the root cause is to use the CPU's fault address and status registers while in the abort handler. You can use the abort mode link register to identify the instruction that caused the fault. That will give a fairly good understanding about the root cause of the abort and why the address 0xFFF8 is being accessed.

Dear Sunil,

We are working on the issue by monitoring the CPU fault registers. I would like keep this post open for a while.

Thanks & Best regards,

Sreekanth challa

Dear QJ,

How can I access the CP15 registers in the SW?  I do not see the registers in the controller memory map or in the reference manual. I assume these are registers of the ARM core. I do not find the starting address of the registers in ARM website as well.

Could you please comment on this. Many thanks in advance.

Best regards,

Sreekanth challa

Hello,

ARM cortex-Rx processor incorporates CP15 for system control. CP15 enables configuration of the Tightly-Coupled Memory (TCM), and protection unit (MPU), and provides system status information.

CP15 registers can only be accessed with MRC and MCR instructions in a privileged mode. The instruction bit pattern of the MCR and MRC instructions is shown in ARM TRM. 

http://infocenter.arm.com/help/topic/com.arm.doc.ddi0363g/DDI0363G_cortex_r4_r1p4_trm.pdf

You can read the CP15 registers through CCS->registers:

Dear QJ,

Sreekanth,

See the Cortex R4 reference manual for instructions on how to read from the fault address and status registers:

https://developer.arm.com/docs/ddi0363/g/system-control/register-descriptions/fault-status-and-address-registers