CCS/TMS570LC4357: CortexR5: File Loader: Memory write failed: Algorithm indicated it failed to erase bank 0, sector 0

Hi Mr Wang,
thanks for the tip but we have done exactly that.

"Algorithm indicated it failed to erase..." 

This erase algorithm, is this F021 code and in RAM? 
If you for example initialize the MPU wrong, can you land in this situation then?

Is there a way to make a project and load it to RAM which erases the Flash ignoring this error message?

Is there another application you can use which overrides this error message and tries to erase it anyway?

We have tried Uniflash but we get the same error message.

Kind regards,
Lars
Bombardier

Hi again QJ,
maybe I have something that may be a clue to the issue. 

HALCoGen was updated 18th of December to version 04.07.01. I downloaded and installed that version two weeks ago. 
At the same time we had some updates in our software along with new hardware/board. 

At the beginning everything worked but after some days we started getting this error message. 

We rolled back our updates but to no change. More MCUs became "locked" and giving "Algorithm indicated it failed to erase...". 
Even one of the previous boards became "locked"! 

Today we rolled back the HALCoGen to 04.07.00 and the problem disappeared (we loaded NEW MCUs, not the "locked" ones since it does not work).

I will also add that we have been used this particular MCU for about 2 years but we have never had this kind of problem. 

When you look at what the HALCoGen update consists of, there are among other things, an errata fix concerning the PLL and the F021 Flash.  

Maybe you can forward this information to the HALCoGen team. It's a mystery though. 

Kind regards,
Lars
Bombardier

Hi QJ,
as I said, it's a mystery and today we have made new discoveries and the HALCoGen is not involved anymore since we get the error with 04.07.00 as well. When I said F021 I meant that the update may concern F021 since the errata file had "F021" in its file name: HL_errata_SSWF021_45.c

But now we have discovered that some new code we have must cause this. We loaded the previous version 10 times yesterday and it worked. Then we loaded the updated version of the code and it worked. Today we tried to load the same application again and we get the error message. So it must be something in that code that causes this. Note that the ECC setup is the same in both versions.

MEMORY
{
    VECTORS   (X)  : origin=0x00000000 length=0x00000020 vfill = 0xffffffff
    FLASH0    (RX) : origin=0x00000020 length=0x001FFFE0 vfill = 0xffffffff
    FLASH1    (RX) : origin=0x00200000 length=0x00200000 vfill = 0xffffffff

    STACKS    (RW) : origin=0x08000000 length=0x00002b00
    RAM       (RW) : origin=0x08002b00 length=0x00075500
    SHAREDRAM (RW) : origin=0x08078000 length=0x00008000

    ECC_VEC   (R)  : origin=0xf0400000 length=0x4            ECC={ input_range=VECTORS }
    ECC_FLA0  (R)  : origin=0xf0400000 + 0x4 length=0x3FFFC  ECC={ input_range=FLASH0 }
    ECC_FLA1  (R)  : origin=0xf0440000 length=0x40000        ECC={ input_range=FLASH1 }
}

ECC
{
   algo_name : address_mask = 0xfffffff8
   hamming_mask = R4
   parity_mask = 0x0c
   mirroring = F021
}

I will also add that the HALCoGen MPU, RAM, PLL etc is the same. The only differences are in spi, sci, het and gio.

Does the problem only happen when you use the linker command file to generate the flash ECC? If you use the "Auto ECC Generation" in CCS, the code can be programmed to the flash without any problem, right?

No, we get the same error with a project using "Auto ECC Generation" and I think the reason for that is that the application currently running in the MCU is preventing the access in some way.

What could possibly cause this? We'll continue to look for the cause.

Is there a way to load a Flash erase program into SRAM and execute it so the malicious code disappears?

Thanks for the help so far.

Kind regards,
Lars

Hi QJ again,
we really need help with this issue now. We are a team who are going to work in the weekend. 

Could you forward this issue to someone who (maybe Anthony or Chuck?) could answer why we get the error message: 

"Memory write failed: Algorithm indicated it failed to erase bank 0, sector 0"

Then we maybe can go forward. 

We've tried to load a flash erase program into SRAM but we cannot execute it. 
After we've connected to the target we can see that the PC points in RAM but we cannot run it. 

Is there some register that needs to be updated?? 

Kind regards,
Lars
Bombardier Transportation
RCS

Lars von Wachenfeldt said:

Is there a way to load a Flash erase program into SRAM and execute it so the malicious code disappears?

- HALCoGen v04.07.00
- F021 Flash API v02.01.01
- TI ARM compiler v18.1.5
- CCS 8.3

The program runs entirely in SRAM and for the main and EEPROM flash banks:

1. Checks if the bank is blank.

2. If the bank is not blank, then erases the bank and then verifies that the bank is blank.

To get the program to run from SRAM took a HalCoGen generated project and made the following modifications:

a. In the target configuration file specify a modified tms570lc43xx.gel which has a OnPreFileLoaded() function which issues a GEL_Reset prior to loading the program. This disables the MPU from any previous program, which may have marked the SRAM memory region as non-executable. Disabling the MPU then allows the HALCoGen start-up code to execute from SRAM.

b. Disable the _memInit_() function call in the HALGoGen _c_int00() start up code. This is because since the program is running in SRAM, the act of initializing the SRAM would corrupt the running program.

c. In the HL_sys_link.cmd linker command script modify the SECTIONS so the program runs in SRAM.

d. In the HALCoGen R5-MPU-PMU settings give execute permission to SRAM, since the program is running in SRAM.

e. In the HALCoGen R5-MPU-PMU settings disable the cache, since with the cache enabled the flash blank check was failing even after the flash had been erased. Since the program doesn't need performance, disabling the cache globally was simpler than adding cache invalidate operations.

The HL_sys_main.c file contains the code which uses F021 Flash API to ensure all flash banks are erased.

The program has been tested on a LAUNCHXL2-570LC43 by:

1. Load a program which executes from flash, using both main flash banks 0 and 1, and also uses the FEE driver to program bank 7.

2. Load and run the TMS570LC43x_flash_erase program which reports that it has erased all flash banks, and the banks are now blank, as indicated by the following output on the CIO console:

[CortexR5] Main bank 0 has been erased and is now blank
Main bank 1 has been erased and is now blank
EEPROM bank 7 has been erased and is now blank

3. Load and run the TMS570LC43x_flash_erase program again. On this run all flash banks are still in the erased state, as indicated by the following output on the CIO console:

[CortexR5] Main bank 0 is blank
Main bank 1 is blank
EEPROM bank 7 is blank

If the program failed to erase the flash, it will report other messages.

TMS570LC43x_flash_erase.zip

Hi Mr. Gillon,
I would like to start with thanking you at TI for the superb service!  

I immediately started importing your project, built and loaded it but stumbled on a problem with the debugger(?). 

I use Spectrum Digital XDS560v2 PRO TRACE. 

The program loads but does not seem to run. If I set a break point at _coreInitRegisters_(); it stops there but I cannot step or run. 
It just stands at this position. 

Thanks again.

Kind regards,
Lars

Hi again, I forgot to say I've used your GEL file. On both DAP and CortexR5. 

Kind regards,
Lars

Lars von Wachenfeldt said:

I use Spectrum Digital XDS560v2 PRO TRACE. 

The program loads but does not seem to run. If I set a break point at _coreInitRegisters_(); it stops there but I cannot step or run. 
It just stands at this position.

To avoid that, in the example project I placed a modified tms570lc43xx.gel file which issues a reset prior to loading the program.

When the Debug Probe in the target configuration is changed, the GEL scripts reset to the default.

In your target configuration, for the CortexR5 use the Browse button for the initialization script and select the tms570lc43xx.gel file which was contained in my example:

[The above screen shot has a XDS110 Debug Probe since that it is what my hardware had]

Lars von Wachenfeldt said:

Hi again, I forgot to say I've used your GEL file. On both DAP and CortexR5. 

Chester Gillon said:

If it is halting there, the MPU is probably still enabled from a previous program with the SRAM region not marked as executable.

To avoid that, in the example project I placed a modified tms570lc43xx.gel file which issues a reset prior to loading the program.

If you set the "Reset the target on a connect" option in the Debug project properties does that help?

Hi,
no, that didn't help. As you could see we have a board with two MCUs. I've loaded the program on a board where MCU 2 is not "locked". The program runs perfect. MCU 1 does not.

Where I'm at right now we only have the XDS560 but tomorrow I can test with XDS110 and XDS200. 

Kind regards,
Lars

Lars von Wachenfeldt said:

I've loaded the program on a board where MCU 2 is not "locked". The program runs perfect. MCU 1 does not.

a. Under the debug project properties untick Auto Run and Launch Options -> Auto Run Options -> On a program load or reset. I.e. to stop at the entry point when loading a program.

b. Start a debug session.

c. If the program stops at _c_init00() check the "MPU Enable" option under ARM Advanced Features:

d. If the MPU Enable is ticked, try unticking the option and then attempting to run the  program.

I tried this without the modified GEL script, and it allowed the program to run.

Hi,
currently I have another problem. It can't build for some reason. I got this message after I've installed 8.3.0 but after some reboot it disappeared. Now I got the same message back again. And that is only for your project. Very strange, Sigh! 

I'll will test your latest suggestion when this annoying problem goes away... 

I first thought you were a TI employee since I "only" saw Guru. Anyway, thanks very much for your effort. 

Kind regards,
Lars

I went around that problem by selecting a fixed lib:

But "MPU enabled/disabled" did not solve the problem unfortunately. 

Tried to erase flash again but get this message still:

This was really a tricky one. 

Lars

Hi again QJ,
if you have had time to read my conversation with Mr Chester Gillon yesterday and where he tried to help me (many thanks again), we still get this error message from the Flash operation "Erase". 

Could you please forward this to someone who could help us and point in some direction?  

Why do we get this error message? 

Kind regards,
Lars

Lars von Wachenfeldt said:

Why do we get this error message?

Can you try and enable Debug Server Logging and attach the log file for the failure case, as this might give some more information.

Hi,
thanks for the idea. I turned on the logging and here's an excerpt from the file where it's firing the error message. I also have attached the whole file.

0x000042CC 129702 3 CortexR5 FLASH C: IFlashDevice::PerformOperation( Erase )
0x000042CC 129702 3 CortexR5 FLASH C: FlashProgrammerUtility::ReportInfo( Erasing Flash memory... )
0x000042CC 129702 3  COM_DBG_IF C: class dsDebugger::OnInfo()
0x000042CC 129702 3  XPCOM C: ( (dsIStringEventCallback*)42D66C90 )->onEvent( CortexR5: Erasing Flash memory...
 )
0x000042CC 129702 3  XPCOM R: ( (dsIStringEventCallback*)42D66C90 )->onEvent( CortexR5: Erasing Flash memory...
 ) = 0x00000000
0x000042CC 129702 3  COM_DBG_IF R: class dsDebugger::OnInfo()
0x000042CC 129702 3 CortexR5 POLL C: Firing DSP_TASK_ERROR_MESSAGE_INFO to all DSP_USER's
0x000042CC 129702 3  COM_DBG_IF C: class DSP_CO_USER::OnInfo()
0x000042CC 129702 3  XPCOM C: ( (dsIStringEventCallback*)53412C68 )->onEvent( CortexR5: Erasing Flash memory...
 )
0x000042CC 129702 3  XPCOM R: ( (dsIStringEventCallback*)53412C68 )->onEvent( CortexR5: Erasing Flash memory...
 ) = 0x00000000
0x000042CC 129702 3  COM_DBG_IF R: class DSP_CO_USER::OnInfo()
0x000042CC 129702 3 CortexR5 POLL C: Firing of DSP_TASK_ERROR_MESSAGE_INFO complete
0x000042CC 129702 3 CortexR5 FLASH R: FlashProgrammerUtility::ReportInfo( Erasing Flash memory... ) = (null)
0x000042CC 129702 3 CortexR5 FLASH C: FlashProgrammerUtility::LookupReg( PC, ??? )
0x000042CC 129702 3 CortexR5 FLASH R: FlashProgrammerUtility::LookupReg( PC, 527 ) = (null)
0x000042CC 129702 3 CortexR5 FLASH C: FlashProgrammerUtility::ReadRegister( 527, ??? )
0x000042CC 129702 3 CortexR5 POLL D: New request of type DSP_RQ_READ_MEM added to polling loop by class DSTargetAccess
0x00002928 129702 3 CortexR5 POLL C: Polling with state STATE_IDLE and status EVENT_DSP_HALT
0x00002928 129702 3  SETUP I: SELECT Registers.dbid AS dbid, Register_Cache.Name AS Name, Register_Cache.Alias AS Alias, Register_Cache.Address AS Offset, Register_Cache.Page AS Page, Registers.Width AS Width, Register_Cache.Module AS Module, Registers.Description AS Description, Register_Cache.Hidden AS Hidden, Register_Cache.Is_Array_Element AS Is_Array_Element, Registers.Halted_Read_Access AS Halted_Read_Access, Registers.Halted_Write_Access AS Halted_Write_Access, Registers.Running_Read_Access AS Running_Read_Access, Registers.Running_Write_Access AS Running_Write_Access FROM Register_Cache, Registers WHERE Register_Cache.Register=Registers.dbid AND Register_Cache.Address>=-9223372036854775281 AND Register_Cache.Address<=-9223372036854775281 AND Register_Cache.Page=121 AND Register_Cache.Processor=21
0x00002928 129702 4 CortexR5 POLL I: Yielding to other threads

Wonder if the problem is the same as we have, the code cannot run?

br
Lars

4353.ds.log

Sorry, forgot to say the log file is from when I clicked the Erase flash button.

Lars von Wachenfeldt said:

I turned on the logging and here's an excerpt from the file where it's firing the error message. I also have attached the whole file.

0x000042CC 132985 3 CortexR5 FLASH R: IFlashDevice::PerformOperation( Erase ) = Algorithm indicated it failed to erase bank 0, sector 0
0x00002928 132985 3 CortexR5 POLL C: Polling with state STATE_READ and status EVENT_DSP_HALT

And prior to that error is the following which is writing the erase algorithm to the device:

0x000042CC 132704 3 CortexR5 FLASH R: FlashProgrammerUtility::WriteAlgorithmToMemory( erasew.alg, f021_R4_be_l2fmc.nfl, C:\ti\ccsv8\ccs_base\DebugServer\bin\..\..\hercules\flash\libraries\ ) = (null)

From a quick look the <ccs_install_root>/ccsv8/ccs_base/hercules/flash/libraries/f021_R4_be_l2fmc.nfl is an archive file of programs which run in SRAM on the Hercules and are built on top of the F021 flash API to erase / program the flash. From the log file can't immediately tell if the problem is the code can't run, or it runs but the flash erase algorithm fails. 

Hi,
is there anyone at Texas Instruments who can help us with this?

Kind regards,
Lars

Hi QJ,

we have done exactly that.

Please re-read the whole thread to get the full picture.

Kind regards,

Lars

Bombardier

Hello Lars,

I tried TMS570LC43x_flash_erase with default gel file on my Launchpad, and I didn't get any problem during compiling, loading, and code execution. The ECC is generated by CCS, and MPU is enabled (under ARM advanced Feature).

I wrote another project (attached) which generates ECC using linker command file. The code is located in flash bank0, and to erase/program the flash bank1.  I didn't get any problem during compiling, loading, and code execution.

Please try my project, if you still get the similar compiling or loading problem, please re-install the flash lib in CCS.

TMS570LC4357_LinkerECC.7z

Hello QJ,
I get the same error as before. This error comes before loading the application when it tries to erase the Flash.

What shall we do next?

Bombardier has a FAE at TI here in Stockholm which I am in contact with.
Shall I contact him and look up the possibility to visit them and bringing our hardware there for testing?

Lars

Lars von Wachenfeldt said:

What shall we do next?

If a Uniflash installation reports the the same error that would help to eliminate the possibility of a corrupt "flash lib in CCS" which was a previous suggestion from QJ Wang as to the cause of the error.

Good idea. I've tested that now but get the same error. I used the ccxml-file for the project.

I can also add that we use two 4357 MCUs on the board. When we program it we bypass one MCU at a time, see image below. We have used this board for more than one year and never got this problem before. That's why we suspected it was some HW error when the made the latest version of the board, since the problem then began. But we were capable to lock a previous version of the board too.

Moreover, the programs that were loaded (which must cause the locking) on the locked MCUs runs as normal when you power up. When you connect to the board it of course stops. We load older versions of the program (our code and not the latest HALCoGen) on not programmed boards from now on. Only speculative, but could it be a mix of our code and the latest HALCoGen? For example, we now skip a BIST code which we had in HL_sys_startup.c before. This is really strange.

Hi QJ,
as I said above:

> What shall we do next?
> Bombardier has a FAE at TI here in Stockholm which I am in contact with.
> Shall I contact him and look up the possibility to visit them and bringing our hardware there for testing?

Here's a link to post regarding a similar situation. The difference is that it's not about a failing flash erase though. 

e2e.ti.com/.../1866658

We now have a board and a running code which we can connect to via CCS using this:

The PC points exactly where the code is halted but we cannot run/step/debug it. The PC just stands still.

I try to erase via On-Chip Flash but no luck:

Hi QJ,
the crystal is fine. We've now tested with a LaunchPad and we managed to "lock" that as well. 

We are not sure but we think it's a mixture of new HALCoGen code and our code. 
We do not think it's a problem with the new HALCoGen code. 

We have put this on ice for the moment. I may contact you a bit later.

Kind regards,
Lars

Hi QJ and thanks alot for your effort. 
And many thanks for the cmd file we got from you. 

We have solved the ECC issue since some several weeks ago (we don't get any more errors) but will have a look at the code you attached anyway. 
It can be useful. :)

Quotation from myself:
"the crystal is fine. We've now tested with a LaunchPad and we managed to "lock" that as well."

You asked me "BTW, have you solved the sector0 erasing issue"?

What I meant by saying "we managed to "lock" LaunchPad as well" is that we cannot erase the flash in the LaunchPad. 
It's nothing about ECC, we connect to the LaunchPad but cannot erase it! (+ our other own boards, pls read from the beginning)

What we did with the LaunchPad were:
1. Programmed our binary
2. (tried to) program our binary again and we got the error message from the erase algorithm
From there it's bricked/locked, anymore tries and it fails

We could maybe send you the locking binary so you can test yourselves. Pls contact me. 

Kind regards,
Lars
Bombardier
Rail Control Solutions

Hello Lars,

Thanks for sending me the project.