• State TI Thinks Resolved
  • Locked Locked
  • Replies 12 replies
  • Answers 1 answer
  • Subscribers 98 subscribers
  • Views 2378 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/AM3356: Boot fails after software update

Colombo Carlo
Mastermind 24160 points
Part Number: AM3356
Other Parts Discussed in Thread: UNIFLASH

Tool/software: Linux

Hello everybody , 

in my system we have a MMC  soldered on the  board .   everything is working  fine  than I do a SW update  ( hereunder all infos )  and  in some cases  ( about 1% of sw update )   system is no more capable to boot ( you see  CCCC on the serial , so even MLO is not loaded properly  ) .   we found this issue since for quality heavy testing is done to be sure  SW upgrade is bulletproof .   when system is not working ( not booting )  , I booted  using NFS and all files are there with proper MD5 checksum.

one extra info :  If I   change  only uEnv.txt  druing the update  the  system  update will  always work ( so not touching MLO and  uboot.img  system will always work)  .

here is  my partition table  and my sequence:

P1 boot (MLO u-boot.img, uEnv.txt) FAT32 about 64MB 

P2 RootFS1 ext4   about  256MB

P3 RootFS2 ext4 about  256MB

P4 Data         ext4 3500mb 

 

system is up and running on P2  ,  I do an update using a script  like this :

1) copy a new    MLO & u-boot.img, in P1

2)  untar  a tar.gz  rootFS in P3

3)  mark  uEnv.txt to enable  P3 as rootfs 

4) reboot ( both HW or SW same results ) .

all the previous is working about  99% of the time ...

please any suggestion is welcome 

thank you 

regards

Carlo

  • 0
    TI__Guru**** 393215 points
    Hi,

    What Linux version is this? Where is the kernel located? Do you erase the old files before copying the new ones? What do you mean by "MMC soldered on the board" - eMMC or SD card slot?
  • 0 in reply to Biser Gatchev-XID
    TI__Mastermind 24160 points
    Hi Biser ,
    it is a eMMC , and kernel is in the rootfs partition I m updating ( so if I update P3 , kernel is in P3 ) .
    I use two rootfs to do update from one to the other and then reboot .
    one extra point : I m now formatting everytime I do an update the Boot partition ( FAT32 )--> in this way no more family ( I am at 700 cycles ) .
    does this make any indication to you ?
    thank you very much
    regards
    Carlo
  • 0 in reply to Colombo Carlo
    TI__Guru**** 393215 points

    Colombo Carlo said:
    in this way no more family ( I am at 700 cycles ) .

    You mean no more failures? I don't understand.

  • 0 in reply to Biser Gatchev-XID
    TI__Mastermind 24160 points
    Hi Biser ,
    sorry for the typo ( it is no more failure not "family " in the previuous sentece ) .

    issue is exactly as my first post "everything is working fine than I do a SW update ( hereunder all infos ) and in some cases ( about 1% of sw update ) system is no more capable to boot ( you see CCCC on the serial , so even MLO is not loaded properly ) . we found this issue since for quality heavy testing is done to be sure SW upgrade is bulletproof . when system is not working ( not booting ) , I booted using NFS and all files are there with proper MD5 checksum"

    I simply added in last post reply to your request on Kernel position and the fact that if I do a format of the BOOT partition ,everytime I do an update , I have no more failures .

    regards
    Carlo
  • 0 in reply to Colombo Carlo
    TI__Guru**** 393215 points
    Thanks Carlo,

    I have asked the software team to comment. They will respond here.
  • 0 in reply to Biser Gatchev-XID
    TI__Guru**** 170625 points
    Hi Carlo,

    Seems your eMMC boot FAT32 partition is getting corrupted in 1% of the cases. This might happen if you have not valid power-down sequence or boot partition update sequence.

    Please double check your power down sequence is correct and align with datasheet. Make sure also you are re-flashing the eMMC boot partition according to the SW documents guidelines.

    Regards,
    Pavel
  • 0 in reply to Pavel Botev
    TI__Mastermind 24160 points
    Hi Pavel ,
    some more details to you and question :
    - I don t have a power down , it is a reboot command but not power cycling , so it cannot be this

    please could you redirect me to the proper SW guide you are mentioniong for reflashing eMMC ? I would like to be sure using proper one
    thank you
    best regards
    Carlo
  • 0 in reply to Colombo Carlo
    TI__Guru**** 170625 points
    Carlo,

    Biser asked one question that is still unanswered: Do you erase the old files before copying the new ones? Make sure before you copy MLO/u-boot.img in P1 (FAT32), you erase the old MLO/u-boot.img

    Can you also test if MLO is NOT updated, and you update u-boot.img/uEnv.txt/rootFS, will you have the same failure?

    Regarding SW guide for re-flashing eMMC, it depends on whether you are re-flashing eMMC from u-boot, user space or flasher tool. Check the below pointers for details:

    processors.wiki.ti.com/.../Linux_Core_U-Boot_User's_Guide
    processors.wiki.ti.com/.../Sitara_Linux_Program_the_eMMC_on_Beaglebone_Black
    processors.wiki.ti.com/.../Sitara_Uniflash_Flash_Programming_with_U-Boot
    processors.wiki.ti.com/.../Sitara_Uniflash_Flash_Programming_with_Linux
    processors.wiki.ti.com/.../AM437x

    Regards,
    Pavel
  • 0 in reply to Pavel Botev
    TI__Mastermind 24160 points

    HI Pavel ,

    we tried  also  erasing both MLO and Uboot ,  and then  rewrite them but no success :  after about 600 cycles we got the failure .(CCCCCCC...)

    we always do sync of the FS   before unmounting and as stated there is no power down , we simplye reboot .

    so  as for now only solution seems to be  to format FS  everytime  : does this make sense to you ?

    FYI  booting the system form NFS   on a corrupted system   MLO and Uboot.img have the proper  md5sum , the one we expected .

    this is table we use  .

    any idea ?  any input is welcome

    [root@media]# fdisk -l                                                   
    Disk /dev/mtdblock0: 1 MiB, 1048576 bytes, 2048 sectors                         
    Units: sectors of 1 * 512 = 512 bytes                                           
    Sector size (logical/physical): 512 bytes / 512 bytes                           
    I/O size (minimum/optimal): 512 bytes / 512 bytes                               
                                                                                    
                                                                                    
    Disk /dev/mmcblk0: 3.7 GiB, 3959422976 bytes, 7733248 sectors                   
    Units: sectors of 1 * 512 = 512 bytes                                           
    Sector size (logical/physical): 512 bytes / 512 bytes                           
    I/O size (minimum/optimal): 512 bytes / 512 bytes                               
    Disklabel type: dos                                                             
    Disk identifier: 0x0003805b                                                     
                                                                                    
    Device         Boot   Start     End Sectors  Size Id Type                       
    /dev/mmcblk0p1 *       2048  124927  122880   60M  c W95 FAT32 (LBA)            
    /dev/mmcblk0p2       124928  624639  499712  244M 83 Linux                      
    /dev/mmcblk0p3       624640 1124351  499712  244M 83 Linux                      
    /dev/mmcblk0p4      1124352 7729151 6604800  3.2G 83 Linux                      
                                                                                    
                                                                                    
    Disk /dev/mmcblk0boot1: 2 MiB, 2097152 bytes, 4096 sectors                      
    Units: sectors of 1 * 512 = 512 bytes                                           
    Sector size (logical/physical): 512 bytes / 512 bytes                           
    I/O size (minimum/optimal): 512 bytes / 512 bytes                               
                                                                                    
                                                                                    
    Disk /dev/mmcblk0boot0: 2 MiB, 2097152 bytes, 4096 sectors                      
    Units: sectors of 1 * 512 = 512 bytes                                           
    Sector size (logical/physical): 512 bytes / 512 bytes                           
    I/O size (minimum/optimal): 512 bytes / 512 bytes   

     

    regards

    Carlo

  • 0 in reply to Colombo Carlo
    TI__Guru**** 170625 points

    Carlo,

    Colombo Carlo said:

    we tried  also  erasing both MLO and Uboot ,  and then  rewrite them but no success :  after about 600 cycles we got the failure .(CCCCCCC...)

    we always do sync of the FS   before unmounting and as stated there is no power down , we simplye reboot .

    From what I understand, you update eMMC manually, with the below commands:

    1. mount /dev/mmcblk0p1

    2. cp MLO /dev/mmcblk0p1

    3. sync

    4 umount /dev/mmcblk0p1

    And you have boot failure after around 600 cycles of eMMC update. Can you also test with some automation update approach like dfu-utils and/or Sitara Uniflash tool?

    Are you sure you do not have abrupt power loss?

    Colombo Carlo said:
    so  as for now only solution seems to be  to format FS  everytime  : does this make sense to you ?

    You might compare eMMC FAT partition parameters (bad blocks, spare blocks, wear leveling, etc) between booting case and non-booting case and track for differences.

    Regards,
    Pavel

  • 0 in reply to Pavel Botev
    TI__Guru**** 170625 points
    Make sure also you do not have unexpected reset.

    You can also try with "fsck" and "badblocks" tests for mmcblk0p1. See the below e2e threads for details:

    e2e.ti.com/.../530162

    e2e.ti.com/.../560946

    Regards,
    Pavel