• Resolved

Linux/AM5728: Linux Memory Policies

Part Number: AM5728

Tool/software: Linux

Hello, I'm working with getting my AM5728-based custom board to boot, and I'm running into a strange hang early in the kernel boot procedure.  I believe it may have to do with my RAM configuration.  During boot, my kernel prints the following messages then hangs indefinitely:

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0

[    0.000000] Linux version 4.9.28-geed43d1050 (tom@tom-ThinkPad-P50s) (gcc version 6.2.1 20161016 (Linaro GCC 6.2-2016.11) ) #6 SMP PREEMPT Mon Oct 2 10:59:18 EDT 2017

[    0.000000] CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=30c5387d

[    0.000000] CPU: div instructions available: patching division code

[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache

[    0.000000] OF: fdt:Machine model: tom's custom board

[    0.000000] bootconsole [earlycon0] enabled

[    0.000000] efi: Getting EFI parameters from FDT:

[    0.000000] efi: UEFI not found.

[    0.000000] Reserved memory: created CMA memory pool at 0x0000000095800000, size 56 MiB

[    0.000000] OF: reserved mem: initialized node ipu2_cma@95800000, compatible id shared-dma-pool

[    0.000000] Reserved memory: created CMA memory pool at 0x0000000099000000, size 64 MiB

[    0.000000] OF: reserved mem: initialized node dsp1_cma@99000000, compatible id shared-dma-pool

[    0.000000] Reserved memory: created CMA memory pool at 0x000000009d000000, size 32 MiB

[    0.000000] OF: reserved mem: initialized node ipu1_cma@9d000000, compatible id shared-dma-pool

[    0.000000] Reserved memory: created CMA memory pool at 0x000000009f000000, size 8 MiB

[    0.000000] OF: reserved mem: initialized node dsp2_cma@9f000000, compatible id shared-dma-pool

[    0.000000] cma: Reserved 24 MiB at 0x000000008e400000

[    0.000000] Memory policy: Data cache writealloc

Could hanging after Data cache writealloc indicate a problem with the main system RAM?  The major difference between my board and the AM5728 EVM is that I only have 256MB of DDR3 RAM with different timings.  I've updated the u-boot configurations for this, and u-boot seems to have no issues with running and relocating itself in the RAM.  u-boot RAM read and write tests also indicate a high degree of coherency.


According to the EVM, the next line in the kernel boot should be:
[    0.000000] OMAP4: Map 0x00000000ffd00000 to fe600000 for dram barrier

which is why I feel like the problem could be memory related.  Is this assumption valid?  If it is what could cause u-boot to work correctly with my RAM and Linux not?

Another possibility is that the addresses given by the IPU/DSP's in the debug lines are accessing memory spaces that are unavailable to them.  Since we have only 256MB of RAM, mapped as 0x80000000 to 0x90000000, and those memory addresses for the DSP/IPU are outside of that range (0x95800000, 0x99000000, 0x9d000000, 0x9f000000, etc) they're trying to access memory that is simply unavailable to them?

Here is the full boot, in case anyone is interested.  Full decompiled device tree is as well.

3581.putty.log

3618.customboard.txt

  • Hi Tom,

    From the info you shared, it seems to me that linux kernel is trying to access memory that is not available. The default PSDK 4 (kernel 4.9) comes with 2GB DRAM, while you have only 256MB DRAM.

    You can also test your DRAM memory at u-boot stage with mw/md commands.

    Regards,
    Pavel



  • Hi Tom,

    Your assumption about accessing memory addresses for the DSP/IPU outside the physical existing memory can cause a crash of system.

    BR
    Tsvetolin Shulev
  • In reply to Pavel Botev:

    Thanks for the clarification, I've followed your recommendation with mw/md commands (as well as re-enabling mtest) and I've found some potential tricky spots in my DRAM.  These appear to be sections in RAM that U-Boot has placed commands or other important data (I've found the FIT/FDT command string in RAM).  But when you said that the default PSDK4 comes with 2GB (to match the AM57x EVM, for sure), is there somewhere in the kernel setup that I have to specify RAM size?  Because I've noticed that the statement that is supposed to print next after my system hangs is 

    [    0.000000] OMAP4: Map 0x00000000ffd00000 to fe600000 for dram barrier

    or something equivalent with regards to my memory boundaries.  So my theory is that my kernel is somehow looking for a  2GB boundary and it's looking past my 256MB boundary, and consequently hanging outside of available space.

    As an intermediary band-aid solution I've removed the DSP/IPU/offending device tree nodes from my device tree for now.

  • In reply to Tom W:

    Tom,

    Tom W
    Thanks for the clarification, I've followed your recommendation with mw/md commands (as well as re-enabling mtest) and I've found some potential tricky spots in my DRAM.  These appear to be sections in RAM that U-Boot has placed commands or other important data (I've found the FIT/FDT command string in RAM). 

    I would suggest you to fix your DRAM first. See if the below pointers will be in help for testing it:

    I will have a look your other question (regarding memory map) and come back to you.

    Regards,
    Pavel



  • In reply to Tom W:

    Tom W
    But when you said that the default PSDK4 comes with 2GB (to match the AM57x EVM, for sure), is there somewhere in the kernel setup that I have to specify RAM size? 

    In u-boot, make sure you have update your dmm_lisa_map_x registers and DTS file which are by default configured for 2GB DRAM:

    u-boot-2017.01/arch/arm/dts/am57xx-beagle-x15-common.dtsi

    memory@0 {
            device_type = "memory";
            reg = <0x0 0x80000000 0x0 0x80000000>;  //2GB DRAM
        };

    In kernel, you can adjust the DTS file from 2GB to 256MB:

    linux-4.9.28/arch/arm/boot/dts/am57xx-beagle-x15-common.dtsi

    memory@0 {
            device_type = "memory";
            reg = <0x0 0x80000000 0x0 0x80000000>;
        };

    See also below pointers for better DRAM config understanding:

    Regards,
    Pavel



  • In reply to Pavel Botev:

    Hi Pavel, thanks for the reply.  I've modified both of these memory@0 nodes to fit my config (256MB starting at 0x80000000):

    memory@0 {
            device_type = "memory";
            reg = <0x0 0x80000000 0x0 0x10000000>;
        };

    I've also modified the reserved-memory node as such:

    reserved-memory {
    /delete-node/ ipu2_cma@95800000;
    /delete-node/ dsp1_cma@99000000;
    /delete-node/ ipu1_cma@9d000000;
    /delete-node/ dsp2_cma@9f000000;

    cmem_block_mem_0: cmem_block_mem@a0000000 {
    reg = <0x0 0x88000000 0x0 0x01000000>;
    };

    };

    so that the DSP/IPU nodes are not used (they are disabled as well later in the DT) and the cmem block is moved to a space in RAM (and shrunk) that exists on my system.

    I've unfortunately had no results.  This is the output I receive:

    Starting kernel ...

    [ 0.000000] Booting Linux on physical CPU 0x0
    [ 0.000000] Linux version 4.9.28-geed43d1050 (tom@tom-ThinkPad-P50s) (gcc version 6.2.1 20161016 (Linaro GCC 6.2-2016.11) ) #16 SMP PREEMPT Wed Oct 4 15:08:57 EDT 2017
    [ 0.000000] CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=30c5387d
    [ 0.000000] CPU: div instructions available: patching division code
    [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
    [ 0.000000] OF: fdt:Machine model: tom's customboard
    [ 0.000000] bootconsole [earlycon0] enabled
    [ 0.000000] efi: Getting EFI parameters from FDT:
    [ 0.000000] efi: UEFI not found.
    [ 0.000000] cma: Reserved 24 MiB at 0x000000008e400000
    [ 0.000000] Memory policy: Data cache writealloc

  • In reply to Tom W:

    Tom W
    cmem_block_mem_0: cmem_block_mem@a0000000 {
    reg = <0x0 0x88000000 0x0 0x01000000>;
    };

    You should remove this node, or at least try with:

    cmem_block_mem_0: cmem_block_mem@88000000 {
    reg = <0x0 0x88000000 0x0 0x01000000>;
    };

    Tom W

    Starting kernel ...

    [ 0.000000] Booting Linux on physical CPU 0x0
    [ 0.000000] Linux version 4.9.28-geed43d1050 (tom@tom-ThinkPad-P50s) (gcc version 6.2.1 20161016 (Linaro GCC 6.2-2016.11) ) #16 SMP PREEMPT Wed Oct 4 15:08:57 EDT 2017
    [ 0.000000] CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=30c5387d
    [ 0.000000] CPU: div instructions available: patching division code
    [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
    [ 0.000000] OF: fdt:Machine model: tom's customboard
    [ 0.000000] bootconsole [earlycon0] enabled
    [ 0.000000] efi: Getting EFI parameters from FDT:
    [ 0.000000] efi: UEFI not found.
    [ 0.000000] cma: Reserved 24 MiB at 0x000000008e400000
    [ 0.000000] Memory policy: Data cache writealloc

    Can you check if ARM A15 LPAE feature is enabled? If yes, disable it and try again.

    Regards,
    Pavel



  • In reply to Pavel Botev:

    You can also add some debug prints to identify where exactly your boot flow stuck

    linux-kernel/arch/arm/mm/mmu.c

    /*
     * paging_init() sets up the page tables, initialises the zone memory
     * maps, and sets up the zero page, bad page and bad page tables.
     */
    void __init paging_init(const struct machine_desc *mdesc)
    {
        void *zero_page;

        build_mem_type_table();   // --> Memory policy: Data cache writealloc
        prepare_page_table();
        map_lowmem();
        memblock_set_current_limit(arm_lowmem_limit);
        dma_contiguous_remap();
        early_fixmap_shutdown();
        devicemaps_init(mdesc); // --> OMAP4: Map 0x00000000ffd00000 to fe600000 for dram barrier
        kmap_init();
        tcm_init();

        top_pmd = pmd_off_k(0xffff0000);

        /* allocate the zero page. */
        zero_page = early_alloc(PAGE_SIZE);

        bootmem_init();

        empty_zero_page = virt_to_page(zero_page);
        __flush_dcache_page(NULL, empty_zero_page);
    }

     

    Check first if the flow goes out of the build_mem_type_table() function, then check if the flow goes inside the devicemaps_init() function.



  • In reply to Pavel Botev:

    Note also that AM335x SK (StarterKit) comes with 256MB DDR3 memory, you can check its device tree file and linux kernel config to get the proper settings for 256MB memory map.

    www.ti.com/.../tmdssk3358



  • In reply to Pavel Botev:

    Thanks again Pavel, I've fixed the memory address for cmem_block_mem@a0000000 and replaced it with cmem_block_mem@88000000.

    I disabled the LPAE, and proceeded further in the boot than before (log is attached) however I've noticed that I'm receiving quite a lot of EXT4 filesystem errors.  Unfortunately my other SD cards are all being used, so I can't verify for certain if the fault is with this SD card or not.  I've also noticed that the line 

    [    0.000000] OMAP4: Map 0x8fe00000 to fe600000 for dram barrier

     

    is still being printed.

    From what I've read about the LPAE, it seems rather necessary for the A-15 architecture.  I'm not planning on going above the 32-bit addressing limit, so I can see why disabling it could make sense, but I'm unsure if it's wise.