DRA7xx large (4GB) DDR Memory Configuration

Thanks you for the moment.

I try to add some debug info to specify better the issue symptoms.

Following the traces of early printk of the stucking kernel with some more debug added in "arch/arm/mm/mmu.c" here:

	/*
	 * Create a mapping for the machine vectors at the high-vectors
	 * location (0xffff0000).  If we aren't using high-vectors, also
	 * create a mapping at the low-vectors virtual address.
	 */
printk("__phys_to_pfn\n");
	map.pfn = __phys_to_pfn(virt_to_phys(vectors));
printk("__phys_to_pfn done\n");
	map.virtual = 0xffff0000;
	map.length = PAGE_SIZE;
	map.type = MT_HIGH_VECTORS;
printk("create_mapping HIGH\n");
	create_mapping(&map, false);

and also here:

static void __init create_mapping(struct map_desc *md, bool force_pages)
{
	unsigned long addr, length, end;
	phys_addr_t phys;
	const struct mem_type *type;
	pgd_t *pgd;

	if (md->virtual != vectors_base() && md->virtual < TASK_SIZE) {
		printk(KERN_WARNING "BUG: not creating mapping for 0x%08llx"
		       " at 0x%08lx in user region\n",
		       (long long)__pfn_to_phys((u64)md->pfn), md->virtual);
		return;
	}

	if ((md->type == MT_DEVICE || md->type == MT_ROM) &&
	    md->virtual >= PAGE_OFFSET &&
	    (md->virtual < VMALLOC_START || md->virtual >= VMALLOC_END)) {
		printk(KERN_WARNING "BUG: mapping for 0x%08llx"
		       " at 0x%08lx out of vmalloc space\n",
		       (long long)__pfn_to_phys((u64)md->pfn), md->virtual);
	}

	type = &mem_types[md->type];

#ifndef CONFIG_ARM_LPAE
	/*
	 * Catch 36-bit addresses
	 */
	if (md->pfn >= 0x100000) {
		create_36bit_mapping(md, type);
		return;
	}
#endif

	addr = md->virtual & PAGE_MASK;
	phys = __pfn_to_phys(md->pfn);
	length = PAGE_ALIGN(md->length + (md->virtual & ~PAGE_MASK));



	if (type->prot_l1 == 0 && ((addr | phys | length) & ~SECTION_MASK)) {
		printk(KERN_WARNING "BUG: map for 0x%08llx at 0x%08lx can not "
		       "be mapped using pages, ignoring.\n",
		       (long long)__pfn_to_phys(md->pfn), addr);
		return;
	}

printk("map for 0x%08llx at 0x%08lx len 0x%08lx\n",(long long)__pfn_to_phys(md->pfn), addr, length);

	pgd = pgd_offset_k(addr);
	end = addr + length;

printk(" pdg 0x%08lx end 0x%08lx\n",pgd, end);
	do {
		unsigned long next = pgd_addr_end(addr, end);
printk(" next 0x%08lx\n",next);
		alloc_init_pud(pgd, addr, next, phys, type, force_pages);

		phys += next - addr;
		addr = next;
printk(" phys 0x%08lx - addr 0x%08lx\n",phys, addr);

	} while (pgd++, addr != end);
}

OUTPUT:

Starting kernel...
Booting Linux on physical CPU 0x0
Linux version 3.8.13-00125-g230ac8a-dirty-01.06.00.01 (root@rebuildhawk-4) (gcc version 4.7 (GCC) ) #11 SMP PREEMPT Fri Nov 4 13:24:34 CET 2016
CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=30c7387d
CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
Machine: Jacinto6 evm board, model: TI DRA7
mta_hw_type from cmdline is =  213
 :::::::::: u-boot_ver : 2013.04-MTA_01_06_00.01-00051-gd5fcabe-dirty :::::::::::
cma: CMA: reserved 32 MiB at 9d000000
cma: CMA: reserved 112 MiB at a8800000
Memory policy: ECC disabled, Data cache writealloc
map for 0x80000000 at 0xc0000000 len 0x2f800000
 pdg 0xc0003018 end 0xef800000
 next 0xef800000
 phys 0x00000000 - addr 0xaf800000
map for 0x9d000000 at 0xdd000000 len 0x02000000
 pdg 0xc0003018 end 0xdf000000
 next 0xdf000000
 phys 0x00000002 - addr 0x9f000000
map for 0xa8800000 at 0xe8800000 len 0x07000000
 pdg 0xc0003018 end 0xef800000
 next 0xef800000
 phys 0x00000002 - addr 0xaf800000
devicemaps_init
early_trap_init
early_trap_done
__phys_to_pfn
__phys_to_pfn done
create_mapping HIGH
map for 0xa87b6000 at 0xffff0000 len 0x00001000
 pdg 0xc0003018 end 0xffff1000
 next 0xffff1000

can you suggest something to check in deep or something to test at least to better investigate this issue?

Thanks

Hi, unfortunatelly the system is in production and we need to add the functionality on K 3.8.

In order to have an idea of amounts of work, can the LPAE support be ported only on kernel un-touching the bootloader?

BR

Fabio

Fabio

The memory configuration is handled during early stages of the boot process as part bootloader relocates itself to run out of DDR. The linux kernel is also loaded and run out of DDR, hence any memory configuration related changes have to be carried out early in the bootloader itself.

In short, memory configuration(EMIF timing parameters) is programmed from the bootloader, hence you will need to update the bootloader(for updated timing parameters and memory adapter/LISA mapping) in addition to enabling LPAE support in the kernel. The linux kernel relies on memory configuration from uboot and we enable the LPAE in kernel to access higher order memory addresses(above 32-bit physical address space)

Hope this helps

Regards

Sriram

Hi,
Is there a performace impact if we enable LPAE ?
Have you measured the performance impact with system LPAE enabled and disabled ?

Hi

We havent specifically done performance measurements. Enabling LPAE by itself should not introduce performance regressions. If additional memory avaiable in the system is now mapped as High memory in Linux and accessed extensively through high memory regions, then there could be additional software overhead in managing these mappings at run time

Regards

Sriram