This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

DRA718: CMA size too large

Part Number: DRA718

Team

We are developing a custom application based on TI provided Vision SDK 02.12.0.0+RobustRVC 4.0 sources.

Several changes were made on U-Boot included at DRA7x Automitive Android SDK, according to https://git.ti.com/android-sdk/u-boot/commits/rvc-4.0 for running RVC application.

Commit https://git.ti.com/android-sdk/u-boot/commit/2e243b279215037a3cec85909c1bc6a6ccb6a90d

-#define DRA7_RPROC_CMA_SIZE_IPU2 0x03800000
+#define DRA7_RPROC_CMA_SIZE_IPU2 0x05000000


Running dra71x-ipu2-fw.xem4 provided by TI (Vision SDK 02.12.0.0+RobustRVC 4.0) or custom application building, the following error rise on boot time.

failed to allocate carveout rsc: da 94000000, pa 0, len 4400000, flags 0

Digging deeper by U-Boot debug traces in board/ti/dra7xx/lateattach.c
#ifdef CONFIG_LATE_ATTACH_DMA_POOL
void*alloc_mem(unsigned long len, unsigned long align){
...
debug("alloc_mem: ..
...
}

We realizes that pageno=0x10000, but mem_count=0x9000, so DRA7_RPROC_CMA_SIZE_IPU2 must be incresed to 0x20000000, to avoid error

Are we missing anything? 0x20000000=512MB is too much for CMA, isn't it?

Thanks in advance

  • Hi Christian,
    The configuration done in u-boot based on commit git.ti.com/.../2e243b279215037a3cec85909c1bc6a6ccb6a90d is tied to the Vision SDK 2.12 memory configuration.
    If you look into the \VISION_SDK_02_12_00_00\vision_sdk\build\tda2xx\mem_segment_definition_1024mb_linux.xs or the generated map files after building at: \VISION_SDK_02_12_00_00\vision_sdk\binaries\tda2xx_evm_robust_rvc\vision_sdk\bin\tda2xx-evm\vision_sdk_ipu2_release.xem4.map
    You will see:
    IPU2_CODE_MEM 94000000 00800000 005d18b3 0022e74d RWIX
    IPU2_DATA_MEM 94800000 03c00000 0307f3fc 00b80c04 RWIX
    DSP1_CODE_MEM 99000000 00200000 00000000 00200000 RWIX
    It is configuring 68MB but since the next is started at 0x99000000, we just cover IPU2 from 0x94000000-0x99000000.

    You can always optimize the IPUMM memory as currently default to 60MB at Vision SDK.
    To reduce IPUMM memory, you can look at: www.ti.com/.../sprac08, but make sure the memory is sufficient for video playback on you r system. The IPUMM itself by default is setting about 45MB considering 2x1080p MPEG4 decoding and 1x1080p H.264 encoding to run at the same time.

    When you said error rise on boot time, is it at u-boot or when kernel is booting.
    If it is when kernel booting, then the issue is that you are missing the changes in dtb to modify the ipu2_cma_pool start and size.
    Look on kernel: \kernel\android-4.4\arch\arm\boot\dts\dra7-evm-robust-rvc.dts:
    &ipu2_cma_pool {
    reg = <0x0 0x94000000 0x0 0x5000000>;
    };

    &mbox_ipu2_ipc3x {
    ti,no-reset-on-init;
    ti,no-idle-on-init;
    };

    &mmu_ipu2 {
    ti,late-attach;
    ti,no-reset-on-init;
    ti,no-idle-on-init;
    };

    &ipu2 {
    ti,late-attach;
    ti,no-reset-on-init;
    ti,no-idle-on-init;
    };
    and for other memories that are being reserved.

    Thanks.
    //Buddy
  • Hello Buddy

    Our IPUx and DSPx memory map configuration are according to your comments. We understand that these are fine and we don't reduce it.

    Error message rise at U-Boot (MLO) time.

    These are the original U-Boot error message

    U-Boot SPL 2016.05-00009-gb142ff9 (Apr 09 2018 - 14:14:49)
    DRA722-GP ES2.1
    Trying to boot from MMC2_2
    failed to allocate carveout rsc: da 94000000, pa 0, len 4400000, flags 0
    handle_resources failed: -12
    load_elf_image_phdr returned error for core IPU2
    Error loading remotecore IPU2!,Continuing with boot ...
    ## No elf image at address 0xa27ff000
    Error loading remotecore DSP2!,Continuing with boot ...
    ## No elf image at address 0xa0fff000
    Error loading remotecore IPU1!,Continuing with boot ...

    We added some custom trace

    alloc_mem: 1-len 983040(0x000f0000) align 8(0x00000008)
    alloc_mem: 2-order 8(0x00000008), ilog2 19(0x00000013)
    alloc_mem: 3-count 256(0x100) mask 0xff pageno 0x0, mem_count 0x9000

    bitmap_find_next_zero_area: 0-mem_count 36864(0x9000), start 0(0x0), count 256(0x100)
    bitmap_find_next_zero_area: 1-index 8(0x8) align_mask 255(0xff)
    bitmap_find_next_zero_area: 2-index 256(0x100) align_mask 255(0xff)
    bitmap_find_next_zero_area: 3-end 512(0x200), count 256(0x100)
    bitmap_find_next_zero_area: 0-mem_count 36864(0x9000), start 257(0x101), count 256(0x100)
    bitmap_find_next_zero_area: 1-index 512(0x200) align_mask 255(0xff)
    bitmap_find_next_zero_area: 2-index 512(0x200) align_mask 255(0xff)
    bitmap_find_next_zero_area: 3-end 768(0x300), count 256(0x100)

    alloc_mem: 4-count 256(0x100) mask 0xff pageno 0x200, mem_count 0x9000

    alloc_mem: 1-len 71303168(0x04400000) align 8(0x00000008)
    alloc_mem: 2-order 15(0x0000000f), ilog2 26(0x0000001a)
    alloc_mem: 3-count 32768(0x8000) mask 0x7fff pageno 0x0, mem_count 0x9000

    bitmap_find_next_zero_area: 0-mem_count 36864(0x9000), start 0(0x0), count 32768(0x8000)
    bitmap_find_next_zero_area: 1-index 8(0x8) align_mask 32767(0x7fff)
    bitmap_find_next_zero_area: 2-index 32768(0x8000) align_mask 32767(0x7fff)
    bitmap_find_next_zero_area: 3-end 65536(0x10000), count 32768(0x8000)

    alloc_mem: 4-count 32768(0x8000) mask 0x7fff pageno 0x10000, mem_count 0x9000
    failed to allocate carveout rsc: da 94000000, pa 0, len 4400000, flags 0
    handle_resources failed: - 12

    To avoid this error we have to increase,

    #define DRA7_RPROC_CMA_SIZE_IPU2 0x20000000

    1-We dont understand why CMA size must be increased to 512MB

    2-Is there any way to debug SDK_VISION+RVC, by U-boot terminal messages?. It would be usefully print out some logs.

  • Hi Christian, 

    Noticing three things, 

    1. We noticed your logs are showing "Trying to boot from MMC2_2", which isn't what we expected. By default RVC release is configured to run from QSPI boot, which includes some startup code being added to the QSPI boot in u-boot.
    2. Your previous comment mentioned the "#ifdef CONFIG_LATE_ATTACH_DMA_POOL" 
    3. Is this being run on a TI-EVM or a custom board? If custom, do you have an TI-EVM you could run this on? 

    Can you also provide a patch for your custom debug prints so we can attempt to replicate? 

    Thanks,

    David

  • Hello David

    It is running on custom board without QSPI (MMC only). We notice that UBoot should be modified to launch RVC.

    First at all we test dra71x-ipu2-fw.xem4 privided by TI on EVM (it runs booting from QSPI). Latter we run the same file on custom board and got the error message. We try with custom dra71x-ipu2-fw.xem4 file with the same error.

    Please understand that the following source is a dirty and quick "looking for error" code.  We modify lateattach.c for custom board, so add it manually

    void bitmap_set(unsigned long *map, int start, int nr)
    {
    	unsigned long *p = map + BIT_WORD(start);
    	const int size = start + nr;
    	int bits_to_set = BITS_PER_LONG - (start % BITS_PER_LONG);
    	unsigned long mask_to_set = BITMAP_FIRST_WORD_MASK(start);
    
    	printf("CHRISTIAN bitmap_set 0: start %d(%#x) count %d(%#x), bits_to_set %d(%#x), mask_to_set %ld(%#lx)\n", start, start, nr, nr, bits_to_set, bits_to_set, mask_to_set, mask_to_set);
    	while (nr - bits_to_set >= 0) {
    		*p |= mask_to_set;
    		nr -= bits_to_set;
    		bits_to_set = BITS_PER_LONG;
    		mask_to_set = ~0UL;
    		p++;
    	}
    	if (nr) {
    		mask_to_set &= BITMAP_LAST_WORD_MASK(size);
    		*p |= mask_to_set;
    	}
    }
    
    unsigned long bitmap_find_next_zero_area(unsigned long *map,
    					 unsigned long size,
    					 unsigned long start,
    					 unsigned int nr,
    					 unsigned long align_mask)
    {
    	unsigned long index, end, i;
    again:
       printf("CHRISTIAN bitmap_find_next_zero_area: 0-mem_count %ld(%#lx), start %ld(%#lx), count %d(%#x)\n", size, size, start, start, nr, nr);
    	index = evm_find_next_zero_bit(map, size, start);
    	printf("CHRISTIAN bitmap_find_next_zero_area: 1-index %ld(%#lx) align_mask %ld(%#lx)\n", index, index, align_mask, align_mask);
    	/* Align allocation */
    	index = (index + align_mask) & ~align_mask;
    	printf("CHRISTIAN bitmap_find_next_zero_area: 2-index %ld(%#lx) align_mask %ld(%#lx)\n", index, index, align_mask, align_mask);
    	end = index + nr;
       printf("CHRISTIAN bitmap_find_next_zero_area: 3-end %ld(%#lx), count %d(%#x)\n", end, end, nr, nr);
    	if (end > size)
    		return end;
    	i = find_next_bit(map, end, index);
    	if (i < end) {
    		start = i + 1;
    		goto again;
    	}
    	return index;
    }
    
    void *alloc_mem(unsigned long len, unsigned long align)
    {
       unsigned long mask;
       unsigned long pageno=0;
       int count, i, b;
       unsigned int order;
       printf("\nCHRISTIAN alloc_mem: 1-len %ld(0x%08lx) align %ld(0x%08lx)\n", len, len, align, align);
       if (len < PAGE_SIZE)
          order = 0;
       else{
          order = __ilog2((len) - 1) - PAGE_SHIFT + 1;
          printf("CHRISTIAN alloc_mem: 2-order %d(0x%08x), ilog2 %d(0x%08x)\n", order, order, __ilog2((len) - 1), __ilog2((len) - 1));
       };
       count = (1 << order);
       mask = (1 << order) - 1;
       printf("CHRISTIAN alloc_mem: 3-count %d(%#x) mask %#lx pageno %#lx, mem_count %#lx\n", count, count, mask, pageno, mem_count);
       /*printf("CHRISTIAN mem_bitmap 0:\n");
       b=1;
       i=0;
       printf("\n%04d:", i);
       for(i=0; i<BITS_TO_LONGS(DRA7_RPROC_MAX_CMA_SIZE >> PAGE_SHIFT); i++){
          printf("%08lx,", mem_bitmap[i]);
          if(b>=18){
             printf("\n%04d:", i);
             b=0;
          };
          b++;
       };
       printf("\n");*/
       pageno = bitmap_find_next_zero_area(mem_bitmap, mem_count, 0, count, mask);
       /*printf("CHRISTIAN mem_bitmap 1:\n");
       b=1;
       i=0;
       printf("\n%04d:", i);
       for(i=0; i<BITS_TO_LONGS(DRA7_RPROC_MAX_CMA_SIZE >> PAGE_SHIFT); i++){
          printf("%08lx,", mem_bitmap[i]);
          if(b>=18){
             printf("\n%04d:", i);
             b=0;
          }
          b++;
       };
       printf("\n");*/
       debug("CHRISTIAN alloc_mem: count %d mask %#lx pageno %#lx\n", count, mask, pageno);
       printf("CHRISTIAN alloc_mem: 4-count %d(%#x) mask %#lx pageno %#lx, mem_count %#lx\n", count, count, mask, pageno, mem_count);
       if (pageno >= mem_count)
          return NULL;
    
       bitmap_set(mem_bitmap, pageno, count);
       /*printf("CHRISTIAN mem_bitmap 2:\n");
       b=1;
       i=0;
       printf("\n%04d:", i);
       for(i=0; i<BITS_TO_LONGS(DRA7_RPROC_MAX_CMA_SIZE >> PAGE_SHIFT); i++){
          printf("%08lx,", mem_bitmap[i]);
          if(b>=18){
             printf("\n%04d:", i);
             b=0;
          }
          b++;
       };
       printf("\n");*/
       return (void *)(mem_base + (pageno << PAGE_SHIFT));
    }
    

  • Hi Christian, 

    Can you provide the CMA base addresses as defined in lateattach.c (ex. DRA7_RPROC_CMA_BASE_IPU1, DRA7_RPROC_CMA_BASE_IPU2...)? The CMA sizes should be set to fit without conflicting with other partitions. 

    Thanks,

    David

  • David

    #define DRA7_RPROC_CMA_BASE_IPU1 0x9d0000000
    #define DRA7_RPROC_CMA_BASE_IPU2 0x940000000
    #define DRA7_RPROC_CMA_BASE_DSP1 0x990000000
    #define DRA7_RPROC_CMA_BASE_DSP2 0x9f0000000

    /common/spl/spl.c
    u32 cores_to_boot[] = { IPU2, DSP2, IPU1 };

    2-Is there any way to debug SDK_VISION+RVC, by U-boot terminal messages?. It would be usefully print out some logs.

    Regards
  • Hi Christian,

    I'm not aware of any way to debug via terminal messages. If you look through the patches included with RVC, you can find robust_rvc_patches/vision_sdk/0011-RVC-To-print-the-RVC-usecase-stastistics.patch as an example of getting data in the core trace.

    With the base addresses defined as they are, it doesn't seem to make sense that you can use a size of 0x20000000 for the CMA size without stepping on other regions. I see you got your calculations based on pageno and count, which are gotten from bitmap_find_next_zero_area.

    Another check, are you using the alloc_mem function as gotten from the #ifdef CONFIG_LATE_ATTACH_DMA_POOL, or the #else section? CONFIG_LATE_ATTACH_DMA_POOL shouldn't be on for robust rvc, and I see you're using "count = (1 << order);" instead of "count = ((len + (PAGE_SIZE - 1)) & ~(PAGE_SIZE - 1)) >> PAGE_SHIFT;" which is what I see defined under the else section. Could this be why we're getting inappropriate values from bitmap_find_next_zero_area?

    Can I double check with you why we're seeing alloc_mem called with length 0x04400000 when we defined size as 0x05000000 as the CMA size?

    Also, just double checking you modified MMC2_2 to include the call to spl_setup_display()

    Thanks,
    David
  • David

    Reviewing dra7xx_evm_rvc_defconfig, I confirm that
    ...
    CONFIG_LATE_ATTACH_DMA_POOL=y
    ...
    So #ifdef CONFIG_LATE_ATTACH_DMA_POOL alloc_mem is being used (Custom traces are on this function)

    As far I know alloc_mem is being called with 0x04400000 because it is taken from m4 binary resource table, isn't it? 0x94000000 and 0x04400000 should be modified at vision_sdk/vision_sdk/build/tda2ex, isn't it?

    By now spl_setup_display is not being used in my U-Boot source code. First at all I ll try to get IPU2 running my custom firmware, and later I ll setup display HW.

    Thanks in advance
  • Hi Christian,

    Is your dra7xx_evm_rvc_defconfig a custom configuration? Double checking against the Robust RVC 4.0 Release Notes, I don't see any mention of that defconfig. It says to build with dra7xx_evm_defconfig instead, which does not define CONFIG_LATE_ATTACH_DMA_POOL. Double checking my .config after building uboot shows "# CONFIG_LATE_ATTACH_DMA_POOL is not set"

    Is there a reason you need CONFIG_LATE_ATTACH_DMA_POOL?

    Thanks,
    David
  • Hello David

    As far I know CONFIG_LATE_ATTACH_DMA_POOL is used to avoid kernel maps (set no-map property) memory used by IPU2.
    Making dra7xx_evm_xxx configs from scratch I see that CONFIG_LATE_ATTACH_DMA_POOL is not set. I'll test our custom config without it, following your advice.

    Address and limit (0x94000000 and 0x04400000) should be modified at vision_sdk/vision_sdk/build/tda2ex, isn't it?

    Regards
  • Hi Christian,

    I don't currently have a Robust RVC 4.0 source setup, so I can't cross verify. The tables your talking about look like they're in vision_sdk/vision_sdk/apps/build/tda2xx/*.xs. I believe it should either be here or in u-boot where these are being specified. The 0x94000000 at least I thought came from u-boot/board/ti/dra7xx/lateattach.c.

    Did disabling CONFIG_LATE_ATTACH_DMA_POOL help?

    Thanks,

    David

  • Hello David

    Disabling CONFIG_LATE_ATTACH_DMA_POOL solve it.

    From U-Boot side is clear that address and range is in lateattach.c.

    Could you confirm, where memory address configuration is located at vision_sdk?

    Thanks in advance

  • Hi Christian,

    I'm glad to hear it worked.

    I can confirm that the memory configuration needs to be changed in two places, vision_sdk/vision_sdk/apps/build/tda2xx/mem_segment_definition_linux.xs and u-boot/board/ti/dra7xx/lateattach.c both have to match in order for this to work. 

    For example, mem_segment_definition_linux.xs needs to define IPU2_START_ADDR as the same as lateattach.c defines DRA7_RPROC_CMA_BASE_IPU2.

    Thanks,

    David