How to run linux kernel from a different physical address than default on TCI6638?

Steve,

Any particular reason you have to allocate memory for DSP to 0x80000000 instead of 0xC0000000? Without modifying memory map for Linux, you can simply set U-boot environment mem_reserve =1024M to reserve 1GB from 0xC0000000 for DSP applications and retain first 1GB (from 0x80000000) for Linux if U-boot detects 2GB memory on your EVM.

We can go through the steps to update Linux starting address other than 0x80000000 if you really need do so.

Regards, Garrett

Garrett:

Because My DSP’s CMD file had be allocated from 0x800000000.If the kernel started from the same 0x80000000,I had to modify DSP project.

         In fact,There is only two way to go.One is to modify DSP program CMD.the other is to modify ARM linux kernel.Of course,I can modify the DSP project.But I just want to know why I failed to modify linux address to 0xC0000000 even I have follow wiki steps below: http://processors.wiki.ti.com/index.php/MCSDK_UG_Chapter_Exploring#How_to_run_Linux_kernel_from_a_different_physical_address_than_default_.280x80000000.29.C2.A0.3F

Garrett Ding said:

Steve,

Any particular reason you have to allocate memory for DSP to 0x80000000 instead of 0xC0000000? Without modifying memory map for Linux, you can simply set U-boot environment mem_reserve =1024M to reserve 1GB from 0xC0000000 for DSP applications and retain first 1GB (from 0x80000000) for Linux if U-boot detects 2GB memory on your EVM.

We can go through the steps to update Linux starting address other than 0x80000000 if you really need do so.

Regards, Garrett

Hi Garrett:

1.the reason

The true reason that I decided to run linux kernel from 0xC0000000 other than 0x80000000 was

I met a problem.The problem like this below:

I built DSP program A.out on CCS. From ARM side I used

mpmcl reset dsp0

mpmcl load dsp0 A.out

when I executed mpmcl load dsp0 A.out, produced error info"load failed (error=-104)".

So I saw memory.h in kernel source code below:

#define KEYSTONE_LOW_PHYS_START  0x80000000ULL
#define KEYSTONE_LOW_PHYS_SIZE  0x80000000ULL /* 2G */

So I guessed that the kernel occupied all of DDR 2G space.I decided to downsize

KEYSTONE_LOW_PHYS_SIZE     0x40000000    1GB

2.

I tried your advice that using mem_reserve = 1024M,Unluckly,the kernel didn't start.

always stop at "Starting Kernel......."

 

 

 

 

Steve

Let’s get back to your initial problem – mpmcl ‘load failed (error=-104)’.

-          Are you using the MPM from MCSDK-3.0.1.12 linux-devkit?

-          Are you able to load mpmsrv_keystone2_example image at \ti\mcsdk_bios_3_00_01_12\examples\mpm\?

-          What does it look like in /var/log/mpmsrv.log when the load failure happens? do you see /dev/dsp0 and /dev/dspmem in your file system?

-          When the U-boot starts up, how much DRAM is detected? Does it look like:

U-Boot 2013.01-00004-g0c2f8a2 (Jul 30 2013 - 12:57:02)

I2C:   ready

DRAM:  1 GiB

-          What’s the memory usage shown in kernel boot log?

[    0.000000] Memory: 512MB = 512MB total

[    0.000000] Memory: 496068k/496068k available, 28220k reserved, 0K highmem

-         In the case above, mem_reserve is 512M, and you can allocate memory starting from 0xA0000000 for DSP, and from 0x80000000 to 0x9FFFFFFF for Linux.

-         If you have 2GB memory detected in U-boot, you can reserve 1024M for DSP application.

Regards, Garrett

Hi, Garrett：

Thank you,As you said,I tried to load dsp program from the start address 0xA000 0000 and the result was exciting.In other words, I could load dsp program successfully,But another questions remained.

1.Why uboot just could recognize 1GB memory space other than 2GB ?

I confirmed that EVM had 2GB DDR3 on board.why uboot just used the first 1GB?

What about the second 1GB address from 0xC000 0000 to 0Xffff ffff?

Whoever DSP or ARM couldn’t use the second 1GB memory space?DSP was not ,ARM was the same not?

What was the second 1GB memory space used to?

 

2.

Other than you said,I tried load dsp program from the sart address 0xC000 0000.Unluckly,I failed and the error info “failed  load  error=-104” again.So I guessed that the second 1GB space from 0xC000 0000 to 0Xffff ffff wan not be used by DSP,Because the uboot recognized only 1GB memory space and the kernel ‘space was 1GB from 0x8000 0000 to 0xC000 0000.

But I just wanted to know why? Where the second 1GB space from 0xC000 0000 to 0Xffff ffff?

 

3

Go back to the beginning question,If I wanted to allocate 2GB memory space like below,How could I do?

DSP :   from 0x 8000 0000 to 0xC000 0000

ARM   from 0xC000 0000 to 0Xffff ffff

By the way,I used mpmsrv and mpmcl execute program in the mcsdk_3_00_01_11 not 3_00_01_12. Because When I used mpm in the 3_00_01_12,

I run /usr/share/matrix_2_0/apps/demo_ipc/demo_ipc.sh  unsuccessfully.So I had to use mcsdk_3_00_01_11.

 Any help will be appreciated!

Garrett Ding said:

Steve

Let’s get back to your initial problem – mpmcl ‘load failed (error=-104)’.

-          Are you using the MPM from MCSDK-3.0.1.12 linux-devkit?

-          Are you able to load mpmsrv_keystone2_example image at \ti\mcsdk_bios_3_00_01_12\examples\mpm\?

-          What does it look like in /var/log/mpmsrv.log when the load failure happens? do you see /dev/dsp0 and /dev/dspmem in your file system?

-          When the U-boot starts up, how much DRAM is detected? Does it look like:

U-Boot 2013.01-00004-g0c2f8a2 (Jul 30 2013 - 12:57:02)

I2C:   ready

DRAM:  1 GiB

-          What’s the memory usage shown in kernel boot log?

[    0.000000] Memory: 512MB = 512MB total

[    0.000000] Memory: 496068k/496068k available, 28220k reserved, 0K highmem

-         In the case above, mem_reserve is 512M, and you can allocate memory starting from 0xA0000000 for DSP, and from 0x80000000 to 0x9FFFFFFF for Linux.

-         If you have 2GB memory detected in U-boot, you can reserve 1024M for DSP application.

Regards, Garrett

Hi Garrett：

 

Let’s continue to focus on the mpm load problem .

I confirmed that DSP program with start address from 0xA000 0000 could be loaded successfully on mem_reserve =512M.On mem_reserve = 512M ,1 GB memory space was allocated like this:

Condition 1:

First 512M  

0x8000 0000    to      0x9fff ffff     //use for linux kernel

Second 512M

0xA000 0000     to     0Xbfff ffff    //used for dsp program

In condition 1, With start address from 0x8000 0000 or 0x9000 0000 DSP program couldn’t be load successfully along with error -104.The same time from 0xA000 0000 DSP program could be load successfully.  It seemed reasonable. But I found some strange situation in the following condition 2 and condition 3.

Condition 2:  mem_reserve = 768M

First 256M

0x8000 0000  TO  0x8fff ffff      //  used for arm linux kernel

Last 768M

0x9000 0000  TO 0Xbfff ffff     //  used for dsp program

In condition 2,I found  DSP program from 0x9000 0000 unexpectedly couldn’t be load successfully. It was so unbelievable ,Why?  The address 0x9000 0000 was valid for DSP program.Why unsuccessfully??????

The strange situation appeared in the condition 3.

Condition 3: mem_reserve = 256M

First 768M

0x8000 0000 TO  0xafff ffff    //used for arm linux kernel

Last 256 M

0xB000 0000   TO  0xbfff ffff   //used for dsp program

 

It was unreasonable that with start address 0xA000 0000 DSP program could be load successfully.

Why ?  Address 0xA000 0000 was valid for ARM linux  and invalid for DSP program .Why loaded successfully?

So,With  strange condition 2 and condition 3.I guessed that  whether DSP program could be loaded successfully  was no matter with  How to allocate memory space 1GB DDR?

What was the secreat ? which  address DSP program from could be loaded  successfully? 

Regards！

Garrett Ding said:

Steve

Let’s get back to your initial problem – mpmcl ‘load failed (error=-104)’.

-          Are you using the MPM from MCSDK-3.0.1.12 linux-devkit?

-          Are you able to load mpmsrv_keystone2_example image at \ti\mcsdk_bios_3_00_01_12\examples\mpm\?

-          What does it look like in /var/log/mpmsrv.log when the load failure happens? do you see /dev/dsp0 and /dev/dspmem in your file system?

-          When the U-boot starts up, how much DRAM is detected? Does it look like:

U-Boot 2013.01-00004-g0c2f8a2 (Jul 30 2013 - 12:57:02)

I2C:   ready

DRAM:  1 GiB

-          What’s the memory usage shown in kernel boot log?

[    0.000000] Memory: 512MB = 512MB total

[    0.000000] Memory: 496068k/496068k available, 28220k reserved, 0K highmem

-         In the case above, mem_reserve is 512M, and you can allocate memory starting from 0xA0000000 for DSP, and from 0x80000000 to 0x9FFFFFFF for Linux.

-         If you have 2GB memory detected in U-boot, you can reserve 1024M for DSP application.

Regards, Garrett

Hi Garrett:

Another thread I fogot. Below was the /var/log/mpmsrv.log when loaded unsuccessfully.

root@keystone-evm:~# cat /var/log/mpmsrv.log
mpminit.c:659:json_get_slaves: Number of slaves 8
mpminit.c:683:json_get_slaves: adding core dsp0 to the list
mpminit.c:683:json_get_slaves: adding core dsp1 to the list
mpminit.c:683:json_get_slaves: adding core dsp2 to the list
mpminit.c:683:json_get_slaves: adding core dsp3 to the list
mpminit.c:683:json_get_slaves: adding core dsp4 to the list
mpminit.c:683:json_get_slaves: adding core dsp5 to the list
mpminit.c:683:json_get_slaves: adding core dsp6 to the list
mpminit.c:683:json_get_slaves: adding core dsp7 to the list
mpminit.c:760:json_get_slaves: Number of slaves Configured 8
mpminit.c:198:mpm_start_server: MPM parsing complete
mpminit.c:205:mpm_start_server: MPM ssm init
mpminit.c:221:mpm_start_server: MPM process cmd file complete
mpmsrv.c:278:mpm_monitor: starting monitor thread
mpmsrv.c:69:mpm_server: waiting for slave message
mpmsrv.c:98:mpm_server: received message of size 44 bytes for cmd 3
mpmsrv.c:168:mpm_server: received reset command
mpmssm.c:142:mpm_ssm_state_idle: received invalid event 4 for dsp0
mpmsrv.c:69:mpm_server: waiting for slave message
mpmsrv.c:98:mpm_server: received message of size 123 bytes for cmd 1
mpmsrv.c:120:mpm_server: received load command for dsp0 filename /home/root/moonshot/test_mpeg2_mpmcl_load_ok/TestAppDecoder_DSPJINGJIAN_90.out
mpmssm.c:329:mpm_load_slave: loading dsp0
mpmdlif.c:219:mpm_get_global_addr: Can't find global address from local address 0x90000000 size 430988 in cfg for dsp0
mpmdlif.c:127:map_and_copy_segment: Unable to get global address 0x90000000
mpmdlif.c:943:DLIF_allocate: map and copy failed for image /home/root/moonshot/test_mpeg2_mpmcl_load_ok/TestAppDecoder_DSPJINGJIAN_90.out with addr 0x90000000, size 0x6938c
mpmdlif.c:1101:DLIF_error: << D L O A D >> ERROR:
mpmdlif.c:1104:DLIF_error: Failed to allocate target memory for static executable.

mpmdlif.c:626:mpm_dlif_load: Image loading failed for file /home/root/moonshot/test_mpeg2_mpmcl_load_ok/TestAppDecoder_DSPJINGJIAN_90.out
mpmssm.c:340:mpm_load_slave: Image loading failed for /home/root/moonshot/test_mpeg2_mpmcl_load_ok/TestAppDecoder_DSPJINGJIAN_90.out
mpmssm.c:303:mpm_ssm_state_error: entered error state for dsp0

mpmsrv.c:69:mpm_server: waiting for slave message
root@keystone-evm:~#

Hi, Steve,

Are you using Rev 1.0 EVM? There is a DDR3 bus width issue which only allows half of the DDR3 memory be accessed. This issue was fixed in Rev 2.0 EVM. Becuase of that, a 2GB DDR3 DIMM only has 1GB space can be accessed.

To have 2GB memory space, you will either need to have a Rev 2.0 EVM or a Rev 1.0 EVM but 4GB DIMM. Only half of the DIMM space can be used in Rev 1.0 EVM.

Attached is a Application Notes for using MPM which we just wrote recently. You may already know it after spending so much time on MPM. The u-boot variable mem_reserve allocates the memory space for the DSP and it can be adjust according to the need.

Rex

0488.MPM memory setting of the device.docx

 

 

Thanks,Rex Chang:

Last week I send you a mail about mpm proble and knew that you were out of office.

Luckly,I got many help from Garrett Ding.I had understand mostly but not totally.

Would you help me to explain  the strange situation I mentioned above this post that 

I failed to load dsp program with the start address from 0x9000 0000 when I set mem_reserve = 768.

Another attached picture was the version of the K2EVM-HK .

it's said made in TAIWAN and the other side of EVM worte "XTCIEVMK2X  REV11"

From above infomation I guessed the EVM was REV1.0,but I was not sure.

What's your opinion?

Any help will be appreciated.

Regards!

Steve.

Hi, Steve,

Did you set your mem_reserve to 768 or 768M?

You have EVM Rev 1.1 which still uses the old silicon. Only Rev 2.0 has the newer silicon which fixed the 64-bit bus issue.

Rex

 

 

Rex:

hi,  I am sure that I set 768M not 768 .

When I set 768M ,that means DSP has memory 768MB and ARM has memory 256MB.

ARM memory space :  From 0x8000 0000  to  0x8fff ffff

DSP  memory space :  From 0x9000 0000  to  0xbfff ffff

Under above conditions,Why load dsp program from 0x9000 0000 unsuccessfully.

and it looks good from 0xA0000 00000 ?

Rex:

Another questions about that only 1GB memory can be accessed even if 2GB on board!

1.

When you said only 1GB space can be accessed.what did you mean?

I thought that it is right to ARM,but not for DSP. For DSP I had made a test . I can read and write from

the start address 0xC000 0000 .And this address range was out of 1GB.How to explain?

So I thought it was right for ARM that only 1GB space could be accessed.what about you?

2.

Thinking in another way.I thought the 2GB DDR3 memory could be allocated like below:

mem_reserve = 512M,evm 1.1

0-512MB           only can be accessed by ARM linux (DSP was forbidden)

512M -   1024M      Share memory  for  ARM and DSP, DSP  and ARM can access .

1024M -  2048M      only can be accessed by  DSP (ARM was forbidden)

Do you agree with me?

So as above described.For ARM 1GB memory can be accessed including 0-512M and 512-1024M.

For DSP  1536M memory can be accessed including 512M-1024M share memory and 1024M - 2048M

Best Regards!

Steve,

ARM accesses DDR3A memory and DSP  mem_reserve on DDR3A and  DDR3B which is 2GB. Please refer to either the schematics or TRM at http://www.advantech.com/Support/TI-EVM/EVMK2HX_sd.aspx.

Rex

Rex Chang said:

Steve,

ARM accesses DDR3A memory and DSP  mem_reserve on DDR3A and  DDR3B which is 2GB. Please refer to either the schematics or TRM at http://www.advantech.com/Support/TI-EVM/EVMK2HX_sd.aspx.

Rex

Hi, Steve,

I don't have answer on top of my head on this one. It will take a while to look into it.

Rex

Hi, Steve,

After changing mem_reserve, it is not as simple as changing your starting address of the DSP application in the build. It is also related to how the kernel device tree and the mpm in the json file are configured. The addresses in those 2 configuration also need to be adjusted.

Rex

Rex：

which json file?  I couldn't find any json file in my kernel souce code?

A default json file is provided in the file system under /etc/mpm (see System Management section in Chapter Developing of the User's Guide) which is the mpm configuration file. You don't want to modify the default one in mpm project and rebuild the whole file system.

Rex

Rex Chang said:

Steve,

ARM accesses DDR3A memory and DSP  mem_reserve on DDR3A and  DDR3B which is 2GB. Please refer to either the schematics or TRM at http://www.advantech.com/Support/TI-EVM/EVMK2HX_sd.aspx.

Rex

Hi Rex:

Rex:

In the post Last week above,you said that ARM accesses DDR3A memory and DSP  mem_reserve on DDR3A and  DDR3B.But I felt it maybe not right when I looked into the tci6638k2k datasheet_SPRS836A.

 

From pic1,we know that :

This region is mapped to DDR3A or DDR3B depending on the state of DDR3A_REMAP_EN pin at boot time. If the pin is ‘1’, this region is mapped to the first 2GB of DDR3A

which is aliased of 08 0000 0000 to 08 7FFF FFFF. If the pin is ‘0’, this region is mapped as 2GB of DDR3B.

 

That means the region on ARM is mapped to DDR3A or DDR3B. it depends the DDR3A_REMAP_EN. Another also from the pic1,we see the region on DSP is definitely mapped to DDR3B.

 

About DDR3A_REMAP_EN, I looked into the K2H_K2EVM-HK_SCH_A102_Rev1_0.pdf from www.advantech.com .From pic2,I can see that DDR3A_REMAP_EN was pull down on boot up.So DDR3A_REMAP_EN = 0 on 6638 starting.So it means that the region 0x8000 0000 to 0xffff ffff is mapped to DDR3B.

So,it is very different with what you said ARM accesses DDR3A memory .

 

Another from pic3,We know that the size of DDR3A is 2GB DDR3 DIMM ,But only 1GB could be accessed by ARM as last week you said because of bus issue. And for DDR3B,There are five SAMSUNG_K4B2G1646E-BCK. Every chip is 256MB.Excluding the chip use for ECC.Only 4x256MB = 1GB can be accessed.

 

So Rex, Do you think you view wright or wrong? On the earth ,DDR3A can be used for ARM or DSP?  Also DDR3B can be used for ARM or DSP? What’s your opinion?

pic1:

pic2:

pic3:

Steve,

You are right. Both DDR3A and DDR3B can be accessed from either ARM or DSP.

Rex

Rex Chang said:

Steve,

You are right. Both DDR3A and DDR3B can be accessed from either ARM or DSP.

Rex

Rex:

Two questions:

Q1:

I guess you don't understand what I said.

I also know that DDR3A and DDR3B can be accessed from ARM or DSP.

But what I concerned was how to avoid  ARM and DSP  to access the same address in the DDR3A or

DDR3B.

How to deal with conflicts when they aeecssed the same address in DDR3A or DDR3B.

How to assign the DDR3A and DDR3B is what I realy want to know.

From the post you said before.Your assigned solution was like below?

DDR3A from 0x8000 0000 to 0x9fff ffff was assigned to ARM
DDR3A from 0xa000 0000 to 0xbfff ffff was assigned to DSP
DDR3B was totally assigned to DSP.

But when I looked into the datasheet of tci6638k2k memory map ,My assigned solution was
like this:

DDR3B from 0x8000 0000 to 0x9fff ffff was assigned to ARM
DDR3B from 0xa000 0000 to 0xbfff ffff was assigned to DSP
DDR3A was totaly assigned to DSP.

which assigned solution was right?

Q2:

Another thread that I cannot understand was the range and start address of
DDR3A.From the datasheet of 6638 memory map,DDR3A start address is 0x8 0000 0000,
the range is from 0x8 0000 0000 to 0x9 FFFF FFFF.But other hand,as you said,start
address of DDR3A was 0x8 000 0000 not 0x8 0000 0000,the range was from 0x8 000 0000
to 0x9 fff ffff.How to explain?

Hi, Steve,

 In the memory map table, there are ARM view and DSP view. From the DSP view column of the memory map table, the address of 0x8000 0000 to DSP is DDR3B only, but can be 3A or 3B on ARM (ARM view column) depending on REMAP_EN setting.

Since on ARM, 3A can be 0x00 8000 0000 (2GB) and 0x08 0000 0000(8GB) (both are shown in memory map table),  the 2GB memory starting at 0x8000 0000 is mapped to the first 2GB of the 8GB at the physical address of 0x8 0000 0000. The difference of the first 2GB memory addressed using 32-bit and 36-bit is that the former does not have IO coherence but the latter do.

Rex

Hello Rex,

I have been following the posts related to XTCIEVMK2X DDR3A and DDR3B accesses.  You say that "both DDR3A and DDR3B can be accessed from either ARM or DSP".  

1. Does this mean that we can do away with one of the DDR3 A or B in our custom design and just reserve memory for ARM and allocate rest to DSP with one DDR3?

2. I tried to do some tests with u-boot to verify if ARM can use DDR3B address ranges .  The address range 0x6000 0000 to 0x7FFF FFFF (DDR3B data).

I tried to change the CONFIG_SYS_SDRAM_BASE defined in Tci6638_evm.h

#define CONFIG_SYS_SDRAM_BASE 0x60000000

and K2_DDR3_START_ADDR defined in tci6638_evm/board.c to 0x60000000.

Then and loaded the u-boot to MSMC RAM at 0x0c001000 and tried to run using JTAG.  I could not boot.  The u-boot stopped at DRAM init.

U-Boot 2013.01-00001-g95c56d3-dirty (Dec 02 2013 - 13:54:20)

I2C: ready
Detected SO-DIMM [ǧSQR-SD3T-2G1333]

If you are saying that, ARM can use DDR3B and vice versa, can you please explain, how can we start ARM from DDR3B?

Regards

Rams

Hi, Ram,

Not knowing your system requirement, I can not suggest if you can leave out one memory. Theoretically, you should be able to bring up U-Boot from DDR3B, but we have not tried it and not sure what are involved. You may also need to change the u-boot env variables. There may be some info in the open source forum for u-boot that you can poke to see how to start from different address.

As it is possible to start u-boot from DDR3B, but running kernel off it if that's your goal will be a different story. kernel uses alias address which may be located in different locations, and that will need to be taken care as well. There also should be info in the Linux open source forum about it.

Rex

Hi Rex,

Thanks for your response.  There has been some ambiguity in understanding DDR3A and DDR3B (ARM and DSP views, aliased addressing involving DDR3A and DDR3B) with reference to the datasheet and what is possible.  Though you have given guidance on theoretical possibilities, it would be good to have specific guidance with practical approach with the EVM.  This will help all keystone 2 users for their system design. 

My objective is to test U-Boot running from  DDR3B to verify the fact that ARM can be configured to use DDR3B as well for working RAM.  I believe this doesnt involve complex changes in u-boot, please let us know the steps.

Regards

Rams

Hi Rex,

It is to update you that I was able to run u-boot on DDR3B.  I removed the DIMM DDR3A from my EVM 2.0 board.  I modified the u-boot to disable the DDR3A initialization and flashed the modified U-boot.  When I powered on, the U-boot stopped at DDR init.  I disabled REMAP_EN bit from BMC and rebooted, and I was able to start U-Boot on DDR3B.

Unfortunately, there are limitations with EVM1.0 and that has caused lot of issues in understanding.

Thanks for your valuable inputs on REMAP_EN setting options through BMC.

Regards

Rams

Hi,Rams:

Would you tell me the details about how you can start U-boot and kernel on DDR3B on Rev2.0?

My board is Rev1.1.Can you start U-boot or kernel on DDR3B on Rev1.1?

Before I tried to reach that goal.But I didn't know how.And when I removed the DDR3A 2GB DIMM,

the uboot would stopped at DRAM init.So I dropped.

Hi Steve,

I am not sure whether Rev1.1 has also got the limitation similar to Rev1.0, maybe Rex can confirm.

But I tried with Rev2.0.  

Please refer 6116.boot.log attached to the post

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/306482.aspx

Regards

Rams

Hi,Rams:

What are those limitations which prevent you from starting up u-boot or kernel on DDR3B

with Rev1.0?Would you tell me some details ?

Any help will be appreciated.

I have tried the patch from http://processors.wiki.ti.com/index.php/MCSDK_UG_Chapter_Exploring#How_to_run_Linux_kernel_from_a_different_physical_address_than_default_.280x80000000.29.C2.A0.3F
An similarly it did not work. But Kernel always stopped at “Start Kernel…..” . Please update the wiki for the latest release of linux kernel distribution.