DRA80XMEVM: Hypervisor code issue

Hi Devendra,

Please give me some time to look further into this issue. Thank you.

Hi Nikhil,

>>Make sure that you are routing the correct secure proxy interrupt to the Linux VM

Yes. I verified it by booting linux without hypervisor and interrupt number is 0x45. "GICv3  69 Level     32c00000.secure_proxy thr_011". If my understanding is wrong let me know.

>>Also, make sure that the secure proxy registers are correctly mapped in 1:1 manner for the VM.

Yes. I verified this also. I am not getting any data abort or any other exception at EL2. So, 1:1 mapping for stage-2 translation seems OK.

Regards,

Devendra 

Hi Devendra,

>>Make sure that you are routing the correct secure proxy interrupt to the Linux VM

I meant this from the hypervisor context.

Make sure you have assigned this SPI number to be routed from GIC to the correct CPUs running that VM.

Regards,

Nikhil D

Hi,

>>Make sure you have assigned this SPI number to be routed from GIC to the correct CPUs running that VM.

Yes. Currently, there is one-to-one mapping between physical IRQ and virtual IRQ. So, I am correctly injecting 0x45 SPI number in Linux.

Regards,

Devendra 

Hi Devendra,

Then hypervisor seems to be working correctly.

Hypervisor is making sure that the memory and interrupts are mapped 1:1 for the VM.

Somehow the TISCI driver is not able to function correctly with this.

Please ensure that you are using the right host ID for the communication with SYSFW. Is it same as the non virtualized case?

Please share any device tree changes that you have made(if any) to  run the kernel in virtual environment.

Also, Is your issue happening in 1st Linux VM or the 2nd VM?

Regards,

Nikhil D

Hi Nikhil,

>>Please ensure that you are using the right host ID for the communication with SYSFW. Is it same as the non virtualized case?

I am not doing any changes in the linux driver or device tree. I am using it as it is.

>>Also, Is your issue happening in 1st Linux VM or the 2nd VM?

Can you elaborate the question? I am running single instance of linux on core0-cluster-0. Core-1 of cluster-0 and cluster-1 are not switched on yet. I verified if Linux instance is running successfully on core-0 of cluster-0 and it is working fine.

Regards,

Devendra 

Hi,

>>I verified if Linux instance is running successfully on core-0 of cluster-0 and it is working fine.

Just to clarify, here I mean, I verified if Linux instance is running successfully on core-0 of cluster-0 without hypervisor code and it is working fine.

Regards,

Devendra 

Hi Devendra,

Understood, the standalone LInux works but hangs in a virtualized environment. That will lead to some suspician on the hypervisor.

Can you find more about the exact failure in terms of any primitve operations like MMR access, interrupt handling/injection in the TISCI driver?

Regards,

Nikhil D

Hi Nikhil,

Is there any way by which I can see TLB cache contains in CCS. I want to see the combined memory attributes (EL1 and EL2) which are outputed while accessing the memory/device. By observing so, I want to verify if memory attibutes at EL2 for stage-2 translation are configured correctly or not. My CCS version is 8.3.1.00004.

Regards,

Devendra 

Nikhil/Ron,

Any update?

Regards,

Devendra 

Nikhil,

>>Ideally, all device memory should be mapped uncached. So you should compare the MMU mapping attributes for the pages in stage1 and stage2

I checked the stage-2 mapping attributes and corresponding stage-1 both seems OK to me. I compared it with the help of lauterbach debugger. 

From the perspective of SCI communication, can you tell me the specific attributes (peripheral memory and DMA if any).

Regards,

Devendra 

Hi Devendra,

I recommed you to compare these attributes with a working Jailhouse hypervisor.

We will know where the issue is based on these results.

Regards,

Nikhil D

Nikhil,

>>I recommed you to compare these attributes with a working Jailhouse hypervisor.

From the other TI posts, my assumption is that there is no working jailhouse on AM65XX yet. If it is wrong, let me know where can I get source code of the same. In the SDK version I am having, I do not have source code of the same.

Regards,

 Devendra 

You can take a look at the upstream jailhouse repo -https://github.com/siemens/jailhouse

It has support for AM65 IDK

Regards,

Nikhil D

Nikhil,

I am going through jailhouse code. I am not sure if it is tested on AM65xx kit or not. Because I could not find the code building and running instructions for the IDK board in the documentation. The secure proxy MMIO which I am looking @0x32c00000, I checked the memory attributes in the jailhouse code and my hypervisor code. It seems ok to me. I checked GIC distributor handling at hypervisor level and initialization is happening as per expectation. Is there anything I must look which I am currently missing?

Regards,

Devendra 

Hi Devendra,

As I said before, as long as the MMRs are accessible from VM and the interrupts are injected correctly, VM's job is already done.

Jailhouse job is done after that. Issue must be somewhere else.

On a different note, Are you using 1:1 address mapping for the kernel's IPA and physical address?

For secure proxy communication, the IPA and the physical address should be same.

Regards,

Nikhil D

Hi Nikhil,

>> as long as the MMRs are accessible from VM and the interrupts are injected correctly

Yes Nikhil, MMR's are accessible from VM and interrupts are injected properly. VM is getting interrupt 0x1B continuously processing it as well.

From VM, it tries to check the SCI version info and transmission is failing that's what I can see on console. 

>>Are you using 1:1 address mapping for the kernel's IPA and physical address

Yes I am.

For your ref, I am attaching word doc where I have captured contains of Linux and hypervisor stage-2 page table. At hypervisor level, I am maintaining 4K as page size.

If needed, we can have audio call on same as well.

Regards,

Devendra 

2526.page_tables.docx

Hi Nikhil,

Any update/comment?

As I said, if you have time, we can have audio call as well.

Regards,

Devendra 

Hi Devendra,

Unfortunately, i don't have any update.

I am out of ideas to debug this further. Nothing seems to be wrong at the hypervisor level.

Regards,

Nikhil D

Hi Nikhil,

At least can you send me the link for official jailhouse demo on AM65xx IDK board? for DRA7xx based board, the code was included in SDK and proper build and execution steps had given.

So, is official demo available for AM65xx IDK board?

Regards,

Devendra 

Nikhil,

After going to above stated link, I am seeing "You currently have no secure software" message.

Regards,

Devendra 

Hi Devendra,

You can contact TI FAEs to enable access for you.

Having a working reference Jailhouse implementation will be useful for you.

Nikhil D