This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/AM5728: jailhouse hypervisor - inter cell communication over uio-ivshmem

Part Number: AM5728
Other Parts Discussed in Thread: TPIC2810

Tool/software: Linux

Hi,


I'm developing over IDK am5728 with PROCESSOR-SDK-LINUX-AM57X version v05.00 and PROCESSOR-SDK-RTOS-AM57X version v05.00.

I'm using the jailhouse hypervisor and I'm able to run linux on core 0 and on core 1 the jailhouse inmate demo processor_sdk_rtos_am57xx_5_00_00_15/demos/jailhouse-inmate/rtos/icss_emac

Now I'm trying to create inter cell communication. I need the cores to be able to send interrupts to one another.

I'm using the uio-ivshmem module from https://github.com/henning-schild-work/ivshmem-guest-code

To simplify things, first I'm trying to send an interrupt from a baremetal inmate to the root-cell Linux. Later I will try with RTOS inmate.

Once I have that working I will need the RTOS inmate to receive an interrupt from the Linux root cell.

In the dts file some GIC inputs are reserved for the inmate:

/ {
        ocp {
                pruss1_eth {
                        status = "disabled";
                };

                pruss2_eth {
                        status = "disabled";
                };

                crossbar_mpu: crossbar@4a002a48 {
                        ti,irqs-skip = <10 44 127 129 133 134 135 136 137 139 140>;
                };
        };
};

in the root cell I set
.vpci_irq_base = 100 - 32

and in the inmate cell
.vpci_irq_base = 134 - 32,

pci and uio_ivshmem init seem fine

[   45.928817] OF: PCI: host bridge /vpci@0 ranges:
[   45.933482] OF: PCI:   MEM 0x30100000..0x30101fff -> 0x30100000
[   45.939690] pci-host-generic 30000000.vpci: ECAM at [mem 0x30000000-0x300fffff] for [bus 00]
[   45.948567] pci-host-generic 30000000.vpci: PCI host bridge to bus 0001:00
[   45.956098] pci_bus 0001:00: root bus resource [bus 00]
[   45.961539] pci_bus 0001:00: root bus resource [mem 0x30100000-0x30101fff]
[   45.968480] pci 0001:00:00.0: [1af4:1110] type 00 class 0xff0000
[   45.968517] pci 0001:00:00.0: reg 0x10: [mem 0x00000000-0x000000ff 64bit]
[   45.968881] PCI: bus0: Fast back to back transfers disabled
[   45.974591] pci 0001:00:00.0: BAR 0: assigned [mem 0x30100000-0x301000ff 64bit]
[   45.982475] virtio-pci 0001:00:00.0: enabling device (0000 -> 0002)
[   45.988997] uio_ivshmem 0001:00:00.0: using jailhouse mode
[   45.995644] The Jailhouse is opening.
[   46.603491] Created Jailhouse cell "AM572X-IDK-ICSS"


root@am57xx-evm:~# lspci -v


0000:00:00.0 PCI bridge: Texas Instruments Multicore DSP+ARM KeyStone II SOC (rev 01) (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 170
        Memory at 20100000 (64-bit, non-prefetchable) [size=1M]
        Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0

        I/O behind bridge: None
        Memory behind bridge: None
        Prefetchable memory behind bridge: None


        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Root Port (Slot-), MSI 00
        Capabilities: [100] Advanced Error Reporting
        Kernel driver in use: pcieport

0001:00:00.0 Unassigned class [ff00]: Red Hat, Inc Inter-VM shared memory


        Subsystem: Red Hat, Inc Inter-VM shared memory

        Flags: bus master, fast devsel, latency 0, IRQ 179


        Memory at 30100000 (64-bit, non-prefetchable) [size=256]
        Kernel driver in use: uio_ivshmem

Still, I can't generate an interrupt from the inmate to the root cell.

cat /proc/interrupts
...
179:          0     GICv2 100 Edge      uio_ivshmem
...

If I understand correctly, I need to write the value 1 to the address 0x3010000c. I'm using gic-demo to do so from the interrupt context.
Whenever the inmate writes to this address I get a dump from the kernel

[   53.306728] ------------[ cut here ]------------
[   53.311518] WARNING: CPU: 0 PID: 28 at /home/stx-ti/Projects/tisdk/build/arago-tmp-external-linaro-toolchain/work-shared/am57xx-evm/kernel-source/drivers/bus/omap_l3_noc.c:147 l3_interrupt_handler+0x254/0x370
[   53.330115] 44000000.ocp:L3 Standard Error: MASTER MPU TARGET PCIE2 (Read): At Address: 0x0010000C : Data Access in User mode during Functional access
[   53.343649] Modules linked in: jailhouse(O) uio_ivshmem(O) uio can_raw can ecatmc r8169 mc_gp_timer ec_master xhci_plat_hcd xhci_hcd usbcore dwc3 udc_core usb_common ti_prueth pru_rproc pruss pruss_intc snd_soc_omap_hdmi_audio omap_aes_driver omap_sham pruss_soc_bus c_can_platform c_can can_dev omap_wdt ahci_platform libahci_platform libahci libata scsi_mod ti_vpe ti_vip ti_sc ti_csc ti_vpdma dwc3_omap rtc_omap extcon_palmas rtc_palmas gpio_pisosr ov2659 omap_des gpio_tpic2810 v4l2_fwnode des_generic crypto_engine omap_crypto omap_remoteproc virtio_rpmsg_bus rpmsg_core remoteproc sch_fq_codel cryptodev(O)
[   53.397569] CPU: 0 PID: 28 Comm: irq/23-l3-app-i Tainted: G        W  O    4.14.40-rt29-gd5443cbd3b #8
[   53.397572] Hardware name: Generic DRA74X (Flattened Device Tree)
[   53.397574] Backtrace: 
[   53.397590] [<c020b220>] (dump_backtrace) from [<c020b504>] (show_stack+0x18/0x1c)
[   53.397597]  r7:00000009 r6:60000013 r5:00000000 r4:c105773c
[   53.397606] [<c020b4ec>] (show_stack) from [<c091609c>] (dump_stack+0x90/0xa4)
[   53.397615] [<c091600c>] (dump_stack) from [<c022a864>] (__warn+0xec/0x104)
[   53.397620]  r7:00000009 r6:c0bc94c8 r5:00000000 r4:d422fe40
[   53.397628] [<c022a778>] (__warn) from [<c022a8bc>] (warn_slowpath_fmt+0x40/0x48)
[   53.397634]  r9:00000011 r8:d4206850 r7:c0bc9334 r6:00000002 r5:c0bc93e8 r4:c0bc9498
[   53.397642] [<c022a880>] (warn_slowpath_fmt) from [<c051a5f0>] (l3_interrupt_handler+0x254/0x370)
[   53.397645]  r3:d42066c0 r2:c0bc9498
[   53.397647]  r4:80080001
[   53.397654] [<c051a39c>] (l3_interrupt_handler) from [<c027fc64>] (irq_forced_thread_fn+0x28/0x7c)
[   53.397659]  r10:c027fc3c r9:d4206bc0 r8:d41df100 r7:00000001 r6:00000000 r5:d41df100
[   53.397661]  r4:d4206bc0
[   53.397667] [<c027fc3c>] (irq_forced_thread_fn) from [<c027ffc0>] (irq_thread+0x130/0x208)
[   53.397671]  r7:00000001 r6:00000000 r5:ffffe000 r4:d4206be4
[   53.397678] [<c027fe90>] (irq_thread) from [<c02481e4>] (kthread+0x164/0x16c)
[   53.397683]  r10:d406bb28 r9:c027fe90 r8:d4206bc0 r7:d422e000 r6:00000000 r5:d4206c00
[   53.397685]  r4:d421e040
[   53.397693] [<c0248080>] (kthread) from [<c02079a0>] (ret_from_fork+0x14/0x34)
[   53.397698]  r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c0248080
[   53.397700]  r4:d4206c00
[   53.397706] ---[ end trace 0000000000000004 ]---


and a different interrupt is incremented:
cat /proc/interrupts
...
23:          3     WUGEN  10 Level     l3-app-irq
...


Is there anything wrong with the cells configuration?

Is writing to 0x3010000c the right way to trigger the uio_ivshmem interrupt in the Linux side?

Can TI supply a working setup for uio-ivshmem comm between a linux root cell and a rtos inmate as a starting point?

Thanks a lot,

Nir.

am572x-rtos-icss.c
/*
 * Jailhouse, a Linux-based partitioning hypervisor
 *
 * Configuration for uart-demo inmate on AM57XX-EVM:
 *
 * Copyright (c) Texas Instruments, Inc.
 *
 * Authors:
 *  Vitaly Andrianiov <vitalya@ti.com>
 *
 * This work is licensed under the terms of the GNU GPL, version 2.  See
 * the COPYING file in the top-level directory.
 */

#include <jailhouse/types.h>
#include <jailhouse/cell-config.h>

#define ARRAY_SIZE(a) sizeof(a) / sizeof(a[0])

struct {
	struct jailhouse_cell_desc cell;
	__u64 cpus[1];
	struct jailhouse_memory mem_regions[10];
	struct jailhouse_irqchip irqchips[1];
	struct jailhouse_pci_device pci_devices[1];
} __attribute__((packed)) config = {
	.cell = {
		.signature = JAILHOUSE_CELL_DESC_SIGNATURE,
		.revision = JAILHOUSE_CONFIG_REVISION,
		.name = "AM572X-IDK-ICSS",
		.flags = JAILHOUSE_CELL_PASSIVE_COMMREG,
		.num_irqchips = 1,
		.cpu_set_size = sizeof(config.cpus),
		.num_memory_regions = ARRAY_SIZE(config.mem_regions),
		.num_pci_devices = ARRAY_SIZE(config.pci_devices),
		.vpci_irq_base = 134 - 32,
	},

	.cpus = {
		0x2,
	},

	.mem_regions = {
		/* OCRAM buffer area */ {
			.phys_start = 0x40340000,
			.virt_start = 0x40340000,
			.size = 0xC0000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED,
		},
		/* UART... */ {
			.phys_start = 0x48020000,
			.virt_start = 0x48020000,
			.size = 0x1000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED,
		},
		/* ... */ {
			.phys_start = 0x48050000,
			.virt_start = 0x48050000,
			.size = 0x10000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED,
		},
		{
		/* TIMER8 */
			.phys_start = 0x48826000,
			.virt_start = 0x48826000,
			.size = 0x1000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED,
		},
		/* L4_CFG */ {
			.phys_start = 0x4a000000,
			.virt_start = 0x4a000000,
			.size = 0xE10000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED,
		},
		/* PRUSS */ {
			.phys_start = 0x4b200000,
			.virt_start = 0x4b200000,
			.size = 0x100000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED,
		},
		/* RAM loader */ {
			.phys_start = 0xed000000,
			.virt_start = 0x0,
			.size = 0x10000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_EXECUTE | JAILHOUSE_MEM_LOADABLE,
		},
		/* RAM RTOS 224MB*/ {
			.phys_start = 0xe0000000,
			.virt_start = 0x80000000,
			.size = 0xd000000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_EXECUTE | JAILHOUSE_MEM_LOADABLE,
		},
		/* IVSHMEM registers for PCI 0001:00:00.0 */ {
			.phys_start = 0x30100000,
			.virt_start = 0x30100000,
			.size = 0x1000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED,
		},
		/* IVSHMEM shared memory region for 0001:00:00.0 */ {
			.phys_start = 0xEEE00000,
			.virt_start = 0xEEE00000,
			.size = 0x100000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_ROOTSHARED,
                },
	},
	.irqchips = {
		/* GIC */ {
			.address = 0x48211000,
			.pin_base = 64,
			.pin_bitmap = {
				0x00001000, 0x0, 0x80000000, 0x000003C2,
			},
		},
	},
	.pci_devices = {
		{ /* 02:00.0 */
			.type = JAILHOUSE_PCI_TYPE_IVSHMEM,
			//.iommu = 1,
			.bdf = 0 << 3,
			.bar_mask = {
				0xffffff00, 0xffffffff, 0x00000000,
				0x00000000, 0x00000000, 0x00000000,
			},
			.num_msix_vectors = 0,
			.shmem_region = 9,
			.shmem_protocol = JAILHOUSE_SHMEM_PROTO_UNDEFINED,
		},  
	},
};
gic-demo.c
/*
 * Jailhouse, a Linux-based partitioning hypervisor
 *
 * Copyright (c) ARM Limited, 2014
 * Copyright (c) Siemens AG, 2014-2017
 *
 * Authors:
 *  Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
 *  Jan Kiszka <jan.kiszka@siemens.com>
 *
 * This work is licensed under the terms of the GNU GPL, version 2.  See
 * the COPYING file in the top-level directory.
 */

#include <mach.h>
#include <inmate.h>

#include "ti-utils.h"

#define BEATS_PER_SEC		10

static u64 ticks_per_beat;
static volatile u64 expected_ticks;

/*
 * Enables blinking LED
 * Banana Pi:           register 0x1c2090c, pin 24
 * Orange Pi Zero:      register 0x1c20810, pin 17
 */
static void *led_reg;
static unsigned int led_pin;

unsigned long counter = 0;
u32* shmem = (unsigned int*)0xeee00000;
u32* regs = (unsigned int*)0x30100000;

static void handle_IRQ(unsigned int irqn)
{
	static u64 min_delta = ~0ULL, max_delta = 0;
	u64 delta;

	if (irqn != TIMER_IRQ)
		return;

	delta = timer_get_ticks() - expected_ticks;
	if (delta < min_delta)
		min_delta = delta;
	if (delta > max_delta)
		max_delta = delta;

	++counter;

	/*printk("Timer fired, jitter: %6ld ns, min: %6ld ns, max: %6ld ns\n",
	       (long)timer_ticks_to_ns(delta),
	       (long)timer_ticks_to_ns(min_delta),
	       (long)timer_ticks_to_ns(max_delta));*/

	ti_putsui("Timer fired, jitter: ", (long)timer_ticks_to_ns(delta), false);
	ti_putsui(" ns, min: ", (long)timer_ticks_to_ns(min_delta), false);
	ti_putsui(" ns, max: ", (long)timer_ticks_to_ns(max_delta), false);
	ti_puts(" ns\n\r");

	if(-1 == counter)
	{
		*shmem = counter;
		mmio_write32(regs + 3, 1);
		counter = 0;
	}

/*	if (led_reg)
		mmio_write32(led_reg, mmio_read32(led_reg) ^ (1 << led_pin));*/

	expected_ticks = timer_get_ticks() + ticks_per_beat;
	timer_start(ticks_per_beat);
}

void inmate_main(void)
{
	/*printk("Initializing the GIC...\n");*/
	ti_puts("Initializing the GIC...\n\r");
	gic_setup(handle_IRQ);
	gic_enable_irq(TIMER_IRQ);

	/*printk("Initializing the timer...\n");*/
	ti_puts("Initializing the timer...\n\r");

	ticks_per_beat = timer_get_frequency() / BEATS_PER_SEC;
	expected_ticks = timer_get_ticks() + ticks_per_beat;
	timer_start(ticks_per_beat);

	led_reg = (void *)(unsigned long)cmdline_parse_int("led-reg", 0);
	led_pin = cmdline_parse_int("led-pin", 0);

	while (1)
		asm volatile("wfi" : : : "memory");
}
pci_am57xx-evm.c
/*
 * Jailhouse, a Linux-based partitioning hypervisor
 *
 * Copyright (c) Texas Insturments. Inc, 2016
 *
 * Authors:
 *  Vitaly Andrianov <vitalya@ti.com>
 *
 * This work is licensed under the terms of the GNU GPL, version 2.  See
 * the COPYING file in the top-level directory.
 */

#include <jailhouse/types.h>
#include <jailhouse/cell-config.h>

#define ARRAY_SIZE(a) sizeof(a) / sizeof(a[0])

struct {
	struct jailhouse_system header;
	__u64 cpus[1];
	struct jailhouse_memory mem_regions[19];
	struct jailhouse_irqchip irqchips[2];
	struct jailhouse_pci_device pci_devices[1];
} __attribute__((packed)) config = {
	.header = {
		.signature = JAILHOUSE_SYSTEM_SIGNATURE,
		.revision = JAILHOUSE_CONFIG_REVISION,
		.hypervisor_memory = {
			.phys_start = 0xef000000,
			.size = 0x1000000,
		},
		.debug_console = {
			.address = 0x48020000,
			.size = 0x1000,
			/* .divider = 26, */
			.flags = JAILHOUSE_CON1_TYPE_8250 |
				JAILHOUSE_CON1_ACCESS_MMIO |
				JAILHOUSE_CON1_REGDIST_4 |
				JAILHOUSE_CON2_TYPE_ROOTPAGE,
		},
		.platform_info = {
			.pci_mmconfig_base = 0x30000000,
			.pci_mmconfig_end_bus = 0,
			.pci_is_virtual = 1,
			.arm = {
				.gic_version = 2,
				.gicd_base = 0x48211000,
				.gicc_base = 0x48212000,
				.gich_base = 0x48214000,
				.gicv_base = 0x48216000,
				.maintenance_irq = 25,
			},
		},
		.root_cell = {
			.name = "AM57XX-EVM",
			.cpu_set_size = sizeof(config.cpus),
			.num_memory_regions = ARRAY_SIZE(config.mem_regions),
			.num_irqchips = ARRAY_SIZE(config.irqchips),
			.num_pci_devices = ARRAY_SIZE(config.pci_devices),
			.vpci_irq_base = 100 - 32,
		},
	},

	.cpus = {
		0x3,
	},

	.mem_regions = {
		/* PCI */ {
			.phys_start = 0x30000000,
			.virt_start = 0x30000000,
			.size = 0x001fffff,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},
		/* OCMCRAM */ {
			.phys_start = 0x40300000,
			.virt_start = 0x40300000,
			.size = 0x80000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},
		/* 0x40380000 - 0x48020000 */ {
			.phys_start = 0x40380000,
			.virt_start = 0x40380000,
			.size = 0x7ca0000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},
		/* UART... */ {
			.phys_start = 0x48020000,
			.virt_start = 0x48020000,
			.size = 0xe0000,//0x00001000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},
		/* 0x48100000 - 0x48281000 */ {
			.phys_start = 0x48100000,
			.virt_start = 0x48100000,
			.size = 0x110000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},
		/*
		 * Leave gap for GIC controller 0x48210000 - 0x41220000
		 */
		/* 0x48220000 - 0x48281000 */ {
			.phys_start = 0x48220000,
			.virt_start = 0x48220000,
			.size = 0x610000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},
		/* OMAP WakeupGen */ {
			.phys_start = 0x48281000,
			.virt_start = 0x48281000,
			.size = 0x1000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},
		/* PRCM MPU */ {
			.phys_start = 0x48243000,
			.virt_start = 0x48243000,
			.size = 0x1000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},
		/* 0x48400000 - 0x48424000 */ {
			.phys_start = 0x48400000,
			.virt_start = 0x48400000,
			.size = 0x24000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},
		/* UART... */ {
			.phys_start = 0x48424000,
			.virt_start = 0x48424000,
			.size = 0x2000,//0x00001000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},

		/* 0x48426000 - 0x48826000 */ {
			.phys_start = 0x48426000,
			.virt_start = 0x48426000,
			.size = 0x400000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},
		/* 0x48826000 - 0x48828000 */ {
			.phys_start = 0x48826000,
			.virt_start = 0x48826000,
			.size = 0x2000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},
		/* 0x48828000 - 0x4B300000 */ {
			.phys_start = 0x48828000,
			.virt_start = 0x48828000,
			.size = 0x2ad8000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},

		/* 0x4B500000 - 0x58000000 */ {
			.phys_start = 0x4B500000,
			.virt_start = 0x4B500000,
			.size = 0xCB00000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},
		/* Display Subsystem & ...*/ {
			.phys_start = 0x58000000,
			.virt_start = 0x58000000,
			.size = 0x8000000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},
		/* RAM */ {
			.phys_start = 0x80000000,
			.virt_start = 0x80000000,
			.size = 0x6EE00000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_EXECUTE,
		},
		/* IVSHMEM shared memory region for 0001:00:00.0 */ {
			.phys_start = 0xEEE00000,
			.virt_start = 0xEEE00000,
			.size = 0x100000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_ROOTSHARED,
                },
		/* communication region */ {
			.phys_start = 0xEEF00000,
			.virt_start = 0xEEF00000,
			.size = 0x00100000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_IO,
		},
		/* Leave hole for hypervisor */

		/* RAM */ {
			.phys_start = 0xF0000000,
			.virt_start = 0xF0000000,
			.size = 0x10000000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
				JAILHOUSE_MEM_EXECUTE,
		},
	},
	.irqchips = {
		/* GIC */ {
			.address = 0x48211000,
			.pin_base = 32,
			.pin_bitmap = {
				0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff
			},
		},
		/* GIC */ {
			.address = 0x48211000,
			.pin_base = 160,
			.pin_bitmap = {
				0xffffffff, 0, 0, 0
			},
		},
	},
	.pci_devices = { 
		{ /* 02:00.0 */
			.type = JAILHOUSE_PCI_TYPE_IVSHMEM,
			//.iommu = 1,
			.bdf = 0 << 3,
			.bar_mask = {
				0xffffff00, 0xffffffff, 0x00000000,
				0x00000000, 0x00000000, 0x00000000,
			},
			.num_msix_vectors = 0,
			.shmem_region = 16,
			.shmem_protocol = JAILHOUSE_SHMEM_PROTO_UNDEFINED,
		},  
	},
};

  • Hi Nir,

    The kernel message seems indicating the inmate is not allowed to use the memory address 0x0010000C (0x3010000c). But since the Processor SDK doesn't provide a demo showing how to use ivshmem, I am unable to comment what the problem is in your test.

    I noticed you are also contacting the Jailhouse community for this issue, so let's first see if the issue can solved there.
  • The inmate configuration allows the use of this memory segment:

    /* IVSHMEM registers for PCI 0001:00:00.0 */ {
    .phys_start = 0x30100000,
    .virt_start = 0x30100000,
    .size = 0x1000,
    .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
    JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED,
    },


    And why is the kernel reporting 0x0010000C ?
  • Nir,

    The kernel dump seems indicating a problem on the Linux side, not the inmate.
    I haven't read the ivshmem implementation, but I guess 0x0010000C is in the memory region of the PCI remapping in Linux. The inmate uses memory segment 0x30100000 and exposes it as a PCI interface through ivshmem, Linux remap the PCI memory. Make sense?
  • When creating the PCI device o the on the Linux side:
    [ 45.933482] OF: PCI: MEM 0x30100000..0x30101fff -> 0x30100000


    In the gic-demo :
    ...
    u32* regs = (unsigned int*)0x30100000;
    ...
    mmio_write32(regs + 3, 1);


    I don't understand where 0x0010000C is coming from...

    Could this be a problem in the irqchip configuration in the cell files?
  • I am still reading to try to understand where 0x0010000C comes from...
    But now can you try "mmio_write32(regs, 1);" instead in the inmate app to see if setting the interrupt mask will trigger the same omap_l3_noc error on the root cell? Setting the interrupt mask is just a local memory access and it shouldn't communicate anything to the root cell, I believe.
  • The result is exactly the same

    Do you think there's something wrong with the cells configuration?
    Can I provide any additional information that will help us set up uio_ivshmem interrupts between the cells?
  • Hi Nir,

    I assume it is cells configuration related, but I could be wrong. There is no demo in the Processor SDK to show how to use ivshmem and I didn't use it before, so I don't have any additional info regarding ivshmem. Sorry.