This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM3351: SGX kernel segfault

Part Number: AM3351
Other Parts Discussed in Thread: DP83640

Hi, 

We are attempting to update our software on an AM335x platform to the latest versions of Linux and have made a fair amount of progress updating to Kernel 5.8 and 5.10, however, we also require support of the SGX accelerated graphics driver and I am encountering difficulties with this.

I am sure that the problem I have is in the device tree, and in particular that a clock is probably not being enabled for the SGX device. I have built a Kernel with support for the SGX kernel module from the git repository git://git.ti.com/graphics/omap5-sgx-ddk-linux.git, using the ti-img-sgx/1.17.4948957/k5.10 branch, and when I boot my device I receive the following kernel crash:

[   14.824048] 8<--- cut here ---

[   14.827158] Unhandled fault: external abort on non-linefetch (0x1008) at 0xe03e0ca8

[   14.834861] pgd = 62a02087

[   14.837583] [e03e0ca8] *pgd=8168a811, *pte=56000653, *ppte=56000453

[   14.843917] Internal error: : 1008 [#1] ARM

[   14.848125] Modules linked in: ti_am335x_adc kfifo_buf industrialio pvrsrvkm(O) tilcdc drm_kms_helper cfbfillrect syscopyarea cfbimgblt sysfillrect sysimgblt fb_sys_fops cfbcopyarea drm drm_panel_orientation_quirks rtc_omap ti_am335x_tscadc fuse

[   14.870065] CPU: 0 PID: 374 Comm: pvrsrvctl Tainted: G           O      5.10.21-yocto-standard #1

[   14.878979] Hardware name: Generic AM33XX (Flattened Device Tree)

[   14.885496] PC is at PollForValueKM+0x14/0xf4 [pvrsrvkm]

[   14.891022] LR is at SGXResetInvalDC.constprop.0+0x68/0xa0 [pvrsrvkm]

[   14.897498] pc : [<bf1180a4>]    lr : [<bf1251a0>]    psr: 600f0013

[   14.903796] sp : c2b7bbd8  ip : 00000000  fp : 0000001c

[   14.909045] r10: c2c290c0  r9 : c29c1000  r8 : bf1380ec

[   14.914297] r7 : e03a4140  r6 : 00000000  r5 : 00000001  r4 : 00000000

[   14.920857] r3 : 000f4240  r2 : 000000ff  r1 : 00000000  r0 : e03e0ca8

[   14.927419] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none

[   14.934590] Control: 10c5387d  Table: 82ce4019  DAC: 00000051

[   14.940367] Process pvrsrvctl (pid: 374, stack limit = 0xb65adc6c)

[   14.946580] Stack: (0xc2b7bbd8 to 0xc2b7c000)

[   14.950964] bbc0:                                                       c2ca0018 c2b92940

[   14.959190] bbe0: c16fbe30 c0857ec0 bf138000 000f4240 bf1241a0 00000000 00000001 00000000

[   14.967417] bc00: e03a4140 bf1380ec c29c1000 c2c290c0 0000001c bf1251a0 00000032 00000000

[   14.975642] bc20: c29c1000 bf125288 00000000 c29c1000 00000000 bf123fd0 c29c1000 00000000

[   14.983869] bc40: 00000000 00000000 00000002 c29c1000 c2b7bd44 bf124f4c c16fbe00 c0304810

[   14.992094] bc60: c16fbd00 c16fbe00 00000000 00000000 c10d2f00 600f0013 00000014 c03049f4

[   15.000320] bc80: c2b7bca4 c16fbd00 c2d20380 c0e04288 00000000 c2d20300 00000000 00000000

[   15.008545] bca0: bf159214 bf116c88 c2b7bcd8 bf116c04 00000000 bf116354 c2b7bcd8 c0e04288

[   15.016770] bcc0: 00000000 00000000 00000000 00000000 bf116ea8 bf116c04 00000000 00000000

[   15.024995] bce0: 00000000 00000000 00000001 00000000 ffffffff c2c2a800 00000006 bf125748

[   15.033221] bd00: 00000014 c016d588 c10d2f00 30313235 c0f44604 c2c2a800 00000000 e03a41a0

[   15.041446] bd20: 00000000 e023e000 bf11d14c bf1231dc 00000000 00000000 00000000 bf116354

[   15.049671] bd40: c2b7bd70 bf116c04 00000000 bf116354 0f0031a0 c0e04288 00000000 00000000

[   15.057898] bd60: ffffffff c0e04288 00360018 c29c1000 c2b4fe80 e023d000 e023e000 bf11d14c

[   15.066125] bd80: c2c290c0 bf123354 00000000 c2c2a800 e023e000 bf117814 c2c2a800 bf117e70

[   15.074351] bda0: c2c2a800 e023e000 e023d000 bf117e84 c2c2a800 bf117e70 00000001 bf117e70

[   15.082577] bdc0: 00000000 bf116038 00000000 e023d000 e023e000 bf11d19c 00000048 c2b7be6c

[   15.090805] bde0: c283c240 bf11e988 c283c240 c2b7be6c c01c671d 00000176 c2b7be6c c2c290c0

[   15.099031] be00: c01c6748 00000000 bea9f94c bf10e0a0 0000001c c28f7800 bf10ded4 bf02e384

[   15.107256] be20: 00000000 c283c240 c2b7be6c c0e04288 c2b7be6c 0000001c c2b7be6c bf138234

[   15.115483] be40: 00000040 bf02e934 0000e280 00000001 bf134a30 00000000 401c6440 bf10ded4

[   15.123708] be60: c2832e40 00000051 00000000 00000048 0000001c bea9f984 bea9f988 00000004

[   15.131934] be80: 00000008 00000001 00000000 c2b7bed0 00444000 b6d33000 b6d12000 c2cd0ae0

[   15.140160] bea0: c283a0c8 c0e04288 b6d33000 c2b7bed0 c2cd0ae0 c283a0c0 b6d12000 c0264840

[   15.148388] bec0: c29e4600 c2cd0ae0 c283a0c0 c0260df0 c1001400 c177a510 c10ef100 c283a540

[   15.156614] bee0: b6d12000 c0260ee4 dfacb828 c0e04288 c2cd0ae0 401c6440 bea9f94c c2832e40

[   15.164841] bf00: c29ced70 00000003 c2832e40 00000036 00000000 c029f3d0 401c6440 c029fda0

[   15.173067] bf20: 00000001 c0260ee4 c29e4600 00000000 b6d12000 c0262ec8 b6d13000 00000255

[   15.181292] bf40: c29e4600 c283a540 00000800 00000000 b6d12000 c29e4644 00001000 c2b7bf7c

[   15.189518] bf60: 00000001 c29e4600 00000001 c0263564 00000001 c02635c0 c0e88f5c c0e04288

[   15.197744] bf80: c2b7bf7c c01c6748 bea9f94c 401c6440 00000036 c0100288 c2b7a000 00000036

[   15.205970] bfa0: 00000000 c0100060 c01c6748 bea9f94c 00000003 401c6440 bea9f94c 0000001c

[   15.214197] bfc0: c01c6748 bea9f94c 401c6440 00000036 b6f4fd34 b6f4fc34 b6f4fd5c 00000000

[   15.222422] bfe0: b6d51ef0 bea9f924 b6d381d8 b6e7726c 200f0010 00000003 00000000 00000000

[   15.231035] [<bf1180a4>] (PollForValueKM [pvrsrvkm]) from [<bf1251a0>] (SGXResetInvalDC.constprop.0+0x68/0xa0 [pvrsrvkm])

[   15.242401] [<bf1251a0>] (SGXResetInvalDC.constprop.0 [pvrsrvkm]) from [<bf125288>] (SGXReset+0x98/0x1cc [pvrsrvkm])

[   15.253325] [<bf125288>] (SGXReset [pvrsrvkm]) from [<bf123fd0>] (SGXInitialise+0x88/0x15c [pvrsrvkm])

[   15.263029] [<bf123fd0>] (SGXInitialise [pvrsrvkm]) from [<bf124f4c>] (SGXPostPowerState.part.0+0x5c/0x118 [pvrsrvkm])

[   15.274130] [<bf124f4c>] (SGXPostPowerState.part.0 [pvrsrvkm]) from [<bf116c88>] (PVRSRVDevicePostPowerStateKM_AnyVaCb+0x84/0x94 [pvrsrvkm])

[   15.287152] [<bf116c88>] (PVRSRVDevicePostPowerStateKM_AnyVaCb [pvrsrvkm]) from [<bf116354>] (List_PVRSRV_POWER_DEV_PVRSRV_ERROR_Any_va+0x6c/0x90 [pvrsrvkm])

[   15.301657] [<bf116354>] (List_PVRSRV_POWER_DEV_PVRSRV_ERROR_Any_va [pvrsrvkm]) from [<bf116ea8>] (PVRSRVSetDevicePowerStateKM+0x58/0x9c [pvrsrvkm])

[   15.315374] [<bf116ea8>] (PVRSRVSetDevicePowerStateKM [pvrsrvkm]) from [<bf125748>] (SGXScheduleCCBCommandKM+0x9c/0x120 [pvrsrvkm])

[   15.327607] [<bf125748>] (SGXScheduleCCBCommandKM [pvrsrvkm]) from [<bf1231dc>] (SGXGetMiscInfoUkernel+0xbc/0x170 [pvrsrvkm])

[   15.339318] [<bf1231dc>] (SGXGetMiscInfoUkernel [pvrsrvkm]) from [<bf123354>] (SGXDevInitCompatCheck+0xc4/0x4c4 [pvrsrvkm])

[   15.350855] [<bf123354>] (SGXDevInitCompatCheck [pvrsrvkm]) from [<bf117e84>] (PVRSRVFinaliseSystem_CompatCheck_AnyCb+0x14/0x4c [pvrsrvkm])

[   15.363787] [<bf117e84>] (PVRSRVFinaliseSystem_CompatCheck_AnyCb [pvrsrvkm]) from [<bf116038>] (List_PVRSRV_DEVICE_NODE_PVRSRV_ERROR_Any+0x3c/0x48 [pvrsrvkm])

[   15.378377] [<bf116038>] (List_PVRSRV_DEVICE_NODE_PVRSRV_ERROR_Any [pvrsrvkm]) from [<bf11d19c>] (PVRSRVInitSrvDisconnectBW+0x50/0x7c [pvrsrvkm])

[   15.391832] [<bf11d19c>] (PVRSRVInitSrvDisconnectBW [pvrsrvkm]) from [<bf11e988>] (BridgedDispatchKM+0x1e0/0x210 [pvrsrvkm])

[   15.403456] [<bf11e988>] (BridgedDispatchKM [pvrsrvkm]) from [<bf10e0a0>] (PVRSRV_BridgeDispatchKM+0x1cc/0x2f4 [pvrsrvkm])

[   15.415769] [<bf10e0a0>] (PVRSRV_BridgeDispatchKM [pvrsrvkm]) from [<bf02e934>] (drm_ioctl+0x2fc/0x340 [drm])

[   15.426345] [<bf02e934>] (drm_ioctl [drm]) from [<c029f3d0>] (vfs_ioctl+0x28/0x3c)

[   15.433980] [<c029f3d0>] (vfs_ioctl) from [<c029fda0>] (sys_ioctl+0xdc/0x75c)

[   15.441170] [<c029fda0>] (sys_ioctl) from [<c0100060>] (ret_fast_syscall+0x0/0x54)

[   15.448779] Exception stack(0xc2b7bfa8 to 0xc2b7bff0)

[   15.453863] bfa0:                   c01c6748 bea9f94c 00000003 401c6440 bea9f94c 0000001c

[   15.462091] bfc0: c01c6748 bea9f94c 401c6440 00000036 b6f4fd34 b6f4fc34 b6f4fd5c 00000000

[   15.470311] bfe0: b6d51ef0 bea9f924 b6d381d8 b6e7726c

[   15.475403] Code: e24dd01c e52de004 e8bd4000 e58d3014 (e5903000)

[   15.481537] ---[ end trace 7830b63b60a72976 ]---

 

I am sure that this is caused by a clock not being enabled for the SGX device, and I have searched high and low for documentation on the device tree for this driver without success.

Could you please help me solving this? 

Regards,

Jon

  • Hello Jon,

    Unfortunately, we have not tested our GPU drivers with Kernel5.10 and our last tested release was Kernel4.19.

    Regards,
    Krunal

  • Hello Krunal,

    I find this very difficult to believe or accept. In the git repo for the SGX Kernel driver git://git.ti.com/graphics/omap5-sgx-ddk-linux.git there is a branch remotes/origin/ti-img-sgx/1.17.4948957/k5.10 which as per TI's convention would be for support for the SGX driver in a 5.10 kernel.

    I do notice that there is no updated recipe in meta-ti for the 5.10 kernel with SGX support, but I would assume that if there is a 5.10 branch for the driver that someone is actually using it?

    Does your statement mean that TI is not supporting this driver in 5.x kernels? I do notice that there is both a 5.4 and 5.10 branch for the driver.

    We have a new product that requires a 5.8 kernel (minimum) because we require support of Bluetooth 5.1 with 2Mbit PHY's which are only supported from Kernel 5.4 and later. We also use PTP extensively and we have now migrated to using the cpsw_new driver which supports our requirement fully, wheras in the 4.19 kernel we had an extensive set of patches to both the cpsw driver and the dp83640 PHY driver, but with the new cpsw drivers these patches do not apply at all, mainly because the PTP patches intercept packets and redirect them to other interfaces internally, and with the new cpsw drivers, the core networking has been switched to XDP. That being said though, the 5.8/5.10 kernel satisfies our needs fully in relation to PTP without us having to patch the kernel, which is a very good thing.

    I am not really concerned if you have tested the GPU drivers with a 5.10 kernel or not, but what would be very useful is proper device tree binding documentation for that driver. I have been unable to find any at all. I guess I could go through the code and generate my own device tree documentation, but as a client of TI, I would expect at the very least that this be documented by TI.

    I would be very receptive to further support if possible

    With kind regards
    Hamish Guthrie

  • Hamish,

    https://git.ti.com/cgit/graphics/omap5-sgx-ddk-linux/log/?h=ti-img-sgx/1.17.4948957/k5.10 is a work in progress branch. We are in middle of a kernel migration (as you can see from the graphics and other kernel/bootloader public repositories updates over the last few weeks)

    Unless you dont see a processor sdk release where everything is verified and released, i dont recomend you to try the intermediate baseline. 

    This is not something we can support until everything is ready and released as processor-sdk.

    Just as a process to make you aware,  some useful info.

    LTS kernel migration for all planned platforms is something we do always. you can look at processor sdk x.y for the past years and see when a first lts based release is available. 

    As part of LTS migration each feature is migrated and verified before a processor sdk release is made. 

    If you see the mentioned branch , it is something we started recently. as you can see that the migration is work in progress.

    In short, i would recomend to take the right combination of processor sdk available today .

    Hope it clarifies.

  • Do you have a timeline as to when the release of an SDK will be taking place with support for 5.10 kernel?