Linux/AM5728: Wakeup error on PCI restore

Stefan Lange15

Part Number: AM5728

Tool/software: Linux

Hello TI team,

we have created a custom AM5728 SOM and baseboard.

In the process of creating the U-Boot and Linux BSP, we have encountered the following problem:

We are testing suspend and wakeup functionality. Upon waking up, kernel shows an error message involving pci_restore.

root@tqma572x-mba57xx:~# echo -n freeze > /sys/power/state
[   14.690323] PM: Syncing filesystems ... done.
[   14.744965] Freezing user space processes ... [   14.750629] (elapsed 0.001 seconds) done.
[   14.754674] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
[   14.774960] PM: suspend of devices complete after 10.100 msecs
[   14.784138] PM: late suspend of devices complete after 3.314 msecs
[   14.793387] PM: noirq suspend of devices complete after 2.998 msecs
<< push gpio button or keyboard button >>
[   18.662299] Unable to handle kernel NULL pointer dereference at virtual address 00000028
[   18.670472] pgd = c0003000
[   18.673219] [00000028] *pgd=80000080004003, *pmd=00000000
[   18.678679] Internal error: Oops: 207 [#1] PREEMPT SMP ARM
[   18.684185] Modules linked in:
[   18.687262] CPU: 0 PID: 303 Comm: kworker/u4:2 Not tainted 4.9.77-rt61-03208-g2b96d14-dirty #57
[   18.695995] Hardware name: Generic DRA74X (Flattened Device Tree)
[   18.702121] Workqueue: events_unbound async_run_entry_fn
[   18.707462] task: edfdba80 task.stack: eddac000
[   18.712015] PC is at pci_restore_msi_state+0x178/0x1fc
[   18.717176] LR is at pci_bus_read_config_word+0x84/0x90
[   18.722425] pc : [<c053cef8>]    lr : [<c051e734>]    psr: 60070013
[   18.722425] sp : eddaddd0 ip : eddadd98 fp : eddaddf4
[   18.733953] r10: c1003d00 r9 : 00000000 r8 : ee072068
[   18.739200] r7 : ee0723dc r6 : 11f8dc20 r5 : ee072000 r4 : 00000000
[   18.745755] r3 : 00000000 r2 : 00000000 r1 : 60070013 r0 : 00000000
[   18.752311] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[   18.759476] Control: 30c5387d Table: add31bc0 DAC: fffffffd
[   18.765246] Process kworker/u4:2 (pid: 303, stack limit = 0xeddac210)
[   18.771714] Stack: (0xeddaddd0 to 0xeddae000)
[   18.776089] ddc0:                                     ee0723dc 0080002c 11f8dc20 00000000
[   18.784302] dde0: ee072000 11f8dc20 eddade14 eddaddf8 c0526b54 c053cd8c c10254fc ee072000
[   18.792514] de00: ee072068 00000000 eddade24 eddade18 c0526c84 c0526a48 eddade44 eddade28
[   18.800728] de20: c052b8d0 c0526c74 00000001 c052b8a8 00000000 00000000 eddade7c eddade48
[   18.808940] de40: c05f5a10 c052b8b4 00000000 c1003d00 00000000 00000000 c08c29e4 00000001
[   18.817150] de60: ee072068 00000010 ee807d00 00000000 eddade9c eddade80 c05f5b30 c05f59e0
[   18.825362] de80: c108c868 ee072068 edf3b980 ee807d00 eddadeb4 eddadea0 c05f5c1c c05f5ab0
[   18.833573] dea0: edf3b990 c1050020 eddadedc eddadeb8 c024e04c c05f5c04 00000000 c1003d00
[   18.841786] dec0: edf3b990 edeeaa80 ee804a00 ee807d00 eddadf14 eddadee0 c0244ca0 c024e008
[   18.849998] dee0: eddadefc eddadef0 c024613c edeeaa80 ee804a00 edeeaa98 ee804a18 ffffe000
[   18.858212] df00: 00000088 c1003d00 eddadf54 eddadf18 c0245bb8 c0244ad0 c0b91384 c10163b4
[   18.866424] df20: c104e566 ee804a00 00000000 00000000 edf3b280 eddac000 edeeaa80 c0245b50
[   18.874636] df40: 00000000 00000000 eddadfac eddadf58 c024ad74 c0245b5c eddadf58 00000000
[   18.882849] df60: 000ab010 edeeaa80 00000000 00000000 eddadf70 eddadf70 00000000 00000000
[   18.891062] df80: eddadf80 eddadf80 eddadfac edf3b280 c024ac64 00000000 00000000 00000000
[   18.899274] dfa0: 00000000 eddadfb0 c0207de8 c024ac70 00000000 00000000 00000000 00000000
[   18.907485] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[   18.915698] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 e3530c32 1a00002e
[   18.923906] Backtrace:
[   18.926378] [<c053cd80>] (pci_restore_msi_state) from [<c0526b54>] (pci_restore_state.part.26+0x118/0x22c)
[   18.936072] r6:11f8dc20 r5:ee072000 r4:00000000
[   18.940713] [<c0526a3c>] (pci_restore_state.part.26) from [<c0526c84>] (pci_restore_state+0x1c/0x20)
[   18.949883] r7:00000000 r6:ee072068 r5:ee072000 r4:c10254fc
[   18.955572] [<c0526c68>] (pci_restore_state) from [<c052b8d0>] (pci_pm_resume_noirq+0x28/0x9c)
[   18.964227] [<c052b8a8>] (pci_pm_resume_noirq) from [<c05f5a10>] (dpm_run_callback+0x3c/0xd0)
[   18.972789] r7:00000000 r6:00000000 r5:c052b8a8 r4:00000001
[   18.978473] [<c05f59d4>] (dpm_run_callback) from [<c05f5b30>] (device_resume_noirq+0x8c/0x154)
[   18.987122] r8:00000000 r7:ee807d00 r6:00000010 r5:ee072068 r4:00000001
[   18.993855] [<c05f5aa4>] (device_resume_noirq) from [<c05f5c1c>] (async_resume_noirq+0x24/0x4c)
[   19.002590] r7:ee807d00 r6:edf3b980 r5:ee072068 r4:c108c868
[   19.008277] [<c05f5bf8>] (async_resume_noirq) from [<c024e04c>] (async_run_entry_fn+0x50/0x120)
[   19.017011] r5:c1050020 r4:edf3b990
[   19.020603] [<c024dffc>] (async_run_entry_fn) from [<c0244ca0>] (process_one_work+0x1dc/0x3f8)
[   19.029250] r7:ee807d00 r6:ee804a00 r5:edeeaa80 r4:edf3b990
[   19.034933] [<c0244ac4>] (process_one_work) from [<c0245bb8>] (worker_thread+0x68/0x5c0)
[   19.043060] r10:c1003d00 r9:00000088 r8:ffffe000 r7:ee804a18 r6:edeeaa98 r5:ee804a00
[   19.050920] r4:edeeaa80
[   19.053466] [<c0245b50>] (worker_thread) from [<c024ad74>] (kthread+0x110/0x118)
[   19.060896] r10:00000000 r9:00000000 r8:c0245b50 r7:edeeaa80 r6:eddac000 r5:edf3b280
[   19.068755] r4:00000000
[   19.071305] [<c024ac64>] (kthread) from [<c0207de8>] (ret_from_fork+0x14/0x2c)
[   19.078557] r8:00000000 r7:00000000 r6:00000000 r5:c024ac64 r4:edf3b280
[   19.085289] Code: e24b301e e2822002 e5950008 ebff85ed (e5d43028)
[   19.091463] ---[ end trace 6f217160e2d510fe ]---
< hang >

In U-Boot no_console_suspend=1 is passed to the kernel.

How should I debug this?

We are using ti-rt-linux-4.9.y from git://git.ti.com/ti-linux-kernel/ti-linux-kernel.git
and mainline denx u-boot 2018.01.

See attached our device tree and defconfig:

/cfs-file/__key/communityserver-discussions-components-files/791/5277.tqma57xx_5F00_defconfig.txt

/cfs-file/__key/communityserver-discussions-components-files/791/4520.tqma572x.dtsi.txt

/cfs-file/__key/communityserver-discussions-components-files/791/2061.tqma572x_2D00_mba57xx.dts.txt

(I had to rename the files to a .txt ending do to .dts ending beeing blocked with inserted files here)

Best regards,

Stefan

over 7 years ago

0 Pavel Botev over 7 years ago

TI__Guru**** 170625 points

Hi Stefan,

Stefan Lange15 said:
We are testing suspend and wakeup functionality. Upon waking up, kernel shows an error message involving pci_restore.

root@tqma572x-mba57xx:~# echo -n freeze > /sys/power/state

Please test suspend with "echo standby" and/or "echo mem", see the below wiki:

http://processors.wiki.ti.com/index.php/Linux_Core_Power_Management_User's_Guide_(v4.4)

Stefan Lange15 said:
We are using ti-rt-linux-4.9.y from git://git.ti.com/ti-linux-kernel/ti-linux-kernel.git

Please try with AM572x TI PSDK v4.03 (kernel 4.9.69):

Linux Kernel

The kernel git repository, branch and commit id can be found below:
Based on Version: 4.9.69
URL: git://git.ti.com/processor-sdk/processor-sdk-linux.git
Branch: processor-sdk-linux-04.03.00
Commit ID: bfe793963ed7b2af3e1c3dd32e7275f449e6259d

0 Stefan Lange15 over 7 years ago in reply to Pavel Botev

Intellectual 345 points

Hello Pavel,

Please test suspend with "echo standby" and/or "echo mem", see the below wiki:

I tried "mem", with a similar behaviour (see below).

I rechecked (I did not state it thouroughly in the first place).

We are using ti-rt-linux-4.9.77 from git://git.ti.com/ti-linux-kernel/ti-linux-kernel.git

I will try with the 4.9.69 sdk kernel soon.

root@tqma572x-mba57xx:~# echo -n standby > /sys/power/state

sh: write error: Invalid argument

root@tqma572x-mba57xx:~# echo -n mem > /sys/power/state

[ 38.219691] PM: Syncing filesystems ... done.

[ 38.265043] Freezing user space processes ... [ 38.270625] (elapsed 0.001 seconds) done.

[ 38.274670] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.

[ 38.295581] PM: suspend of devices complete after 10.766 msecs

[ 38.304879] PM: late suspend of devices complete after 3.400 msecs

[ 38.314250] PM: noirq suspend of devices complete after 3.157 msecs

[ 38.320588] Disabling non-boot CPUs ...

[ 38.365572] CPU1: shutdown

< push button >

[ 38.396544] Successfully put all powerdomains to target state

[ 38.396544] Suspended for 15.241 seconds

[ 38.396582] Enabling non-boot CPUs ...

[ 38.425769] CPU1: smp_ops.cpu_die() returned, trying to resuscitate

[ 38.432086] genirq: __irq_set_trigger chip GIC-0, irq 18, flags 8, parent 0x0

[ 38.439344] genirq: __irq_set_trigger chip GIC-0, irq 19, flags 8, parent 0x0

[ 38.447010] CPU1 is up

[ 38.450571] Unable to handle kernel NULL pointer dereference at virtual address 00000028

[ 38.459578] pgd = c0003000

[ 38.462298] [00000028] *pgd=80000080004003, *pmd=00000000

[ 38.467773] Internal error: Oops: 207 [#1] PREEMPT SMP ARM

[ 38.473279] Modules linked in:

[ 38.476357] CPU: 0 PID: 294 Comm: kworker/u4:4 Not tainted 4.9.77-rt61-03213-g91a1dc0-dirty #64

[ 38.485090] Hardware name: Generic DRA74X (Flattened Device Tree)

[ 38.491216] Workqueue: events_unbound async_run_entry_fn

[ 38.496557] task: eddd9a00 task.stack: ed87c000

[ 38.501109] PC is at pci_restore_msi_state+0x178/0x1fc

[ 38.506272] LR is at pci_bus_read_config_word+0x84/0x90

[ 38.511521] pc : [<c053cef8>] lr : [<c051e734>] psr: 600f0013

[ 38.511521] sp : ed87ddd0 ip : ed87dd98 fp : ed87ddf4

[ 38.523047] r10: c1003d00 r9 : 00000000 r8 : ee072068

[ 38.528295] r7 : ee0723dc r6 : 11f8dc20 r5 : ee072000 r4 : 00000000

[ 38.534851] r3 : 00000000 r2 : 00000000 r1 : 600f0013 r0 : 00000000

[ 38.541408] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user

[ 38.548574] Control: 30c5387d Table: ade4d380 DAC: fffffffd

[ 38.554345] Process kworker/u4:4 (pid: 294, stack limit = 0xed87c210)

[ 38.560811] Stack: (0xed87ddd0 to 0xed87e000)

[ 38.565189] ddc0: ee0723dc 0080002c 11f8dc20 00000000

[ 38.573403] dde0: ee072000 11f8dc20 ed87de14 ed87ddf8 c0526b54 c053cd8c c10254fc ee072000

[ 38.581618] de00: ee072068 00000000 ed87de24 ed87de18 c0526c84 c0526a48 ed87de44 ed87de28

[ 38.589832] de20: c052b8d0 c0526c74 00000001 c052b8a8 00000000 00000000 ed87de7c ed87de48

[ 38.598046] de40: c05f5a10 c052b8b4 00000000 c1003d00 00000000 00000000 c08c903c 00000001

[ 38.606259] de60: ee072068 00000010 ee807d00 00000000 ed87de9c ed87de80 c05f5b30 c05f59e0

[ 38.614474] de80: c108d868 ee072068 ee2fdc80 ee807d00 ed87deb4 ed87dea0 c05f5c1c c05f5ab0

[ 38.622687] dea0: ee2fdc90 c1051020 ed87dedc ed87deb8 c024e04c c05f5c04 00000000 c1003d00

[ 38.630902] dec0: ee2fdc90 ed80d980 ee804a00 ee807d00 ed87df14 ed87dee0 c0244ca0 c024e008

[ 38.639117] dee0: ed87defc ed87def0 c024613c ed80d980 ee804a00 ed80d998 ee804a18 ffffe000

[ 38.647332] df00: 00000088 c1003d00 ed87df54 ed87df18 c0245bb8 c0244ad0 c0b952b4 c10163b4

[ 38.655546] df20: c104ef66 ee804a00 00000000 00000000 eda59440 ed87c000 ed80d980 c0245b50

[ 38.663760] df40: 00000000 00000000 ed87dfac ed87df58 c024ad74 c0245b5c ed87df58 00000000

[ 38.671973] df60: 000aa018 ed80d980 00000000 00000000 ed87df70 ed87df70 00000000 00000000

[ 38.680187] df80: ed87df80 ed87df80 ed87dfac eda59440 c024ac64 00000000 00000000 00000000

[ 38.688399] dfa0: 00000000 ed87dfb0 c0207de8 c024ac70 00000000 00000000 00000000 00000000

[ 38.696613] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000

[ 38.704825] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000

[ 38.713034] Backtrace:

[ 38.715507] [<c053cd80>] (pci_restore_msi_state) from [<c0526b54>] (pci_restore_state.part.26+0x118/0x22c)

[ 38.725201] r6:11f8dc20 r5:ee072000 r4:00000000

[ 38.729843] [<c0526a3c>] (pci_restore_state.part.26) from [<c0526c84>] (pci_restore_state+0x1c/0x20)

[ 38.739015] r7:00000000 r6:ee072068 r5:ee072000 r4:c10254fc

[ 38.744705] [<c0526c68>] (pci_restore_state) from [<c052b8d0>] (pci_pm_resume_noirq+0x28/0x9c)

[ 38.753360] [<c052b8a8>] (pci_pm_resume_noirq) from [<c05f5a10>] (dpm_run_callback+0x3c/0xd0)

[ 38.761921] r7:00000000 r6:00000000 r5:c052b8a8 r4:00000001

[ 38.767608] [<c05f59d4>] (dpm_run_callback) from [<c05f5b30>] (device_resume_noirq+0x8c/0x154)

[ 38.776257] r8:00000000 r7:ee807d00 r6:00000010 r5:ee072068 r4:00000001

[ 38.782989] [<c05f5aa4>] (device_resume_noirq) from [<c05f5c1c>] (async_resume_noirq+0x24/0x4c)

[ 38.791724] r7:ee807d00 r6:ee2fdc80 r5:ee072068 r4:c108d868

[ 38.797410] [<c05f5bf8>] (async_resume_noirq) from [<c024e04c>] (async_run_entry_fn+0x50/0x120)

[ 38.806144] r5:c1051020 r4:ee2fdc90

[ 38.809738] [<c024dffc>] (async_run_entry_fn) from [<c0244ca0>] (process_one_work+0x1dc/0x3f8)

[ 38.818388] r7:ee807d00 r6:ee804a00 r5:ed80d980 r4:ee2fdc90

[ 38.824072] [<c0244ac4>] (process_one_work) from [<c0245bb8>] (worker_thread+0x68/0x5c0)

[ 38.832200] r10:c1003d00 r9:00000088 r8:ffffe000 r7:ee804a18 r6:ed80d998 r5:ee804a00

[ 38.840061] r4:ed80d980

[ 38.842609] [<c0245b50>] (worker_thread) from [<c024ad74>] (kthread+0x110/0x118)

[ 38.850038] r10:00000000 r9:00000000 r8:c0245b50 r7:ed80d980 r6:ed87c000 r5:eda59440

[ 38.857900] r4:00000000

[ 38.860449] [<c024ac64>] (kthread) from [<c0207de8>] (ret_from_fork+0x14/0x2c)

[ 38.867703] r8:00000000 r7:00000000 r6:00000000 r5:c024ac64 r4:eda59440

[ 38.874436] Code: e24b301e e2822002 e5950008 ebff85ed (e5d43028)

[ 38.880699] ---[ end trace c61f477aab694fab ]---

0 Pavel Botev over 7 years ago in reply to Stefan Lange15

TI__Guru**** 170625 points

Stefan Lange15 said:

We are using ti-rt-linux-4.9.77 from git://git.ti.com/ti-linux-kernel/ti-linux-kernel.git

I will try with the 4.9.69 sdk kernel soon.

Please try with PSDK v4.03, kernel 4.9.69, and report the result.

Regards,
Pavel

0 Stefan Lange15 over 7 years ago in reply to Pavel Botev

Intellectual 345 points

Hello Pavel,

I tried the same with the kernel 4.9.69 from git://git.ti.com/processor-sdk/processor-sdk-linux.git

I used the same defconfig and device tree as before.

Now I see an even stranger behaviour, involving the dss (see below).

I cross-checked and ran our kernel 4.9.77-rt and above 4.9.69 on the AM572x-idk (using am572x-idk.dtb) and found it working correctly.

I have the notion that this might have something to do with our hardware. There is one particular specialty about our hardware that I am suspicious about:

We do not have a 32k Quartz attached to the CPU. Instead we use a seperate I2C RTC as RTC.

To get this working, we deleted out the respective device rtc node in our device tree overlaying the dra7.dtsi. (our full device trees are provided in the first post to this thread)

ocp {
/delete-node/ rtc@48838000;

Do you have any suggestions on how continue debugging? Is the missing 32k clock a problem for performing wakeup operations?

Regards,

Stefan

echo -n mem > /sys/power/state
[   29.910407] PM: Syncing filesystems ... done.
[   29.917041] omapdss APPLY error: SYNC_LOST on channel tv, restarting the output with video overlays disabled
[   30.017781] omapdss APPLY error: timeout waiting for digit out to stop
[   30.057826] Freezing user space processes ... (elapsed 0.001 seconds) done.
[   30.066024] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
[   30.086558] PM: suspend of devices complete after 10.422 msecs
[   30.095689] PM: late suspend of devices complete after 3.236 msecs
[   30.104977] PM: noirq suspend of devices complete after 3.052 msecs
[   30.111302] Disabling non-boot CPUs ...
[   30.115831] CPU1: shutdown
< push botton >
[   30.127783] omapdss APPLY error: timeout waiting for digit out to start
[   30.134430] Unhandled fault: asynchronous external abort (0x1211) at 0x00000000
[   30.141770] pgd = c0003000
[   30.144488] [00000000] *pgd=80000080004003, *pmd=00000000
[   30.149930] Internal error: : 1211 [#1] SMP ARM
[   30.154477] Modules linked in:
[   30.157554] CPU: 0 PID: 19 Comm: kworker/0:1 Tainted: G        W       4.9.69-06892-g9ce43c7 #1
[   30.166289] Hardware name: Generic DRA74X (Flattened Device Tree)
[   30.172416] Workqueue: events dispc_error_worker
[   30.177059] task: ee89f6c0 task.stack: eea72000
[   30.181613] PC is at dispc_write_irqenable+0x28/0x2c
[   30.186601] LR is at _omap_dispc_set_irqs+0x3c/0x40
[   30.191501] pc : [<c055543c>]    lr : [<c0564e5c>]    psr: 600f0093
[   30.191501] sp : eea73e00  ip : eea73e10  fp : eea73e0c
[   30.203027] r10: 600f0013  r9 : 00000001  r8 : 600f0013
[   30.208276] r7 : 0000800c  r6 : eea73e48  r5 : c1089b08  r4 : 00000000
[   30.214828] r3 : f18a7000  r2 : 08125640  r1 : c1089b68  r0 : 08125640
[   30.221384] Flags: nZCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   30.228635] Control: 30c5387d  Table: ade07540  DAC: fffffffd
[   30.234405] Process kworker/0:1 (pid: 19, stack limit = 0xeea72210)
[   30.240698] Stack: (0xeea73e00 to 0xeea74000)
[   30.245076] 3e00: eea73e1c eea73e10 c0564e5c c0555420 eea73e44 eea73e20 c0564fe0 c0564e2c
[   30.253291] 3e20: 00000001 0000800c ee14a060 00000000 c1089738 00000001 eea73e6c eea73e48
[   30.261502] 3e40: c056562c c0564f5c 00000000 00020002 eea73e50 eea73e50 c1089738 c1089738
[   30.269715] 3e60: eea73ea4 eea73e70 c0564130 c0565554 c108968c 00000001 eea73ea4 00000001
[   30.277926] 3e80: 00008000 ee14a060 c1089b08 c0bc2730 00000005 c1003d00 eea73eb4 eea73ea8
[   30.286139] 3ea0: c0560074 c0564060 eea73edc eea73eb8 c05653f4 c0560060 c1089b74 eea43880
[   30.294352] 3ec0: eed352c0 eed38900 00000000 00000000 eea73f14 eea73ee0 c024456c c0565294
[   30.302563] 3ee0: eed352d8 ffffe000 00000008 eea43880 eed352c0 eea43898 eed352d8 ffffe000
[   30.310777] 3f00: 00000008 c1003d00 eea73f54 eea73f18 c0245524 c0244388 c0b91ea8 c10163f4
[   30.318989] 3f20: c104e1e5 eed352c0 00000000 00000000 eea4ca80 eea72000 eea43880 c02454bc
[   30.327201] 3f40: 00000000 00000000 eea73fac eea73f58 c024a778 c02454c8 eea79194 00000000
[   30.335414] 3f60: 00000001 eea43880 00000000 00000000 eea73f70 eea73f70 00000000 00000000
[   30.343625] 3f80: eea73f80 eea73f80 eea73fac eea4ca80 c024a668 00000000 00000000 00000000
[   30.351839] 3fa0: 00000000 eea73fb0 c0207ca0 c024a674 00000000 00000000 00000000 00000000
[   30.360053] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[   30.368263] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
[   30.376473] Backtrace:
[   30.378942] [<c0555414>] (dispc_write_irqenable) from [<c0564e5c>] (_omap_dispc_set_irqs+0x3c/0x40)
[   30.388032] [<c0564e20>] (_omap_dispc_set_irqs) from [<c0564fe0>] (omap_dispc_unregister_isr+0x90/0xb0)
[   30.397469] [<c0564f50>] (omap_dispc_unregister_isr) from [<c056562c>] (dispc_mgr_enable_sync+0xe4/0x124)
[   30.407080]  r9:00000001 r8:c1089738 r7:00000000 r6:ee14a060 r5:0000800c r4:00000001
[   30.414861] [<c0565548>] (dispc_mgr_enable_sync) from [<c0564130>] (dss_mgr_enable_compat+0xdc/0x12c)
[   30.424116]  r5:c1089738 r4:c1089738
[   30.427710] [<c0564054>] (dss_mgr_enable_compat) from [<c0560074>] (dss_mgr_enable+0x20/0x24)
[   30.436272]  r10:c1003d00 r9:00000005 r8:c0bc2730 r7:c1089b08 r6:ee14a060 r5:00008000
[   30.444132]  r4:00000001
[   30.446679] [<c0560054>] (dss_mgr_enable) from [<c05653f4>] (dispc_error_worker+0x16c/0x1a0)
[   30.455160] [<c0565288>] (dispc_error_worker) from [<c024456c>] (process_one_work+0x1f0/0x41c)
[   30.463809]  r9:00000000 r8:00000000 r7:eed38900 r6:eed352c0 r5:eea43880 r4:c1089b74
[   30.471592] [<c024437c>] (process_one_work) from [<c0245524>] (worker_thread+0x68/0x5fc)
[   30.479719]  r10:c1003d00 r9:00000008 r8:ffffe000 r7:eed352d8 r6:eea43898 r5:eed352c0
[   30.487579]  r4:eea43880
[   30.490128] [<c02454bc>] (worker_thread) from [<c024a778>] (kthread+0x110/0x118)
[   30.497556]  r10:00000000 r9:00000000 r8:c02454bc r7:eea43880 r6:eea72000 r5:eea4ca80
[   30.505415]  r4:00000000
[   30.507963] [<c024a668>] (kthread) from [<c0207ca0>] (ret_from_fork+0x14/0x34)
[   30.515219]  r8:00000000 r7:00000000 r6:00000000 r5:c024a668 r4:eea4ca80
[   30.521950] Code: e593201c e1c02002 e5832018 e583001c (e89da800)
[   30.528072] ---[ end trace 7bdebaa8f134490c ]---
[   30.532832] Unable to handle kernel paging request at virtual address ffffffec
[   30.540085] pgd = c0003000
[   30.542800] [ffffffec] *pgd=80000080007003, *pmd=affae003, *pte=00000000
[   30.549556] Internal error: Oops: 207 [#2] SMP ARM
[   30.554363] Modules linked in:
[   30.557438] CPU: 0 PID: 19 Comm: kworker/0:1 Tainted: G      D W       4.9.69-06892-g9ce43c7 #1
[   30.566170] Hardware name: Generic DRA74X (Flattened Device Tree)
[   30.572296] task: ee89f6c0 task.stack: eea72000
[   30.576845] PC is at kthread_data+0x10/0x18
[   30.581046] LR is at wq_worker_sleeping+0x10/0xd8
[   30.585769] pc : [<c024b1e8>]    lr : [<c0245b6c>]    psr: 200f0193
[   30.585769] sp : eea73bf8  ip : eea73c08  fp : eea73c04
[   30.597294] r10: c1052880  r9 : c0e676c0  r8 : ee89fab8
[   30.602538] r7 : c1004234  r6 : ee89f6c0  r5 : c0e676c0  r4 : eed356c0
[   30.609093] r3 : 00000000  r2 : 0420806c  r1 : 00000000  r0 : ee89f6c0
[   30.615646] Flags: nzCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   30.622897] Control: 30c5387d  Table: ade07540  DAC: fffffffd
[   30.628670] Process kworker/0:1 (pid: 19, stack limit = 0xeea72210)
[   30.634963] Stack: (0xeea73bf8 to 0xeea74000)
[   30.639336] 3be0:                                                       eea73c1c eea73c08
[   30.647548] 3c00: c0245b6c c024b1e4 eed356c0 c0e676c0 eea73c7c eea73c20 c08c5cc8 c0245b68
[   30.655760] 3c20: ee8c6b40 c1050e14 eea73c44 eea73c38 c02932b0 c0254fa0 2dece000 00000000
[   30.663973] 3c40: eed356c0 c02932a4 00000000 c0e62280 00000023 ee89f6c0 eea73904 ee898000
[   30.672186] 3c60: eea73c90 00000000 ee89fa30 00001211 eea73c8c eea73c80 c0254fa0 c08c59cc
[   30.680397] 3c80: eea73cac eea73c90 c02317ec c0254f1c eea73c90 eea73c90 c022d5f4 bf000000
[   30.688609] 3ca0: eea73cec eea73cb0 c020b7cc c0231128 eea72210 0000000b eea73cfc 600f0193
[   30.696820] 3cc0: c0214af0 c1009234 00001211 00000007 00000000 eea73db0 eea72000 600f0013
[   30.705033] 3ce0: eea73cfc eea73cf0 c020b8ec c020b58c eea73dac eea73d00 c02012f8 c020b8d8
[   30.713244] 3d00: 00000000 c08c9c08 00000007 00000000 00000000 00000000 eea73d94 eea73d28
[   30.721457] 3d20: c0283ed0 c0283904 00000400 00000001 c1054644 00000000 c100403c ffffe000
[   30.729667] 3d40: 00000001 c0b954f4 c10040b0 c1017eec 600f0013 00000000 c1054a70 00000007
[   30.737880] 3d60: 00000297 00000000 00000000 00000000 00000000 c1054130 00000006 00000003
[   30.746092] 3d80: 0000003b c1017eec eea73df4 c055543c 600f0093 ffffffff eea73de4 600f0013
[   30.754305] 3da0: eea73e0c eea73db0 c020bf30 c0201254 08125640 c1089b68 08125640 f18a7000
[   30.762516] 3dc0: 00000000 c1089b08 eea73e48 0000800c 600f0013 00000001 600f0013 eea73e0c
[   30.770729] 3de0: eea73e10 eea73e00 c0564e5c c055543c 600f0093 ffffffff c0bc281c 00000000
[   30.778940] 3e00: eea73e1c eea73e10 c0564e5c c0555420 eea73e44 eea73e20 c0564fe0 c0564e2c
[   30.787153] 3e20: 00000001 0000800c ee14a060 00000000 c1089738 00000001 eea73e6c eea73e48
[   30.795365] 3e40: c056562c c0564f5c 00000000 00020002 eea73e50 eea73e50 c1089738 c1089738
[   30.803577] 3e60: eea73ea4 eea73e70 c0564130 c0565554 c108968c 00000001 eea73ea4 00000001
[   30.811788] 3e80: 00008000 ee14a060 c1089b08 c0bc2730 00000005 c1003d00 eea73eb4 eea73ea8
[   30.820002] 3ea0: c0560074 c0564060 eea73edc eea73eb8 c05653f4 c0560060 c1089b74 eea43880
[   30.828212] 3ec0: eed352c0 eed38900 00000000 00000000 eea73f14 eea73ee0 c024456c c0565294
[   30.836423] 3ee0: eed352d8 ffffe000 00000008 eea43880 eed352c0 eea43898 eed352d8 ffffe000
[   30.844634] 3f00: 00000008 c1003d00 eea73f54 eea73f18 c0245524 c0244388 c0b91ea8 c10163f4
[   30.852847] 3f20: c104e1e5 eed352c0 00000000 00000000 eea4ca80 eea72000 eea43880 c02454bc
[   30.861057] 3f40: 00000000 00000000 eea73fac eea73f58 c024a778 c02454c8 eea79194 00000000
[   30.869268] 3f60: 00000001 eea43880 00000000 00000000 eea73f70 eea73f70 00000001 00010001
[   30.877482] 3f80: eea73f80 eea73f80 eea73fac eea4ca80 c024a668 00000000 00000000 00000000
[   30.885693] 3fa0: 00000000 eea73fb0 c0207ca0 c024a674 00000000 00000000 00000000 00000000
[   30.893905] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[   30.902116] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
[   30.910324] Backtrace:
[   30.912792] [<c024b1d8>] (kthread_data) from [<c0245b6c>] (wq_worker_sleeping+0x10/0xd8)
[   30.920922] [<c0245b5c>] (wq_worker_sleeping) from [<c08c5cc8>] (__schedule+0x308/0x6bc)
[   30.929046]  r5:c0e676c0 r4:eed356c0
[   30.932643] [<c08c59c0>] (__schedule) from [<c0254fa0>] (do_task_dead+0x90/0x94)
[   30.940073]  r10:00001211 r9:ee89fa30 r8:00000000 r7:eea73c90 r6:ee898000 r5:eea73904
[   30.947934]  r4:ee89f6c0
[   30.950483] [<c0254f10>] (do_task_dead) from [<c02317ec>] (do_exit+0x6d0/0x9e4)
[   30.957826] [<c023111c>] (do_exit) from [<c020b7cc>] (die+0x24c/0x34c)
[   30.964381]  r7:bf000000
[   30.966925] [<c020b580>] (die) from [<c020b8ec>] (arm_notify_die+0x20/0x58)
[   30.973918]  r10:600f0013 r9:eea72000 r8:eea73db0 r7:00000000 r6:00000007 r5:00001211
[   30.981779]  r4:c1009234
[   30.984325] [<c020b8cc>] (arm_notify_die) from [<c02012f8>] (do_DataAbort+0xb0/0xb4)
[   30.992103] [<c0201248>] (do_DataAbort) from [<c020bf30>] (__dabt_svc+0x50/0x80)
[   30.999531] Exception stack(0xeea73db0 to 0xeea73df8)
[   31.004601] 3da0:                                     08125640 c1089b68 08125640 f18a7000
[   31.012815] 3dc0: 00000000 c1089b08 eea73e48 0000800c 600f0013 00000001 600f0013 eea73e0c
[   31.021028] 3de0: eea73e10 eea73e00 c0564e5c c055543c 600f0093 ffffffff
[   31.027673]  r8:600f0013 r7:eea73de4 r6:ffffffff r5:600f0093 r4:c055543c
[   31.034407] [<c0555414>] (dispc_write_irqenable) from [<c0564e5c>] (_omap_dispc_set_irqs+0x3c/0x40)
[   31.043495] [<c0564e20>] (_omap_dispc_set_irqs) from [<c0564fe0>] (omap_dispc_unregister_isr+0x90/0xb0)
[   31.052930] [<c0564f50>] (omap_dispc_unregister_isr) from [<c056562c>] (dispc_mgr_enable_sync+0xe4/0x124)
[   31.062540]  r9:00000001 r8:c1089738 r7:00000000 r6:ee14a060 r5:0000800c r4:00000001
[   31.070321] [<c0565548>] (dispc_mgr_enable_sync) from [<c0564130>] (dss_mgr_enable_compat+0xdc/0x12c)
[   31.079578]  r5:c1089738 r4:c1089738
[   31.083171] [<c0564054>] (dss_mgr_enable_compat) from [<c0560074>] (dss_mgr_enable+0x20/0x24)
[   31.091735]  r10:c1003d00 r9:00000005 r8:c0bc2730 r7:c1089b08 r6:ee14a060 r5:00008000
[   31.099595]  r4:00000001
[   31.102139] [<c0560054>] (dss_mgr_enable) from [<c05653f4>] (dispc_error_worker+0x16c/0x1a0)
[   31.110617] [<c0565288>] (dispc_error_worker) from [<c024456c>] (process_one_work+0x1f0/0x41c)
[   31.119268]  r9:00000000 r8:00000000 r7:eed38900 r6:eed352c0 r5:eea43880 r4:c1089b74
[   31.127045] [<c024437c>] (process_one_work) from [<c0245524>] (worker_thread+0x68/0x5fc)
[   31.135171]  r10:c1003d00 r9:00000008 r8:ffffe000 r7:eed352d8 r6:eea43898 r5:eed352c0
[   31.143032]  r4:eea43880
[   31.145578] [<c02454bc>] (worker_thread) from [<c024a778>] (kthread+0x110/0x118)
[   31.153008]  r10:00000000 r9:00000000 r8:c02454bc r7:eea43880 r6:eea72000 r5:eea4ca80
[   31.160868]  r4:00000000
[   31.163413] [<c024a668>] (kthread) from [<c0207ca0>] (ret_from_fork+0x14/0x34)
[   31.170665]  r8:00000000 r7:00000000 r6:00000000 r5:c024a668 r4:eea4ca80
[   31.177398] Code: e1a0c00d e92dd800 e24cb004 e59033c8 (e5130014)
[   31.183519] ---[ end trace 7bdebaa8f134490d ]---
[   31.188154] Fixing recursive fault but reboot is needed!

0 Pavel Botev over 7 years ago in reply to Stefan Lange15

TI__Guru**** 170625 points

Stefan Lange15 said:

We do not have a 32k Quartz attached to the CPU. Instead we use a seperate I2C RTC as RTC.

To get this working, we deleted out the respective device rtc node in our device tree overlaying the dra7.dtsi. (our full device trees are provided in the first post to this thread)

ocp {
/delete-node/ rtc@48838000;

Do you have any suggestions on how continue debugging? Is the missing 32k clock a problem for performing wakeup operations?

I do not think 32k clock is causing you that problem. Can you also delete rtc@48838000 node from AM572x IDK DTS and test suspend/resume on AM572x IDK. Will AM572x IDK still work correct?

For HW check and debug, refer to the below pointers:

0 Yordan Kovachev over 7 years ago in reply to Pavel Botev

TI__Guru**** 161600 points

Closing this.

Processors

Processors forum

Linux/AM5728: Wakeup error on PCI restore