HI experts,
I got cpu stalled(one core) when open serial.
out environment is tda4vm + ublox gps , sdk 8.1(ti-processor-sdk-linux-j7-evm-08_01_00_07) kernel 5.10.65
when issue occurred , interrupters of serial increase quickly as following.
log is attached.
Thanks!
Xiangqun
2022-03-14 20:52:43.963] t2514 u-blox ubx_moduleIf.cpp:284 CGpsIf::start: (188284928):
[2022-03-14 20:52:43.979] main 125 gps start 0
[2022-03-14 20:53:04.004] [22218.578103] rcu: INFO: rcu_preempt self-detected stall on CPU
[2022-03-14 20:53:04.004] [22218.583842] rcu: 0-...!: (5250 ticks this GP) idle=b6e/1/0x4000000000000004 softirq=16152/16152 fqs=0
[2022-03-14 20:53:04.004] [22218.593212] (t=5250 jiffies g=19365 q=9)
[2022-03-14 20:53:04.004] [22218.597208] rcu: rcu_preempt kthread starved for 5250 jiffies! g19365 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0
[2022-03-14 20:53:04.004] [22218.607615] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[2022-03-14 20:53:04.004] [22218.616722] rcu: RCU grace-period kthread stack dump:
[2022-03-14 20:53:04.047] [22218.621756] task:rcu_preempt state:I stack: 0 pid: 11 ppid: 2 flags:0x00000028
[2022-03-14 20:53:04.047] [22218.630088] Call trace:
[2022-03-14 20:53:04.047] [22218.632534] __switch_to+0xf4/0x148
[2022-03-14 20:53:04.047] [22218.636013] __schedule+0x2e4/0x720
[2022-03-14 20:53:04.047] [22218.639488] schedule+0x70/0x108
[2022-03-14 20:53:04.047] [22218.642702] schedule_timeout+0x17c/0x280
[2022-03-14 20:53:04.047] [22218.646700] rcu_gp_kthread+0x41c/0x9c8
[2022-03-14 20:53:04.047] [22218.650522] kthread+0x154/0x158
[2022-03-14 20:53:04.047] [22218.653736] ret_from_fork+0x10/0x34
[2022-03-14 20:53:04.048] [22218.657302] Task dump for CPU 0:
[2022-03-14 20:53:04.048] [22218.660516] task:simple-gpsd state:R running task stack: 0 pid: 2515 ppid: 1770 flags:0x00000002
[2022-03-14 20:53:04.090] [22218.670406] Call trace:
[2022-03-14 20:53:04.090] [22218.672842] dump_backtrace+0x0/0x1b0
[2022-03-14 20:53:04.090] [22218.676491] show_stack+0x18/0x68
[2022-03-14 20:53:04.090] [22218.679792] sched_show_task+0x154/0x180
[2022-03-14 20:53:04.090] [22218.683701] dump_cpu_task+0x44/0x54
[2022-03-14 20:53:04.090] [22218.687262] rcu_dump_cpu_stacks+0xb0/0xf0
[2022-03-14 20:53:04.090] [22218.691343] rcu_sched_clock_irq+0x934/0xc50
[2022-03-14 20:53:04.090] [22218.695598] update_process_times+0x60/0xa0
[2022-03-14 20:53:04.091] [22218.699767] tick_sched_handle.isra.22+0x40/0x50
[2022-03-14 20:53:04.091] [22218.704369] tick_sched_timer+0x4c/0xa8
[2022-03-14 20:53:04.133] [22218.708190] __hrtimer_run_queues+0x11c/0x1b0
[2022-03-14 20:53:04.133] [22218.712531] hrtimer_interrupt+0xe8/0x240
[2022-03-14 20:53:04.133] [22218.716528] arch_timer_handler_phys+0x30/0x40
[2022-03-14 20:53:04.133] [22218.720956] handle_percpu_devid_irq+0x84/0x148
[2022-03-14 20:53:04.133] [22218.725472] generic_handle_irq+0x30/0x48
[2022-03-14 20:53:04.133] [22218.729467] __handle_domain_irq+0x64/0xc0
[2022-03-14 20:53:04.133] [22218.733550] gic_handle_irq+0xc0/0x138
[2022-03-14 20:53:04.133] [22218.737284] el1_irq+0xc8/0x180
[2022-03-14 20:53:04.134] [22218.740412] efi_header_end+0xa8/0x26c
[2022-03-14 20:53:04.134] [22218.744147] irq_exit+0xc0/0xd0
[2022-03-14 20:53:04.134] [22218.747275] __handle_domain_irq+0x68/0xc0
[2022-03-14 20:53:04.176] [22218.751355] gic_handle_irq+0xc0/0x138
[2022-03-14 20:53:04.176] [22218.755089] el1_irq+0xc8/0x180
[2022-03-14 20:53:04.176] [22218.758218] preempt_schedule_irq+0x44/0x90
[2022-03-14 20:53:04.176] [22218.762385] arm64_preempt_schedule_irq+0x14/0x20
[2022-03-14 20:53:04.176] [22218.767072] el1_irq+0xe0/0x180
[2022-03-14 20:53:04.176] [22218.770202] omap_8250_startup+0x124/0x218
[2022-03-14 20:53:04.176] [22218.774284] serial8250_startup+0x18/0x38
[2022-03-14 20:53:04.177] [22218.778278] uart_startup.part.25+0x194/0x2c8
[2022-03-14 20:53:04.177] [22218.782619] uart_port_activate+0x60/0x98
[2022-03-14 20:53:04.177] [22218.786615] tty_port_open+0x90/0x110
[2022-03-14 20:53:04.177] [22218.790265] uart_open+0x1c/0x30
[2022-03-14 20:53:04.219] [22218.793480] tty_open+0x11c/0x4d0
[2022-03-14 20:53:04.219] [22218.796783] chrdev_open+0xa8/0x1a8
[2022-03-14 20:53:04.219] [22218.800259] do_dentry_open+0x12c/0x390
[2022-03-14 20:53:04.219] [22218.804080] vfs_open+0x2c/0x38
[2022-03-14 20:53:04.219] [22218.807208] path_openat+0x95c/0xcc0
[2022-03-14 20:53:04.219] [22218.810769] do_filp_open+0x7c/0xf8
[2022-03-14 20:53:04.219] [22218.814243] do_sys_openat2+0x200/0x2b0
[2022-03-14 20:53:04.220] [22218.818064] do_sys_open+0x58/0xa0
[2022-03-15 10:34:09.532] sh-5.0# cat /proc/interrupts | grep serial
[2022-03-15 10:34:19.239] 29: 163455 0 GICv3 224 Level 2800000.serial
[2022-03-15 10:34:19.239] 30: 92027351 0 GICv3 225 Level 2810000.serial
[2022-03-15 10:34:19.239] sh-5.0# cat /proc/interrupts | grep serial
[2022-03-15 10:34:21.175] 29: 163469 0 GICv3 224 Level 2800000.serial
[2022-03-15 10:34:21.175] 30: 92876746 0 GICv3 225 Level 2810000.serial
[2022-03-15 10:34:21.175] sh-5.0# cat /proc/interrupts | grep serial
[2022-03-15 10:34:23.383] 29: 163483 0 GICv3 224 Level 2800000.serial
[2022-03-15 10:34:23.383] 30: 93846128 0 GICv3 225 Level 2810000.serial
[2022-03-15 10:34:23.383] sh-5.0# [71493.962103] rcu: INFO: rcu_preempt self-detected stall on CPU
[2022-03-15 10:34:23.554] [71493.967840] rcu: 0-....: (12284981 ticks this GP) idle=b6e/1/0x4000000000000004 softirq=16152/16152 fqs=4148776
[2022-03-15 10:34:23.554] [71493.978074] (t=12324096 jiffies g=19365 q=45228)
[2022-03-15 10:34:23.554] [71493.982762] Task dump for CPU 0:
[2022-03-14 20:52:42.543] t2511 u-blox gps_thread.cpp:185 engineStop: (End) Client count 0 [2022-03-14 20:52:42.544] t2511 u-blox gps_thread.cpp:340 checkShutdownState: Command the receiver to turn off! [2022-03-14 20:52:42.544] t2511 u-blox ubxgpsstate.cpp:404 CUbxGpsState::changeState: A receiver state change has been detected. Turn receiver Off! [2022-03-14 20:52:42.545] t2511 u-blox gps_thread.cpp:395 handleCmdInput: (3096244368): Cmd received (5) [2022-03-14 20:52:42.545] t2511 u-blox gps_thread.cpp:399 handleCmdInput: Exit thread cmd received [2022-03-14 20:52:42.586] t2511 u-blox gps_thread.cpp:851 ubx_thread: Left main loop [2022-03-14 20:52:42.586] t2511 u-blox ubx_serial.cpp:403 CSerialPort::closeSerial: CSerialPort::closeSerial: fd = 5 [2022-03-14 20:52:42.587] t2511 u-blox gps_thread.cpp:853 ubx_thread: Thread exiting [2022-03-14 20:52:42.587] t2510 u-blox ubx_moduleIf.cpp:521 endControlThread: Thread exited ok=1 [2022-03-14 20:52:42.588] ++++183+++++ [2022-03-14 20:52:42.588] total used free shared buff/cache available [2022-03-14 20:52:42.588] Mem: 2271936 154048 1995776 25728 122112 1867648 [2022-03-14 20:52:42.589] Swap: 0 0 0 [2022-03-14 20:52:43.002] t2514 u-blox ubx_cfg.cpp:59 CCfg::load: fileName="/etc/u-blox.conf" [2022-03-14 20:52:43.002] t2514 u-blox ubx_cfg.cpp:164 CCfg::get: item="SERIAL_DEVICE" def="/dev/ttyACM0" => "/dev/ttyS3" [2022-03-14 20:52:43.003] t2514 u-blox ubx_cfg.cpp:149 CCfg::get: item="BAUDRATE" def=9600 => 921600 [2022-03-14 20:52:43.003] t2514 u-blox ubx_cfg.cpp:149 CCfg::get: item="BAUDRATE_DEF" def=9600 => 38400 [2022-03-14 20:52:43.003] t2514 u-blox ubx_cfg.cpp:149 CCfg::get: item="I2C_TX_READY_ENABLED" def=0 => 0 [2022-03-14 20:52:43.004] t2514 u-blox ubx_cfg.cpp:149 CCfg::get: item="I2C_TX_READY_RECV_PIO" def=0 => 0 [2022-03-14 20:52:43.005] t2514 u-blox ubx_cfg.cpp:149 CCfg::get: item="I2C_TX_READY_HOST_GPIO" def=0 => 0 [2022-03-14 20:52:43.045] t2514 u-blox ubx_cfg.cpp:149 CCfg::get: item="RESET_HOST_GPIO" def=0 => 353 [2022-03-14 20:52:43.046] t2514 u-blox ubx_cfg.cpp:149 CCfg::get: item="STOP_TIMEOUT" def=10 => 10 [2022-03-14 20:52:43.046] t2514 u-blox ubx_moduleIf.cpp:218 CGpsIf::init: Driver version 3.30 [2022-03-14 20:52:43.046] t2514 u-blox ubx_moduleIf.cpp:222 CGpsIf::init: libParser version 1.04 [2022-03-14 20:52:43.047] t2514 u-blox ubx_moduleIf.cpp:225 CGpsIf::init: [2022-03-14 20:52:43.047] t2514 u-blox ubx_moduleIf.cpp:232 CGpsIf::init: (188284928): Initializing - pid 2514 [2022-03-14 20:52:43.088] t2514 u-blox ubx_moduleIf.cpp:242 CGpsIf::init: set_capabilities=17(GPS_CAPABILITY_SCHEDULING+GPS_CAPABILITY_ON_DEMAND_TIME) [2022-03-14 20:52:43.088] fgps_set_capabilities 45 capabilities 11 [2022-03-14 20:52:43.089] fgps_create_thread 33 create thread gps thread [2022-03-14 20:52:43.089] t2514 u-blox ubx_moduleIf.cpp:252 CGpsIf::init: Wait for ubx-thread to get ready [2022-03-14 20:52:43.089] t2515 u-blox gps_thread.cpp:558 ubx_thread: (2373382288): Gps background thread started [2022-03-14 20:52:43.090] t2515 u-blox gps_thread.cpp:196 handle_init: (2373382288): Init state. Signal main-thread to continue [2022-03-14 20:52:43.131] t2515 u-blox gps_thread.cpp:134 signal_cmd_complete: Signal to waiting threads the result of the last command: 1 [2022-03-14 20:52:43.131] t2515 u-blox gps_thread.cpp:139 signal_cmd_complete: Signaling of result completed. Wait for main-thread [2022-03-14 20:52:43.132] t2514 u-blox ubx_moduleIf.cpp:261 CGpsIf::init: Initialized complete: result 1. ubx-thread ready [2022-03-14 20:52:43.133] t2514 u-blox ubx_moduleIf.cpp:271 CGpsIf::init: Signal to the waiting ubx-thread that main-thread is ready [2022-03-14 20:52:43.133] t2515 u-blox gps_thread.cpp:145 signal_cmd_complete: Wait for ubx-main to signalise that everything is ready for normal operation [2022-03-14 20:52:43.174] t2514 u-blox ubx_moduleIf.cpp:460 CGpsIf::gpsStatus: GPS status change: 0 (GPS_STATUS_NONE) => 4 (GPS_STATUS_ENGINE_OFF) [2022-03-14 20:52:43.175] fgps_status_callback 56 Gps Status 0xffffe9aa3a80 [2022-03-14 20:52:43.175] t2514 u-blox ubx_moduleIf.cpp:277 CGpsIf::init: Signaling of readyness completed. Return normally and let ubx-thread continue [2022-03-14 20:52:43.176] main 121 init ret 0 [2022-03-14 20:52:43.176] t2515 u-blox gps_thread.cpp:151 signal_cmd_complete: Initialisation complete. main-thread seems to be ready. Continue. [2022-03-14 20:52:43.227] t2515 u-blox gps_thread.cpp:199 handle_init: Signaled main-thread to continue [2022-03-14 20:52:43.228] t2515 u-blox gps_thread.cpp:607 ubx_thread: Starting ubx-thread main loop [2022-03-14 20:52:43.228] t2515 u-blox gps_thread.cpp:434 connectReceiver: Open/Reopen the serial port [2022-03-14 20:52:43.229] t2515 u-blox ubx_serial.cpp:114 CSerialPort::openSerial: CSerialPort::openSerial: pTty = /dev/ttyS3, ttybaud = 921600, blocksize = 1 [2022-03-14 20:52:43.229] t2515 u-blox ubx_serial.cpp:403 CSerialPort::closeSerial: CSerialPort::closeSerial: fd = -1 [2022-03-14 20:52:43.963] t2514 u-blox ubx_moduleIf.cpp:284 CGpsIf::start: (188284928): [2022-03-14 20:52:43.979] main 125 gps start 0 [2022-03-14 20:53:04.004] [22218.578103] rcu: INFO: rcu_preempt self-detected stall on CPU [2022-03-14 20:53:04.004] [22218.583842] rcu: 0-...!: (5250 ticks this GP) idle=b6e/1/0x4000000000000004 softirq=16152/16152 fqs=0 [2022-03-14 20:53:04.004] [22218.593212] (t=5250 jiffies g=19365 q=9) [2022-03-14 20:53:04.004] [22218.597208] rcu: rcu_preempt kthread starved for 5250 jiffies! g19365 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 [2022-03-14 20:53:04.004] [22218.607615] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [2022-03-14 20:53:04.004] [22218.616722] rcu: RCU grace-period kthread stack dump: [2022-03-14 20:53:04.047] [22218.621756] task:rcu_preempt state:I stack: 0 pid: 11 ppid: 2 flags:0x00000028 [2022-03-14 20:53:04.047] [22218.630088] Call trace: [2022-03-14 20:53:04.047] [22218.632534] __switch_to+0xf4/0x148 [2022-03-14 20:53:04.047] [22218.636013] __schedule+0x2e4/0x720 [2022-03-14 20:53:04.047] [22218.639488] schedule+0x70/0x108 [2022-03-14 20:53:04.047] [22218.642702] schedule_timeout+0x17c/0x280 [2022-03-14 20:53:04.047] [22218.646700] rcu_gp_kthread+0x41c/0x9c8 [2022-03-14 20:53:04.047] [22218.650522] kthread+0x154/0x158 [2022-03-14 20:53:04.047] [22218.653736] ret_from_fork+0x10/0x34 [2022-03-14 20:53:04.048] [22218.657302] Task dump for CPU 0: [2022-03-14 20:53:04.048] [22218.660516] task:simple-gpsd state:R running task stack: 0 pid: 2515 ppid: 1770 flags:0x00000002 [2022-03-14 20:53:04.090] [22218.670406] Call trace: [2022-03-14 20:53:04.090] [22218.672842] dump_backtrace+0x0/0x1b0 [2022-03-14 20:53:04.090] [22218.676491] show_stack+0x18/0x68 [2022-03-14 20:53:04.090] [22218.679792] sched_show_task+0x154/0x180 [2022-03-14 20:53:04.090] [22218.683701] dump_cpu_task+0x44/0x54 [2022-03-14 20:53:04.090] [22218.687262] rcu_dump_cpu_stacks+0xb0/0xf0 [2022-03-14 20:53:04.090] [22218.691343] rcu_sched_clock_irq+0x934/0xc50 [2022-03-14 20:53:04.090] [22218.695598] update_process_times+0x60/0xa0 [2022-03-14 20:53:04.091] [22218.699767] tick_sched_handle.isra.22+0x40/0x50 [2022-03-14 20:53:04.091] [22218.704369] tick_sched_timer+0x4c/0xa8 [2022-03-14 20:53:04.133] [22218.708190] __hrtimer_run_queues+0x11c/0x1b0 [2022-03-14 20:53:04.133] [22218.712531] hrtimer_interrupt+0xe8/0x240 [2022-03-14 20:53:04.133] [22218.716528] arch_timer_handler_phys+0x30/0x40 [2022-03-14 20:53:04.133] [22218.720956] handle_percpu_devid_irq+0x84/0x148 [2022-03-14 20:53:04.133] [22218.725472] generic_handle_irq+0x30/0x48 [2022-03-14 20:53:04.133] [22218.729467] __handle_domain_irq+0x64/0xc0 [2022-03-14 20:53:04.133] [22218.733550] gic_handle_irq+0xc0/0x138 [2022-03-14 20:53:04.133] [22218.737284] el1_irq+0xc8/0x180 [2022-03-14 20:53:04.134] [22218.740412] efi_header_end+0xa8/0x26c [2022-03-14 20:53:04.134] [22218.744147] irq_exit+0xc0/0xd0 [2022-03-14 20:53:04.134] [22218.747275] __handle_domain_irq+0x68/0xc0 [2022-03-14 20:53:04.176] [22218.751355] gic_handle_irq+0xc0/0x138 [2022-03-14 20:53:04.176] [22218.755089] el1_irq+0xc8/0x180 [2022-03-14 20:53:04.176] [22218.758218] preempt_schedule_irq+0x44/0x90 [2022-03-14 20:53:04.176] [22218.762385] arm64_preempt_schedule_irq+0x14/0x20 [2022-03-14 20:53:04.176] [22218.767072] el1_irq+0xe0/0x180 [2022-03-14 20:53:04.176] [22218.770202] omap_8250_startup+0x124/0x218 [2022-03-14 20:53:04.176] [22218.774284] serial8250_startup+0x18/0x38 [2022-03-14 20:53:04.177] [22218.778278] uart_startup.part.25+0x194/0x2c8 [2022-03-14 20:53:04.177] [22218.782619] uart_port_activate+0x60/0x98 [2022-03-14 20:53:04.177] [22218.786615] tty_port_open+0x90/0x110 [2022-03-14 20:53:04.177] [22218.790265] uart_open+0x1c/0x30 [2022-03-14 20:53:04.219] [22218.793480] tty_open+0x11c/0x4d0 [2022-03-14 20:53:04.219] [22218.796783] chrdev_open+0xa8/0x1a8 [2022-03-14 20:53:04.219] [22218.800259] do_dentry_open+0x12c/0x390 [2022-03-14 20:53:04.219] [22218.804080] vfs_open+0x2c/0x38 [2022-03-14 20:53:04.219] [22218.807208] path_openat+0x95c/0xcc0 [2022-03-14 20:53:04.219] [22218.810769] do_filp_open+0x7c/0xf8 [2022-03-14 20:53:04.219] [22218.814243] do_sys_openat2+0x200/0x2b0 [2022-03-14 20:53:04.220] [22218.818064] do_sys_open+0x58/0xa0 [2022-03-14 20:53:04.220] [22218.821451] __arm64_sys_openat+0x24/0x30 [2022-03-14 20:53:04.220] [22218.825448] el0_svc_common.constprop.4+0x8c/0x180 [2022-03-14 20:53:04.220] [22218.830222] do_el0_svc+0x24/0x90 [2022-03-14 20:53:04.220] [22218.833523] el0_svc+0x14/0x20 [2022-03-14 20:53:04.251] [22218.836565] el0_sync_handler+0x90/0xb8 [2022-03-14 20:53:04.251] [22218.840386] el0_sync+0x160/0x180 [2022-03-14 20:53:04.251] t2514 u-blox ubx_moduleIf.cpp:297 CGpsIf::stop: (188284928): [2022-03-14 20:53:04.251] main 129 gps stop 0 [2022-03-14 20:53:04.251] t2514 u-blox ubx_moduleIf.cpp:517 endControlThread: Send thread exit command [2022-03-14 20:54:07.021] [22281.590104] rcu: INFO: rcu_preempt self-detected stall on CPU [2022-03-14 20:54:07.021] [22281.595840] rcu: 0-....: (20938 ticks this GP) idle=b6e/1/0x4000000000000004 softirq=16152/16152 fqs=5310 [2022-03-14 20:54:07.022] [22281.605555] (t=21003 jiffies g=19365 q=80) [2022-03-14 20:54:07.022] [22281.609723] Task dump for CPU 0: [2022-03-14 20:54:07.023] [22281.612937] task:simple-gpsd state:R running task stack: 0 pid: 2515 ppid: 1770 flags:0x00000002 [2022-03-14 20:54:07.023] [22281.622829] Call trace: [2022-03-14 20:54:07.024] [22281.625273] dump_backtrace+0x0/0x1b0 [2022-03-14 20:54:07.024] [22281.628920] show_stack+0x18/0x68 [2022-03-14 20:54:07.024] [22281.632223] sched_show_task+0x154/0x180 [2022-03-14 20:54:07.064] [22281.636134] dump_cpu_task+0x44/0x54 [2022-03-14 20:54:07.064] [22281.639694] rcu_dump_cpu_stacks+0xb0/0xf0 [2022-03-14 20:54:07.064] [22281.643776] rcu_sched_clock_irq+0x934/0xc50 [2022-03-14 20:54:07.065] [22281.648031] update_process_times+0x60/0xa0 [2022-03-14 20:54:07.065] [22281.652202] tick_sched_handle.isra.22+0x40/0x50 [2022-03-14 20:54:07.066] [22281.656802] tick_sched_timer+0x4c/0xa8 [2022-03-14 20:54:07.066] [22281.660623] __hrtimer_run_queues+0x11c/0x1b0 [2022-03-14 20:54:07.066] [22281.664965] hrtimer_interrupt+0xe8/0x240 [2022-03-14 20:54:07.067] [22281.668961] arch_timer_handler_phys+0x30/0x40 [2022-03-14 20:54:07.067] [22281.673390] handle_percpu_devid_irq+0x84/0x148 [2022-03-14 20:54:07.107] [22281.677907] generic_handle_irq+0x30/0x48 [2022-03-14 20:54:07.107] [22281.681901] __handle_domain_irq+0x64/0xc0 [2022-03-14 20:54:07.108] [22281.685984] gic_handle_irq+0xc0/0x138 [2022-03-14 20:54:07.108] [22281.689718] el1_irq+0xc8/0x180 [2022-03-14 20:54:07.108] [22281.692846] efi_header_end+0xa8/0x26c [2022-03-14 20:54:07.109] [22281.696580] irq_exit+0xc0/0xd0 [2022-03-14 20:54:07.109] [22281.699708] __handle_domain_irq+0x68/0xc0 [2022-03-14 20:54:07.109] [22281.703789] gic_handle_irq+0xc0/0x138 [2022-03-14 20:54:07.110] [22281.707522] el1_irq+0xc8/0x180 [2022-03-14 20:54:07.110] [22281.710651] preempt_schedule_irq+0x44/0x90 [2022-03-14 20:54:07.110] [22281.714820] arm64_preempt_schedule_irq+0x14/0x20 [2022-03-14 20:54:07.150] [22281.719507] el1_irq+0xe0/0x180 [2022-03-14 20:54:07.150] [22281.722635] omap_8250_startup+0x124/0x218 [2022-03-14 20:54:07.151] [22281.726717] serial8250_startup+0x18/0x38 [2022-03-14 20:54:07.151] [22281.730712] uart_startup.part.25+0x194/0x2c8 [2022-03-14 20:54:07.151] [22281.735052] uart_port_activate+0x60/0x98 [2022-03-14 20:54:07.152] [22281.739047] tty_port_open+0x90/0x110 [2022-03-14 20:54:07.152] [22281.742695] uart_open+0x1c/0x30 [2022-03-14 20:54:07.152] [22281.745911] tty_open+0x11c/0x4d0 [2022-03-14 20:54:07.152] [22281.749215] chrdev_open+0xa8/0x1a8 [2022-03-14 20:54:07.153] [22281.752690] do_dentry_open+0x12c/0x390 [2022-03-14 20:54:07.153] [22281.756511] vfs_open+0x2c/0x38 [2022-03-14 20:54:07.153] [22281.759639] path_openat+0x95c/0xcc0 [2022-03-14 20:54:07.198] [22281.763200] do_filp_open+0x7c/0xf8 [2022-03-14 20:54:07.198] [22281.766674] do_sys_openat2+0x200/0x2b0 [2022-03-14 20:54:07.198] [22281.770495] do_sys_open+0x58/0xa0 [2022-03-14 20:54:07.199] [22281.773883] __arm64_sys_openat+0x24/0x30 [2022-03-14 20:54:07.199] [22281.777879] el0_svc_common.constprop.4+0x8c/0x180 [2022-03-14 20:54:07.199] [22281.782653] do_el0_svc+0x24/0x90 [2022-03-14 20:54:07.199] [22281.785954] el0_svc+0x14/0x20 [2022-03-14 20:54:07.200] [22281.788995] el0_sync_handler+0x90/0xb8 [2022-03-14 20:54:07.200] [22281.792816] el0_sync+0x160/0x180 [2022-03-14 20:55:10.038] [22344.602102] rcu: INFO: rcu_preempt self-detected stall on CPU [2022-03-14 20:55:10.038] [22344.607837] rcu: 0-....: (36641 ticks this GP) idle=b6e/1/0x4000000000000004 softirq=16152/16152 fqs=10608 [2022-03-14 20:55:10.039] [22344.617639] (t=36756 jiffies g=19365 q=134) [2022-03-14 20:55:10.039] [22344.621892] Task dump for CPU 0: [2022-03-14 20:55:10.040] [22344.625106] task:simple-gpsd state:R running task stack: 0 pid: 2515 ppid: 1770 flags:0x00000002 [2022-03-14 20:55:10.040] [22344.634998] Call trace: [2022-03-14 20:55:10.041] [22344.637441] dump_backtrace+0x0/0x1b0 [2022-03-14 20:55:10.041] [22344.641089] show_stack+0x18/0x68 [2022-03-14 20:55:10.041] [22344.644391] sched_show_task+0x154/0x180 [2022-03-14 20:55:10.081] [22344.648299] dump_cpu_task+0x44/0x54 [2022-03-14 20:55:10.081] [22344.651860] rcu_dump_cpu_stacks+0xb0/0xf0 [2022-03-14 20:55:10.082] [22344.655943] rcu_sched_clock_irq+0x934/0xc50 [2022-03-14 20:55:10.082] [22344.660198] update_process_times+0x60/0xa0 [2022-03-14 20:55:10.083] [22344.664366] tick_sched_handle.isra.22+0x40/0x50 [2022-03-14 20:55:10.083] [22344.668968] tick_sched_timer+0x4c/0xa8 [2022-03-14 20:55:10.083] [22344.672789] __hrtimer_run_queues+0x11c/0x1b0 [2022-03-14 20:55:10.084] [22344.677130] hrtimer_interrupt+0xe8/0x240 [2022-03-14 20:55:10.084] [22344.681126] arch_timer_handler_phys+0x30/0x40 [2022-03-14 20:55:10.085] [22344.685555] handle_percpu_devid_irq+0x84/0x148 [2022-03-14 20:55:10.124] [22344.690072] generic_handle_irq+0x30/0x48 [2022-03-14 20:55:10.124] [22344.694067] __handle_domain_irq+0x64/0xc0 [2022-03-14 20:55:10.125] [22344.698150] gic_handle_irq+0xc0/0x138 [2022-03-14 20:55:10.125] [22344.701884] el1_irq+0xc8/0x180 [2022-03-14 20:55:10.125] [22344.705012] efi_header_end+0xa8/0x26c [2022-03-14 20:55:10.125] [22344.708747] irq_exit+0xc0/0xd0 [2022-03-14 20:55:10.126] [22344.711875] __handle_domain_irq+0x68/0xc0 [2022-03-14 20:55:10.126] [22344.715956] gic_handle_irq+0xc0/0x138 [2022-03-14 20:55:10.126] [22344.719690] el1_irq+0xc8/0x180 [2022-03-14 20:55:10.127] [22344.722819] preempt_schedule_irq+0x44/0x90 [2022-03-14 20:55:10.127] [22344.726986] arm64_preempt_schedule_irq+0x14/0x20 [2022-03-14 20:55:10.167] [22344.731673] el1_irq+0xe0/0x180 [2022-03-14 20:55:10.167] [22344.734802] omap_8250_startup+0x124/0x218 [2022-03-14 20:55:10.168] [22344.738884] serial8250_startup+0x18/0x38 [2022-03-14 20:55:10.168] [22344.742878] uart_startup.part.25+0x194/0x2c8
