TLV320AIC3254: Kernel Errors and Boot Failures Due to TLV32x4 Driver

Alec Schuster

Part Number: TLV320AIC3254
Other Parts Discussed in Thread: TLV320AIC3204,

Hello,

We are using your codec in one of our products and have successfully integrated it into Ubuntu 22.04.3 by using the provided Linux drivers. However, there is a bug in the driver that is resulting in errors such as "Scheduling While Atomic" and attempts to write to memory that does not exist. We do not have the option of patching the driver ourselves with a custom kernel due to customer requirements. Are there any other steps we can take to fix this issue or do we have to wait for a patch? It is causing boot issues.

When we disable the driver (which disables our audio capabilities), the boot issues go away.

Below are some logs I have gathered regarding the issue:

[ 10.027250] BUG: scheduling while atomic: systemd-udevd/432/0x00000002

[ 10.045960] BUG: scheduling while atomic: systemd-udevd/432/0x00000000

[ 10.074657] BUG: scheduling while atomic: systemd-udevd/432/0x00000002

[ 10.083558] BUG: scheduling while atomic: systemd-udevd/432/0x00000000

[ 10.101876] Unable to handle kernel read from unreadable memory at virtual address 0000000000000078

[ 10.111066] Mem abort info:

[ 10.113894] ESR = 0x0000000096000004

[ 10.117693] EC = 0x25: DABT (current EL), IL = 32 bits

[ 10.123079] SET = 0, FnV = 0

[ 10.126172] EA = 0, S1PTW = 0

[ 10.129352] FSC = 0x04: level 0 translation fault

[ 10.134297] Data abort info:

[ 10.137212] ISV = 0, ISS = 0x00000004

[ 10.141098] CM = 0, WnR = 0

[ 10.144104] user pgtable: 4k pages, 48-bit VAs, pgdp=0000000102ef0000

[ 10.150639] [0000000000000078] pgd=0000000000000000, p4d=0000000000000000

[ 10.157532] Internal error: Oops: 96000004 [#1] PREEMPT SMP

[ 10.163185] Modules linked in: snd_soc_tlv320aic32x4_spi brcmfmac snd_soc_simple_card(+) brcmutil snd_soc_t lv320aic32x4_i2c cfg80211 snd_soc_bcm2835_i2s snd_soc_tlv320aic32x4 snd_soc_simple_card_utils snd_bcm2835(CE) snd_soc_core snd_compress snd_seq_midi ac97_bus raspberrypi_hwmon snd_seq_midi_event snd_pcm_dmaengine snd_raw midi snd_pcm snd_seq bcm2835_codec(CE) bcm2835_isp(CE) v4l2_mem2mem bcm2835_v4l2(CE) bcm2835_mmal_vchiq(CE) sn d_seq_device videobuf2_vmalloc snd_timer videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 videobuf2_common snd videodev bcm2835_gpiomem mc vc_sm_cma(CE) rpivid_mem uio_pdrv_genirq nvmem_rmem uio sch_fq_codel rtc_ds13 07 drm pstore_blk ramoops reed_solomon pstore_zone ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compr ess raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor xor_neon raid6_pq libcrc32c raid1 raid0 multipath linear uas usb_storage rtc_pcf85063 dwc2 roles i2c_mux_pinctrl udc_core crct10dif_ce i2c _mux

[ 10.163386] i2c_bcm2835 phy_generic aes_arm64

[ 10.255713] CPU: 3 PID: 432 Comm: systemd-udevd Tainted: G WC E 5.15.0-1035-raspi #38-Ubuntu

[ 10.265335] Hardware name: Raspberry Pi Compute Module 4 Rev 1.1 (DT)

[ 10.271866] pstate: 204000c5 (nzCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)

[ 10.278931] pc : pick_next_task_fair+0x1f4/0x490

[ 10.283618] lr : pick_next_task_fair+0x1f0/0x490

[ 10.288298] sp : ffff80000869b130

[ 10.291656] x29: ffff80000869b130 x28: ffff80000869bd60 x27: 0000000000000030

[ 10.298901] x26: ffff4767bef7f800 x25: ffffba53cdc2d5f8 x24: ffff4766c459a000

[ 10.306144] x23: 0000000000000000 x22: ffff4767bef7f700 x21: 0000000000000000

[ 10.313388] x20: 0000000000000000 x19: 0000000000000000 x18: 0000000000000000

[ 10.320632] x17: 2020202020452020 x16: 4357202020202020 x15: 202047203a646574

[ 10.327875] x14: 0000000000000000 x13: 38613178302f3461 x12: 3178302b636e7973

[ 10.335117] x11: 0000000000000000 x10: 000000000000000a x9 : ffffba53cd4c8c98

[ 10.342360] x8 : 000000000000000a x7 : 0000000000000001 x6 : 0000000000000126

[ 10.349602] x5 : ffff4766c8b6d800 x4 : 0000000001674174 x3 : ffffba53ce565c80

[ 10.356846] x2 : 0000000000000000 x1 : ffffba53ce565c80 x0 : 000000003ddd1465

[ 10.364090] Call trace:

[ 10.366567] pick_next_task_fair+0x1f4/0x490

[ 10.370898] __schedule+0x1d8/0x8b0

[ 10.374436] schedule+0x70/0x170

[ 10.377709] schedule_timeout+0xa0/0x1c4

[ 10.381687] wait_for_completion_timeout+0x88/0x110

[ 10.386634] bcm2835_i2c_xfer+0xe8/0x3a4 [i2c_bcm2835]

[ 10.391853] __i2c_transfer+0xa4/0x4b0

[ 10.395656] i2c_transfer+0x68/0x130

[ 10.399281] regmap_i2c_read+0x64/0xb0

[ 10.403083] _regmap_raw_read+0xb4/0x270

[ 10.407062] _regmap_bus_read+0x4c/0x84

[ 10.410952] _regmap_read+0x80/0x200

[ 10.414578] _regmap_update_bits+0xd4/0x110

[ 10.418822] _regmap_select_page+0xb8/0x160

[ 10.423064] _regmap_raw_read+0x10c/0x270

[ 10.427132] _regmap_bus_read+0x4c/0x84

[ 10.431022] _regmap_read+0x80/0x200

[ 10.434647] _regmap_update_bits+0xd4/0x110

[ 10.438890] regmap_update_bits_base+0x6c/0xa0

[ 10.443398] clk_aic32x4_bdiv_set_parent+0x34/0x40 [snd_soc_tlv320aic32x4]

[ 10.450381] clk_core_set_parent_nolock+0x158/0x3b0

[ 10.455331] clk_set_parent+0x48/0x180

[ 10.459133] aic32x4_component_probe+0x6c/0x1b0 [snd_soc_tlv320aic32x4]

[ 10.465849] snd_soc_component_probe+0x30/0x84 [snd_soc_core]

[ 10.471737] soc_probe_component+0x1dc/0x374 [snd_soc_core]

[ 10.477445] snd_soc_bind_card+0x27c/0x7c0 [snd_soc_core]

[ 10.482977] snd_soc_register_card+0xfc/0x114 [snd_soc_core]

[ 10.488778] devm_snd_soc_register_card+0x54/0xb0 [snd_soc_core]

[ 10.494927] asoc_simple_probe+0x1fc/0x3f0 [snd_soc_simple_card]

[ 10.501028] platform_probe+0x70/0x110

[ 10.504831] really_probe+0xd0/0x490

[ 10.508458] __driver_probe_device+0x148/0x190

[ 10.512966] driver_probe_device+0xec/0x180

[ 10.517209] __driver_attach+0x104/0x240

[ 10.521187] bus_for_each_dev+0x78/0xd0

[ 10.525077] driver_attach+0x2c/0x40

[ 10.528702] bus_add_driver+0x154/0x270

[ 10.532594] driver_register+0x80/0x13c

[ 10.536486] __platform_driver_register+0x30/0x40

[ 10.541255] asoc_simple_card_init+0x28/0x1000 [snd_soc_simple_card]

[ 10.547706] do_one_initcall+0x4c/0x2c0

[ 10.551596] do_init_module+0x50/0x260

[ 10.555397] load_module+0xa30/0xbd0

[ 10.559021] __do_sys_finit_module+0xa8/0x114

[ 10.563438] __arm64_sys_finit_module+0x28/0x3c

[ 10.568031] invoke_syscall+0x50/0x120

[ 10.571835] el0_svc_common.constprop.0+0x180/0x1a0

[ 10.576784] do_el0_svc+0x30/0xb0

[ 10.580145] el0_svc+0x4c/0x170

[ 10.583330] el0t_64_sync_handler+0xa4/0x12c

[ 10.587660] el0t_64_sync+0x1a4/0x1a8

[ 10.591377] Code: f9403e60 aa1303e1 97ffe43e f9403a73 (f9403e80)

[ 10.597558] ---[ end trace 673aa74947e0992c ]---

[ 10.602240] note: systemd-udevd[432] exited with preempt_count 2

[ 20.192047] mmc1: Timeout waiting for hardware interrupt.

[ 70.108043] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:

[ 70.114227] rcu: 0-...!: (1 GPs behind) idle=9fd/1/0x4000000000000002 softirq=2997/3009 fqs=1

[ 70.123059] (detected by 2, t=15002 jiffies, g=1193, q=7041)

[ 70.128887] Task dump for CPU 0:

[ 70.132156] task:systemd-journal state:R running task stack: 0 pid: 390 ppid: 1 flags:0x000008 00

[ 70.142224] Call trace:

[ 70.144698] __switch_to+0xf8/0x150

[ 70.148243] __dentry_kill+0x6c/0x1e0

[ 70.151956] rcu: rcu_preempt kthread timer wakeup didn't happen for 14874 jiffies! g1193 f0x0 RCU_GP_WAIT_F QS(5) ->state=0x402

[ 70.163515] rcu: Possible timer handling issue on cpu=3 timer-softirq=292

[ 70.170487] rcu: rcu_preempt kthread starved for 14875 jiffies! g1193 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=3

[ 70.181075] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.

[ 70.190338] rcu: RCU grace-period kthread stack dump:

[ 70.195457] task:rcu_preempt state:I stack: 0 pid: 15 ppid: 2 flags:0x00000008

[ 70.203931] Call trace:

[ 70.206405] __switch_to+0xf8/0x150

[ 70.209941] __schedule+0x328/0x8b0

[ 70.213478] schedule+0x70/0x170

[ 70.216749] schedule_timeout+0xa0/0x1c4

[ 70.220727] rcu_gp_fqs_loop+0x120/0x370

[ 70.224704] rcu_gp_kthread+0x128/0x170

[ 70.228591] kthread+0x128/0x134

[ 70.231864] ret_from_fork+0x10/0x20

[ 70.235488] rcu: Stack dump where RCU GP kthread last ran:

[ 70.241047] Task dump for CPU 3:

[ 70.244315] task:kworker/3:3 state:I stack: 0 pid: 424 ppid: 2 flags:0x00000008

[ 70.252798] Workqueue: 0x0 (events)

[ 70.256428] Call trace:

[ 70.258901] __switch_to+0xf8/0x150

[ 70.262437] 0x0

Looking forward to your reply, thank you!

over 2 years ago

0 Shenghao Ding over 2 years ago

TI__Genius 12180 points

kindly tell me kernel version and platform and how do you register the sound card.

0 Alec Schuster over 2 years ago in reply to Shenghao Ding

Prodigy 10 points

The kernel version is 5.15.0-1035-raspi on Ubuntu 22.04.3 LTS. Below is the device tree I use to register the sound card:

/dts-v1/;
/plugin/;

/ {
compatible = "brcm,bcm2711", "brcm, bcm2835";

fragment@0 {
target = <&i2s>;
__overlay__ {
status = "okay";
};
};

fragment@1 {
target-path = "/";
__overlay__ {
codec_reg_1v8: codec-reg-1v8 {
compatible = "regulator-fixed";
regulator-name = "tlv320aic3204_1v8";
regulator-min-microvolt = <1800000>;
regulator-max-microvolt = <1800000>;
regulator-always-on;
};
};
};

fragment@2 {
target = <&gpio>;
__overlay__ {
codec_rst: codec-rst {
brcm,pins = <16>;
brcm,function = <1>; // BCM2835_FSEL_GPIO_OUT
};
};
};

fragment@3 {
target = <&i2c1>;
__overlay__ {
#address-cells = <1>;
#size-cells = <0>;
status = "okay";

/* audio external oscillator */
codec_osc: codec_osc {
compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <12000000>; /* 12 MHz */
};

codec1: tlv320aic32x4@18 {
#sound-dai-cells = <0>;
compatible = "ti,tlv320aic32x4";
reg = <0x18>;

clocks = <&codec_osc>;
clock-names = "mclk";

iov-supply = <&vdd_3v3_reg>;
ldoin-supply = <&vdd_3v3_reg>;

gpio-controller;
#gpio-cells = <2>;
reset-gpios = <&gpio 16 1>; // GPIO_ACTIVE_LOW

status = "okay";
};
};
};
fragment@4 {
target = <&sound>;
__overlay__ {
compatible = "simple-audio-card";
i2s-controller = <&i2s>;
status = "okay";

simple-audio-card,name = "tlv320aic3254";

simple-audio-card,widgets =
"Microphone", "Mic Jack",
"Headphone", "Headphone Jack";
simple-audio-card,routing =
"Headphone Jack", "HPL",
"Headphone Jack", "HPR",
"IN1_L", "Mic Jack",
"IN1_R", "Mic Jack",
"IN2_L", "Mic Jack",
"IN2_R", "Mic Jack",
"Mic Jack", "Mic Bias";

simple-audio-card,format = "i2s";
simple-audio-card,bitclock-master = <&rpi>;
simple-audio-card,frame-master = <&rpi>;
rpi: simple-audio-card,cpu {
sound-dai = <&i2s>;
};
dailink0_master: simple-audio-card,codec {
sound-dai = <&codec1>;
};

};
};
};

0 Shenghao Ding over 2 years ago in reply to Alec Schuster

TI__Genius 12180 points

From the log, why did there call snd_soc_tlv320aic32x4_spi?

Kindly to check whether the i2c bus work with i2c-tools.

0 Alec Schuster over 2 years ago in reply to Shenghao Ding

Prodigy 10 points

The i2c bus is working properly. We are able to control the codec and record audio in Ubuntu. These boot issues are intermittent and do not happen every time (although the kernel bug happens each time).

The device tree defines this device as compatible so it is most likely linking all the modules related to it. However, we have SPI disabled so that module should not be used.

The codec works properly as long as the OS can boot. It appears to be a driver issue with this codec because once we disable this device in our boot config we can boot reliably. The core issue is that loading in the driver for this device specified in the device tree is resulting in the issue above that prevents boot. Based on the logs, it appears to be an issue with scheduling a task.

If boot is successful, the device functions as we expect. But, we would like to be able to fix this boot issue as our device is required to boot reliably.

0 Shenghao Ding over 2 years ago in reply to Alec Schuster

TI__Genius 12180 points

I think this is not an easy issue to handle. According to the log. the issue occurred in clk_set_parent called by snd_soc_component_probe, kindly check whether the clk is ready during bootup every time.

0 Alec Schuster over 2 years ago in reply to Shenghao Ding

Prodigy 10 points

We confirmed that the mclk derived from our crystal oscillator is always available on reboots. The word clock and bit clock do not become active until we start recording, which happens after the device is fully booted. It appears to us that the hardware is functioning properly during a reboot. Is there anything else we should take a look at? It seems like clk_set_parent is being called from an atomic context when it should not be.

0 Shenghao Ding over 2 years ago in reply to Alec Schuster

TI__Genius 12180 points

According to your input, I want to confirm from you whether this issue occured during disabling the audio driver? I have a bit confusion.

0 Alec Schuster over 2 years ago in reply to Shenghao Ding

Prodigy 10 points

This issue occurs when the driver is enabled. It goes away when the driver is disabled. Therefore, we believe the issue is with the driver itself.

0 Shenghao Ding over 2 years ago in reply to Alec Schuster

TI__Genius 12180 points

Let's have a try on the following way

Implent .mute_stream in snd_soc_component_driver, and move all the code in the probe into mute_stream, then check whether the issue occur.

0 Alec Schuster over 2 years ago in reply to Shenghao Ding

Prodigy 10 points

We are not able to modify the driver and must use what is included with the kernel/Ubuntu. If this issue cannot be resolved by a patch that will be in future releases then we will have to switch to a different codec supplier with better Linux compatibility.

0 Shenghao Ding over 2 years ago in reply to Alec Schuster

TI__Genius 12180 points

Hi Alex

I'm extremely sorry to see that. Would you be so kind and ship your platform to me, we can reproduce the issue, looking into it and make a patch for you?

As you know, we have no similar environment to reproduce this issue. Looking forward to your platform. Thanks.

Audio

Audio forum

TLV320AIC3254: Kernel Errors and Boot Failures Due to TLV32x4 Driver