TDA4VM: SDK 7.0 Kernel Panic with openssl with -iter or -pbkdf2 options

Pouyan Azari

Part Number: TDA4VM

Hi everyone,

Using the following openssl command line parameters leads to kernel panic on the TDA4 EVM on the SDK7.0. The same did not happen on the SDK 6.2:

openssl aes-256-cbc -pbkdf2 -salt -in test.txt -out test.txt.enc -pass pass:test #On You Ubuntu Machine
scp test.txt.enc root@YOUR-EVM:/home/root/
sudo openssl aes-256-cbc -salt -pbkdf2 -d -in test.txt.enc -out test.txt -pass pass:test # On your EVM

root@j7-evm:~# sudo openssl aes-256-cbc -salt -pbkdf2 -d -in test.txt.enc -out test.txt -pass pass:test
[  345.882145] Unable to handle kernel paging request at virtual address fbffa0a8345e0040
[  345.890072] Mem abort info:
[  345.892896]   ESR = 0x96000044
[  345.895946]   EC = 0x25: DABT (current EL), IL = 32 bits
[  345.901246]   SET = 0, FnV = 0
[  345.904292]   EA = 0, S1PTW = 0
[  345.907423] Data abort info:
[  345.910289]   ISV = 0, ISS = 0x00000044
[  345.914120]   CM = 0, WnR = 1
[  345.917081] [fbffa0a8345e0040] address between user and kernel address ranges
[  345.924202] Internal error: Oops: 96000044 [#1] PREEMPT SMP
[  345.929758] Modules linked in: xt_conntrack xt_MASQUERADE xt_addrtype iptable_filter iptable_nat nf_nat nf_conntrack nf_defrag_ipv4 libcrc32c ip_tables x_tables br_netfilter bridge stp llc xfrm_user xfrm_algo md5 ecb aes_neon_bs aes_neon_blk des_generic libdes cbc xhci_plat_hcd xhci_hcd usbcore ti_am335x_adc kfifo_buf cdns3 pru_rproc roles irq_pruss_intc udc_core usb_common omap_rng rng_core crct10dif_ce snd_soc_j721e_evm j721e_cpsw_virt_mac ti_k3_r5_remoteproc ti_am335x_tscadc cdns_mhdp pruss sa2ul ntb_hw_epf sha512_generic ti_k3_dsp_remoteproc ntb pci_endpoint_test authenc virtio_rpmsg_bus cdns3_ti snd_soc_pcm3168a_i2c snd_soc_pcm3168a pvrsrvkm(O) rti_wdt sch_fq_codel rpmsg_kdrv_switch jailhouse(O) cryptodev(O) ipv6 nf_defrag_ipv6
[  345.994551] CPU: 1 PID: 1453 Comm: openssl Tainted: G           O      5.4.40-g66cf445b76 #2
[  346.002965] Hardware name: Texas Instruments K3 J721E SoC (DT)
[  346.008778] pstate: 60000005 (nZCv daif -PAN -UAO)
[  346.013557] pc : __memcpy+0x74/0x180
[  346.017121] lr : scatterwalk_copychunks+0xdc/0x1e0
[  346.021895] sp : ffff80001794f9d0
[  346.025194] x29: ffff80001794f9d0 x28: ffff00084ff22a00 
[  346.030489] x27: ffff00084ff22a00 x26: 0000000000000000 
[  346.035783] x25: fbffa0a8345e0040 x24: ffff80001794fa78 
[  346.041077] x23: 0000000000000020 x22: 0000000000000020 
[  346.046371] x21: 0000000000000020 x20: 0000002000200000 
[  346.051665] x19: ffff000000000000 x18: 0000000000000000 
[  346.056959] x17: 0000000000000000 x16: 0000000000000000 
[  346.062253] x15: 0000000000000000 x14: 0000000000000000 
[  346.067547] x13: 0000000000000000 x12: 0000000000000000 
[  346.072840] x11: 0000000000000000 x10: 0000000000000000 
[  346.078134] x9 : 0000000000000000 x8 : d13d594f97983f83 
[  346.083428] x7 : fdb196c2ecaa48e1 x6 : fbffa0a8345e0040 
[  346.088722] x5 : 0000000000000e90 x4 : 0000000000000000 
[  346.094015] x3 : 0000000000000020 x2 : 0000000000000020 
[  346.099309] x1 : ffff00082bf4f180 x0 : fbffa0a8345e0040 
[  346.104603] Call trace:
[  346.107038]  __memcpy+0x74/0x180
[  346.110252]  scatterwalk_map_and_copy+0x7c/0xd8
[  346.114771]  sa_sham_update+0x194/0x230 [sa2ul]
[  346.119290]  cryptodev_hash_update+0x30/0xc0 [cryptodev]
[  346.124586]  hash_n_crypt.isra.0+0xf0/0x120 [cryptodev]
[  346.129795]  crypto_run+0xc8/0x478 [cryptodev]
[  346.134224]  cryptodev_ioctl+0x3c4/0xb10 [cryptodev]
[  346.139173]  do_vfs_ioctl+0x964/0xb48
[  346.142820]  ksys_ioctl+0x78/0xa8
[  346.146120]  __arm64_sys_ioctl+0x1c/0x28
[  346.150028]  el0_svc_common.constprop.0+0x68/0x160
[  346.154802]  el0_svc_handler+0x20/0x80
[  346.158536]  el0_svc+0x8/0xc
[  346.161406] Code: 540000ab a8c12027 a88120c7 a8c12027 (a88120c7) 
[  346.167481] ---[ end trace 97e6dd6f8f5e0f7f ]---
[  346.172105] note: openssl[1453] exited with preempt_count 2

Also the first command gives Kernel panic on the EVM:

[  691.757970] Unable to handle kernel paging request at virtual address fbffa0a8375c0040
[  691.765905] Mem abort info:
[  691.768745]   ESR = 0x96000044
[  691.771796]   EC = 0x25: DABT (current EL), IL = 32 bits
[  691.777096]   SET = 0, FnV = 0
[  691.780142]   EA = 0, S1PTW = 0
[  691.783273] Data abort info:
[  691.786139]   ISV = 0, ISS = 0x00000044
[  691.789964]   CM = 0, WnR = 1
[  691.792923] [fbffa0a8375c0040] address between user and kernel address ranges
[  691.800044] Internal error: Oops: 96000044 [#2] PREEMPT SMP
[  691.805599] Modules linked in: xt_conntrack xt_MASQUERADE xt_addrtype iptable_filter iptable_nat nf_nat nf_conntrack nf_defrag_ipv4 libcrc32c ip_tables x_tables br_netfilter bridge stp llc xfrm_user xfrm_algo md5 ecb aes_neon_bs aes_neon_blk des_generic libdes cbc xhci_plat_hcd xhci_hcd usbcore ti_am335x_adc kfifo_buf cdns3 pru_rproc roles irq_pruss_intc udc_core usb_common omap_rng rng_core crct10dif_ce snd_soc_j721e_evm j721e_cpsw_virt_mac ti_k3_r5_remoteproc ti_am335x_tscadc cdns_mhdp pruss sa2ul ntb_hw_epf sha512_generic ti_k3_dsp_remoteproc ntb pci_endpoint_test authenc virtio_rpmsg_bus cdns3_ti snd_soc_pcm3168a_i2c snd_soc_pcm3168a pvrsrvkm(O) rti_wdt sch_fq_codel rpmsg_kdrv_switch jailhouse(O) cryptodev(O) ipv6 nf_defrag_ipv6
[  691.870390] CPU: 1 PID: 1679 Comm: openssl Tainted: G      D    O      5.4.40-g66cf445b76 #2
[  691.878803] Hardware name: Texas Instruments K3 J721E SoC (DT)
[  691.884617] pstate: 60000005 (nZCv daif -PAN -UAO)
[  691.889396] pc : __memcpy+0x74/0x180
[  691.892959] lr : scatterwalk_copychunks+0xdc/0x1e0
[  691.897732] sp : ffff8001204af9d0
[  691.901031] x29: ffff8001204af9d0 x28: ffff00082c3f2a00 
[  691.906326] x27: ffff00082c3f2a00 x26: 0000000000000000 
[  691.911620] x25: fbffa0a8375c0040 x24: ffff8001204afa78 
[  691.916914] x23: 0000000000000020 x22: 0000000000000020 
[  691.922208] x21: 0000000000000020 x20: 0000002000200000 
[  691.927502] x19: ffff000000000000 x18: 0000000000000000 
[  691.932796] x17: 0000000000000000 x16: 0000000000000000 
[  691.938089] x15: 0000000000000000 x14: 0000000000000000 
[  691.943383] x13: 0000000000000000 x12: 0000000000000000 
[  691.948677] x11: 0000000000000000 x10: 0000000000000000 
[  691.953970] x9 : 0000000000000000 x8 : d5323fa8b5edffcb 
[  691.959264] x7 : f4e49c95f163fccd x6 : fbffa0a8375c0040 
[  691.964558] x5 : 000000000000f6d0 x4 : 0000000000000000 
[  691.969852] x3 : 0000000000000020 x2 : 0000000000000020 
[  691.975146] x1 : ffff00082c2f0940 x0 : fbffa0a8375c0040 
[  691.980440] Call trace:
[  691.982875]  __memcpy+0x74/0x180
[  691.986089]  scatterwalk_map_and_copy+0x7c/0xd8
[  691.990610]  sa_sham_update+0x194/0x230 [sa2ul]
[  691.995130]  cryptodev_hash_update+0x30/0xc0 [cryptodev]
[  692.000426]  hash_n_crypt.isra.0+0xf0/0x120 [cryptodev]
[  692.005636]  crypto_run+0xc8/0x478 [cryptodev]
[  692.010065]  cryptodev_ioctl+0x3c4/0xb10 [cryptodev]
[  692.015014]  do_vfs_ioctl+0x964/0xb48
[  692.018661]  ksys_ioctl+0x78/0xa8
[  692.021961]  __arm64_sys_ioctl+0x1c/0x28
[  692.025871]  el0_svc_common.constprop.0+0x68/0x160
[  692.030645]  el0_svc_handler+0x20/0x80
[  692.034380]  el0_svc+0x8/0xc
[  692.037250] Code: 540000ab a8c12027 a88120c7 a8c12027 (a88120c7) 
[  692.043326] ---[ end trace 97e6dd6f8f5e0f80 ]---
[  692.047940] note: openssl[1679] exited with preempt_count 2

Thanks,

Pouyan

over 3 years ago

0 Pouyan Azari over 3 years ago

Intellectual 620 points

Removing cryptodev as mentioned here solves the problem, so it seems that the cryptodev driver has a bug.

rmmod cryptodev

0 Keerthy J over 3 years ago in reply to Pouyan Azari

TI__Guru*** 132170 points

Hi Pouyan,

Did you recollect if you has used cryptodev in 6.02 as well? Was the openssl using the hardware crypto or A72?

- Keerthy

0 Pouyan Azari over 3 years ago in reply to Keerthy J

Intellectual 620 points

Hi Keerthy,

I tried the same command on the EVM using the original SDK 6.2:

openssl aes-256-cbc -pbkdf2 -salt -in test.txt -out test.txt.enc -pass pass:test

it works, the cryptodev module is also loaded there. But I dont know if it is used.

Thanks,

Pouyan

0 Keerthy J over 3 years ago in reply to Pouyan Azari

TI__Guru*** 132170 points

Hi Pouyan,

I checked quickly on 6.02 the command:

sudo openssl aes-256-cbc -salt -pbkdf2 -d -in test.txt.enc -out test.txt -pass pass:test

Does not seem to be using hardware cyrpto on 6.02 & it is using the hardware crypto with 7.0.
By looking at the trace: sa_sham_update+0x194/0x230 [sa2ul]

I will check internally and get back to you on this in couple of days. Are you blocked on this
or are you good with rmmod cryptodev for now?

- Keerthy

0 Pouyan Azari over 3 years ago in reply to Keerthy J

Intellectual 620 points

Hi Keerthy,

it is not a blocker at the moment.

But it would be helpful if it is solved soon.

Thanks,

Pouyan

0 Keerthy J over 3 years ago in reply to Pouyan Azari

TI__Guru*** 132170 points

Hi Pouyan,

It seems like openssl with SA2UL(hardware accelerator) is known to fail with SHA algorithms
and hence openssl usage of sa2ul for SHA algorithms is disabled in the latest upstream Linux kernel.

This was done with a bunch of patches effectively disabling usage of hardware accelerator
for openssl SHA. sa_sham_update is clearly in the call stack that seems to be causing the problem.

I would recommend to use openssl with software algorithms.

Best Regards,
Keerthy

0 Pouyan Azari over 3 years ago in reply to Keerthy J

Intellectual 620 points

Hi Keerthy,

Thanks for the infromation.

I am using the existing openssl delivered in the SDK7.0. Other than disabling the cryptodev module is there anyway to explicitly use openssl with software algos? (A command line parameter?)

best regards,

Pouyan

0 Keerthy J over 3 years ago in reply to Pouyan Azari

TI__Guru*** 132170 points

Hi Pouyan,

The method that you are employing is just fine. Can you just reply with that so
that it will be useful for others.

Best Regards,
Keerthy

0 Keerthy J over 3 years ago in reply to Keerthy J

TI__Guru*** 132170 points

Pouyan,

rmmod cryptodev

You have already mentioned that. Thanks for the information. Anything else that
i need check on this?

Best Regards,
Keerthy

0 Keerthy J over 3 years ago in reply to Keerthy J

TI__Guru*** 132170 points

Hi Pouyan,

If there are no further questions on this could you please resolve this?

Best Regards,
Keerthy

Processors

Processors forum

TDA4VM: SDK 7.0 Kernel Panic with openssl with -iter or -pbkdf2 options