• State Resolved
  • Locked Locked
  • Replies 20 replies
  • Subscribers 107 subscribers
  • Views 8581 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM335X: ADC lock-up in kernel v3.14

Craig McQueen
Intellectual 850 points

I'm using Yocto to build for BeagleBone Black, using the linux-ti-staging kernel, branch ti-linux-3.14.y, using the meta-ti layer.


I'd like to use several ADC channels, so I've configured as follows in the device tree:

&tscadc {
	status = "okay";

	adc {
		ti,adc-channels = <0 1 4 6 7>;
	};
};

If I just read a single ADC channel, then everything works okay. So this is fine:

while true; do echo ----; cat /sys/bus/iio/devices/iio\:device0/in_voltage4_raw; done


But, if I try to read multiple channels, then it quickly locks up. This locks up:

while true; do echo ----; cat /sys/bus/iio/devices/iio\:device0/in_voltage*_raw; done

It typically runs just for a brief time such as 1 second or less, then halts. It's not a system crash, but the process is stuck in the "uninterruptible wait" state according to ps.

I looked at the ti_am335x_adc.c driver, and thought it could do with a mutex to protect most of the code in tiadc_read_raw(). I tried adding a mutex, but that didn't help.

Looking in ti_am335x_tscadc.c, I suspect it would be locking up in am335x_tscadc_need_adc().

Can anyone confirm this behaviour and propose a fix?

  • 0
    TI__Guru**** 393215 points

    Hi,

    See this wiki for ADC driver usage: http://processors.wiki.ti.com/index.php/Linux_Core_ADC_User%27s_Guide I don't think you can replace channel number with asterisk.

  • 0 in reply to Biser Gatchev-XID
    Intellectual 850 points

    Thank you for the reference. I am reading channels in one-shot mode.

    You said "I don't think you can replace channel number with asterisk". Yes I can, when doing this at a command prompt, which I was doing. It is the standard shell wildcard, so it expands to all the matching filenames. For example:

    # ls /sys/bus/iio/devices/iio\:device0/in_voltage*_raw
/sys/bus/iio/devices/iio:device0/in_voltage0_raw
/sys/bus/iio/devices/iio:device0/in_voltage1_raw
/sys/bus/iio/devices/iio:device0/in_voltage4_raw
/sys/bus/iio/devices/iio:device0/in_voltage6_raw
/sys/bus/iio/devices/iio:device0/in_voltage7_raw

    Then to show current ADC values:

    # cat /sys/bus/iio/devices/iio\:device0/in_voltage*_raw
4094
3595
2546
4076
3832

    What I was showing before is that if I do the above command multiple times (in a shell while loop), then it doesn't take long before it locks up.

    Could someone at TI please try the command in my original message, to verify if they get the same lock-up of the ADC driver.

  • 0 in reply to Craig McQueen
    TI__Guru**** 161600 points

    Hi Craig,

    I verified the behavior on the BBB. I think this is normal, you cannot read all channels at once.
    I modified the script as follows:
    #!/bin/sh
    while true; do
    sleep 1
    echo ------
    cat /sys/bus/iio/devices/iio\:device0/in_voltage1_raw
    sleep 1
    cat /sys/bus/iio/devices/iio\:device0/in_voltage0_raw
    sleep 1
    cat /sys/bus/iio/devices/iio\:device0/in_voltage4_raw
    sleep 1
    done

    And was able to read adc channels for as long as I want. Board is behaving ok, see log:
    root@am335x-evm:~# ./test.sh
    ------
    3515
    3917
    2275
    ------
    3547
    3919
    2292
    ------
    3553
    3913
    2232
    ------
    3545
    3925
    2284
    ------
    3516
    3929
    2288
    ------
    3504
    3924
    2243
    ^C
    root@am335x-evm:~#

    If you need you can lower the sleep time in the shell script, you can even remove the sleep lines (board is stable, no hangs). Or you can create a user space driver to read adc channels; this should lower the time between consecutive reads significantly. 

    Best Regards,
    Yordan

  • 0 in reply to Yordan Kovachev
    TI__Guru 55325 points
    I’m not familiar with how the software driver works, but maybe the following description of the ADC hardware will help answer your question.

    The AM335x Touchscreen Controller has a single ADC with eight analog inputs. The single ADC is connected to one of eight analog inputs via an 8-to-1 analog multiplexer. Therefore, only one input can be measured for each ADC conversion.

    Also note: The maximum sample rate defined in the data sheet applies to the ADC, so the maximum sample rate of each input would be the data sheet maximum value divided by number of inputs being measured.

    Regards,
    Paul
  • 0 in reply to Yordan Kovachev
    Intellectual 850 points

    Yordan, it's the driver's responsibility to implement correct timing of ADC reads, not the application's. The driver is getting stuck in an "uninterruptible sleep", so that is definitely a bug in the driver which needs fixing.

  • 0 in reply to peaves
    Intellectual 850 points
    peaves, the hardware details of the ADC is good to know, in that it can explain why a read of a particular ADC input might take some time (say, 100 µs for example), to allow for channel set up and conversion. But the driver should robustly handle the channel setup and wait for conversion to complete, so the application's read will block until it's done. The driver definitely shouldn't lock up the application in an uninterruptible sleep.

    Is this something that an engineer at TI can fix in the near future?
  • 0 in reply to Yordan Kovachev
    Intellectual 850 points
    Can we please have a follow-up on this issue?

    First of all, it is a bad bug, because:

    * It results in a process getting stuck in an "uninterruptible sleep"
    * It thus prevents processes from reading more than one ADC input
    * The proposed work-around can't work if more than one process or kernel driver wants to read ADC inputs

    In my device, I have several kernel drivers that need to read ADC inputs, for different purposes. The only possible way for these to work is for the TI ADC driver to be fixed.

    Can TI provide a fix for this driver, and if so, in what time frame?
  • 0 in reply to Craig McQueen
    Intellectual 850 points
    I could fix this myself, however, I've had difficulty contributing kernel fixes back to TI, so it would be more beneficial for other users if TI engineers could do it. Please let me know if TI engineers can fix this, and in what time frame.
  • 0 in reply to Yordan Kovachev
    Intellectual 850 points
    Can TI provide a fix for this driver, and if so, in what time frame?
  • 0 in reply to Yordan Kovachev
    Intellectual 850 points
    Hello, I am disappointed not to hear anything further from TI regarding this. The answers given so far (that the "uninterruptible sleep" lock-up behaviour is normal) are unsatisfactory. I would at least like to have some further feedback from someone at TI as to whether they are able to address this driver bug in a timely fashion, or whether I am on my own with coming up with a driver fix.
  • 0 in reply to Yordan Kovachev
    Intellectual 850 points
    In my device, I want to implement several kernel drivers that need to read ADC inputs, for different purposes. Because they are separate drivers, it is not possible to coordinate their ADC reads (as you said, adding a sleep between reads of different channels) so that they don't trigger this TI ADC driver bug. How would you recommend using the TI ADC driver reliably in this scenario?
  • 0 in reply to Craig McQueen
    Intellectual 330 points

    I am also observing this behaviour of the ti_am335x_adc driver getting into an uninterruptible sleep.

    I'm currently using TI SDK V.8, and to trigger this behaviour I'm calling iio_read_channel_raw with no delay between calls on the same channel. 

    Is there any progress on correcting this issue from TI? 

    Thanks,

    Sean

  • 0
    Prodigy 30 points
    According to this thread (www.serverphorums.com/read.php) and
    taking into account that you don't use touch screen and only ADC, you need to fix only one file:
    --- ti_am335x_tscadc.c.orig 2015-03-11 23:48:00.000000000 +0300
    +++ ti_am335x_tscadc.c 2015-06-23 13:43:26.716490956 +0300
    @@ -101,7 +101,6 @@
    void am335x_tsc_se_set_once(struct ti_tscadc_dev *tsadc, u32 val)
    {
    spin_lock_irq(&tsadc->reg_lock);
    - tsadc->reg_se_cache |= val;
    am335x_tscadc_need_adc(tsadc);

    tscadc_writel(tsadc, REG_SE, val);

    As described, In one shot mode, sequencer automatically disables all enabled steps at
    the end of each cycle. (both ADC steps and TSC steps) Hence these steps
    need not be saved in reg_se_cache for clearing these steps at a later
    stage.
  • 0 in reply to Biser Gatchev-XID
    Prodigy 140 points
    Hi Biser:

    I want to research how to using ADC and related sample, but I find many links in processors.wiki.ti.com/.../Processor_SDK_Linux_ADC were requested with following message
    "
    Access Denied
    You don't have permission to access "processors.wiki.ti.com/.../Linux_Core_ADC_User's_Guide" on this server.
    Reference #18.377145cb.1484534053.9d6231b4
    "

    Could you help me to solve this problem, otherwise I have no idea to find solution to solve problem related ADC.

    Best Regards,
    Alvin
  • 0 in reply to Alvin Chang
    Intellectual 850 points
    I can verify that—I can't access the link either.
  • 0 in reply to Craig McQueen
    Prodigy 60 points

    Craig McQueen,

    We have same issue with linux kernel 4.4.12. Application gets hang at read() system call while reading from "/sys/bus/iio/devices/iio\:device0/in_voltage0_raw" with interval of 100ms.

    Can you share the information how did you resolved the ADC hangs issue from kernel?

    Regards,

    Darshan

  • 0 in reply to Darshan Patel38
    Intellectual 850 points

    I had to apply two patches:

    • kernel commit "iio: adc: ti_am335x_adc: Increase timeout value waiting for ADC sample", which you may find as commit 7175cce1c3f1d8c8840d2004f78f96a3904249b5 or 023e76b1d1fd4857ba6af6e99dfbefd667aae6d1. 
    • kernel commit "iio: adc: ti_am335x_adc: Protect FIFO1 from concurrent access", which you may find as commit 90c43ec6997a892448f1f86180a515f59cafd8a3 or a1f72192d04cf4cef432f37b590f6c5f9c8f9791.

  • 0 in reply to Darshan Patel38
    Intellectual 850 points
    Also see e2e.ti.com/.../1945876 which addresses this issue for the 4.4.12 kernel directly.
  • 0 in reply to Craig McQueen
    Prodigy 60 points

    Craig McQueen,

    We have already applied the patches which you have suggested. Also we have tested with Linux 4.4.41. but unfortunately both kernel gets hangs in 5KV ESD test.

    Is there any other solution for the same issue?

    Regards,

    Darshan

  • 0 in reply to Darshan Patel38
    Intellectual 850 points

    I haven't seen an issue with the ADC driver during ESD tests myself. It would be worth asking this as a separate question, which hopefully an engineer from TI could answer.

    My only other suggestion is if you're accessing the ADC from another kernel driver (as I am), to ensure it uses the iio_read_channel_raw() call to get an ADC reading, and doesn't call the ADC driver's .read_raw() function directly.