This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM625: 1 second timeout in spi_sync()

Part Number: AM625

Hello TI,

I Hope you can help me with the following issue:
I have build a Driver for controlling Leds that can be chained together similiar to apa102 leds.

So My Driver works, with PIO but my kernel thread uses a lot of cycles, I wanted to switch to DMA but noticed an issue, if I look at my logic analyzer output i see that in PIO mode the data isn't nicely packed up but it's there all complete (sometimes a gap as the processor is probably interrupting the kernel process), and everything is all fine and good.
how ever when switching to DMA the stream of data has big gaps between transmissions and that gap is almost exactly 1 second give or take a few milliseconds.

I don't know why this is happening exactly and hope you can help me out.

The Driver is a framebuffer driver and a led driver,
I have setup the framebuffer driver to be memory mapable, and it uses a kernel thread that is triggerd by a timer every 33ms or so (to get a 30 packets per second roughtly)
This happens through the frame_timer_callback
which wakes up the frame_task (which is the frame_thread_fn)
The data is the coppied inside the rgbled_write(); into a private buffer which is used to encode the format for the data and then transmit it via spi.
this private buffer is allocated with devm_kzalloc()
the spi is setup with 8 bits per word and transmits a total of 248 bytes, it does transmit only is that an issue? did i maybe get my dmas setup wrong, it was very difficult to find the right numbers.
is it because i use a gpio as chip select?

 /* thread */
static int frame_thread_fn(void *data) {
    // struct kitt_fb_par *par = data;
    set_freezable();
    while (!kthread_should_stop()) {
        try_to_freeze();
        set_current_state(TASK_INTERRUPTIBLE);
        schedule();
        if (kthread_should_stop())
            break;
        /* do frame work */
        if (led_spi)
            rgbled_write(led_spi, (uint32_t *)videomemory, 60);
    }
    return 0;
}
static unsigned long setup_frame_thread(struct kitt_fb_par *par) {
    pr_info("Setting up a frame thread.\n");
    frame_thread = kthread_create(frame_thread_fn, par, "frame thread");
    if(frame_thread) {
        par->frame_task = frame_thread;
        wake_up_process(frame_thread);
        return 0;
    }

    pr_err("Could not create and run the frame thread.");
    return -ENOMEM;
}

static int probe(..) {
..
    hrtimer_init(&par->frame_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
    par->frame_timer.function = frame_timer_callback;
    hrtimer_start(&par->frame_timer, ms_to_ktime(33), HRTIMER_MODE_REL);
..
}



There are no error messages in dmesg log.
and the way i found the issue was around spi_sync() was by timing it with ktime()

further device info:

Linux version 6.6.138-7.2.0-devel (oe-user@oe-host) (aarch64-tdx-linux-gcc (GCC) 13.3.0, GNU ld (GNU Binutils) 2.42.0.20240723) #1 SMP PREEMPT Fri May  8 07:27:41 UTC 2026
Machine model: Toradex Verdin AM62 WB on Verdin Development Board

root@verdin-am62-15599560:~# tdx-info

Software summary
------------------------------------------------------------
Bootloader:               U-Boot
Kernel version:           6.6.138-7.2.0-devel #1 SMP PREEMPT Fri May  8 07:27:41 UTC 2026
Kernel command line:      root=PARTUUID=076c4a2a-01 ro rootwait console=tty1 console=ttyS2,115200 no_console_suspend rauc.slot=A
Distro name:              NAME="TDX Wayland with XWayland"
Distro version:           VERSION_ID=7.2.0-devel-20251204100250-build.0
Distro variant:           -
Hostname:                 verdin-am62-15599560
------------------------------------------------------------

Hardware info
------------------------------------------------------------
HW model:                 Toradex Verdin AM62 WB on Verdin Development Board
Toradex version:          0072 V1.2A
Serial number:            15599560
Processor arch:           aarch64
------------------------------------------------------------
root@verdin-am62-15599560:~#

root@verdin-am62-15599560:~# cat /sys/kernel/debug/dmaengine/summary
dma0 (485c0100.dma-controller): number of channels: 48
 dma0chan0    | 2b00000.audio-controller:tx (MEM_TO_DEV, tchan0 [0x2000 -> 0xc500], PDMA[ ACC32 BURST ], TR mode)
 dma0chan1    | 2b00000.audio-controller:rx (DEV_TO_MEM, rchan6 [0x4500 -> 0xa006], PDMA[ ACC32 BURST ], TR mode)
 dma0chan2    | in-use (MEM_TO_MEM, bchan0)

dma1 (485c0000.dma-controller): number of channels: 35
 dma1chan0    | 20110000.spi:rx0 (DEV_TO_MEM, rchan0 [0x4304 -> 0x9000], rflow0, PDMA, Packet mode)
 dma1chan1    | 20110000.spi:tx0 (MEM_TO_DEV, tchan0 [0x1000 -> 0xc304], tflow0, PDMA, Packet mode)
 dma1chan2    | 40900000.crypto:rx1 (DEV_TO_MEM, rchan22 [0x7506 -> 0x9016], rflow43, PSI-L Native[ EPIB PSDsize:64 ], Packet mode)
 dma1chan3    | 40900000.crypto:rx2 (DEV_TO_MEM, rchan23 [0x7507 -> 0x9017], rflow44, PSI-L Native[ EPIB PSDsize:64 ], Packet mode)
 dma1chan4    | 40900000.crypto:tx (MEM_TO_DEV, tchan28 [0x101c -> 0xf501], tflow91, PSI-L Native[ EPIB PSDsize:64 ], Packet mode)
root@verdin-am62-15599560:~#

this is in my devicetree for the spi device:

&main_spi1 {
	#address-cells = <1>;
	#size-cells = <0>;
	status = "okay";

	dmas = <&main_pktdma 0xC304 0>, <&main_pktdma 0x4304 0>;
	dma-names = "tx0", "rx0";

	sparkled: spark-led@0 {
		compatible = "wurth,spark-led";

		reg = <0>;
		
		spi-max-frequency = <115264>;
		num-cs = <1>;
		cs-gpios = <&main_gpio1 93 GPIO_ACTIVE_HIGH>;
		spi-cs-high;

		vled-supply = <&reg_V5V0_LED>;
	};
};

Hopefully you can help me in the right direction or have any ideas why this might happen

  • Hi Robert,
    Apologies for the delay. Please let me re-assign the thread to the right expert.

  • Hi Robert,

    Please check the conversation in the thread linked below and the mentioned kernel patch resolve the delay issue.

     AM62L-EVSE-DEV-EVM: dmaengine: ti: k3-udma: 1 second polling delay 

  • Thank you very much! i will try the patch that was suggested, but i am running kernel version 6.6.x so it doesn't really apply nicely, trying to apply it manually currently. 

  • i made a backport of the patch for my case (kernel version 6.6.138) and that works.

    From 09c725b55630a8e1c723c709cca62debc53221d6 Mon Sep 17 00:00:00 2001
    From: Robert van der Tuuk <robert@kitt.nl>
    Date: Thu, 28 May 2026 08:56:16 +0000
    Subject: [PATCH] Backport of patch from
     https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/commit/drivers/spi/spi-omap2-mcspi.c?h=ti-linux-6.1.y&id=ea7d9188568669e6a1eb2e413b6c46bc208f0eb8
     Forum issue:
     https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1648054/am62l-evse-dev-evm-dmaengine-ti-k3-udma-1-second-polling-delay
     https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1649459/am625-1-second-timeout-in-spi_sync/6360082
    
    original message:
    
    spi: spi-omap2-mcspi: Use EOW interrupt for completion when DMA enabled
    In MCSPI controller EOW interrupt is triggered when the channel has
    transmitted the set number of bytes in MCSPI_XFERLEVEL[31-16] WCNT,
    this can be used to signal the completion of a TX/RX when the internal
    FIFO is enabled, when DMA is enabled the internal FIFO is always enabled.
    Waiting for the DMA completion adds unpredictable delays due to the
    non-realtime completion calculation mechanism.
    
    Remove the dma_tx_completion and dma_rx_completion and rely on the
    MCSPI controller EOW interrupt to signal transaction completion.
    This fixes the real-time performance issues in master and slave mode
    when DMA was enabled which resulted from the DMA completion calculation
    delays.
    
    Since the MCSPI driver now uses internal mechanism to identify a transfer
    completion we disable the TX and RX DMA completion callback and remove
    DMA_PREP_INTERRUPT.
    
    Signed-off-by: Robert van der Tuuk <robert@kitt.nl>
    ---
     drivers/spi/spi-omap2-mcspi.c | 158 +++++++++++-----------------------
     1 file changed, 50 insertions(+), 108 deletions(-)
    
    diff --git a/drivers/spi/spi-omap2-mcspi.c b/drivers/spi/spi-omap2-mcspi.c
    index 35ca8fda45aa..9d1b43a4e55a 100644
    --- a/drivers/spi/spi-omap2-mcspi.c
    +++ b/drivers/spi/spi-omap2-mcspi.c
    @@ -91,10 +91,6 @@
     struct omap2_mcspi_dma {
     	struct dma_chan *dma_tx;
     	struct dma_chan *dma_rx;
    -
    -	struct completion dma_tx_completion;
    -	struct completion dma_rx_completion;
    -
     	char dma_rx_ch_name[14];
     	char dma_tx_ch_name[14];
     };
    @@ -116,7 +112,7 @@ struct omap2_mcspi_regs {
     };
     
     struct omap2_mcspi {
    -	struct completion	txdone;
    +	struct completion	txrxdone;
     	struct spi_controller	*ctlr;
     	/* Virtual base address of the controller */
     	void __iomem		*base;
    @@ -377,30 +373,6 @@ static int mcspi_wait_for_completion(struct  omap2_mcspi *mcspi,
     	return 0;
     }
     
    -static void omap2_mcspi_rx_callback(void *data)
    -{
    -	struct spi_device *spi = data;
    -	struct omap2_mcspi *mcspi = spi_controller_get_devdata(spi->controller);
    -	struct omap2_mcspi_dma *mcspi_dma = &mcspi->dma_channels[spi_get_chipselect(spi, 0)];
    -
    -	/* We must disable the DMA RX request */
    -	omap2_mcspi_set_dma_req(spi, 1, 0);
    -
    -	complete(&mcspi_dma->dma_rx_completion);
    -}
    -
    -static void omap2_mcspi_tx_callback(void *data)
    -{
    -	struct spi_device *spi = data;
    -	struct omap2_mcspi *mcspi = spi_controller_get_devdata(spi->controller);
    -	struct omap2_mcspi_dma *mcspi_dma = &mcspi->dma_channels[spi_get_chipselect(spi, 0)];
    -
    -	/* We must disable the DMA TX request */
    -	omap2_mcspi_set_dma_req(spi, 0, 0);
    -
    -	complete(&mcspi_dma->dma_tx_completion);
    -}
    -
     static void omap2_mcspi_tx_dma(struct spi_device *spi,
     				struct spi_transfer *xfer,
     				struct dma_slave_config cfg)
    @@ -415,12 +387,9 @@ static void omap2_mcspi_tx_dma(struct spi_device *spi,
     	dmaengine_slave_config(mcspi_dma->dma_tx, &cfg);
     
     	tx = dmaengine_prep_slave_sg(mcspi_dma->dma_tx, xfer->tx_sg.sgl,
    -				     xfer->tx_sg.nents,
    -				     DMA_MEM_TO_DEV,
    -				     DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
    +					xfer->tx_sg.nents, DMA_MEM_TO_DEV, DMA_CTRL_ACK);
    +
     	if (tx) {
    -		tx->callback = omap2_mcspi_tx_callback;
    -		tx->callback_param = spi;
     		dmaengine_submit(tx);
     	} else {
     		/* FIXME: fall back to PIO? */
    @@ -446,6 +415,9 @@ omap2_mcspi_rx_dma(struct spi_device *spi, struct spi_transfer *xfer,
     	struct omap2_mcspi_cs	*cs = spi->controller_state;
     	void __iomem		*chstat_reg = cs->base + OMAP2_MCSPI_CHSTAT0;
     	struct dma_async_tx_descriptor *tx;
    +	dma_cookie_t dma_rx_cookie = 0;
    +	struct dma_tx_state mcspi_dma_rxstate;
    +	enum dma_status dma_status;
     
     	mcspi = spi_controller_get_devdata(spi->controller);
     	mcspi_dma = &mcspi->dma_channels[spi_get_chipselect(spi, 0)];
    @@ -502,12 +474,10 @@ omap2_mcspi_rx_dma(struct spi_device *spi, struct spi_transfer *xfer,
     	}
     
     	tx = dmaengine_prep_slave_sg(mcspi_dma->dma_rx, sg_out[0],
    -				     out_mapped_nents[0], DMA_DEV_TO_MEM,
    -				     DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
    +				     out_mapped_nents[0], DMA_DEV_TO_MEM, DMA_CTRL_ACK);
    +
     	if (tx) {
    -		tx->callback = omap2_mcspi_rx_callback;
    -		tx->callback_param = spi;
    -		dmaengine_submit(tx);
    +		dma_rx_cookie = dmaengine_submit(tx);
     	} else {
     		/* FIXME: fall back to PIO? */
     	}
    @@ -515,10 +485,20 @@ omap2_mcspi_rx_dma(struct spi_device *spi, struct spi_transfer *xfer,
     	dma_async_issue_pending(mcspi_dma->dma_rx);
     	omap2_mcspi_set_dma_req(spi, 1, 1);
     
    -	ret = mcspi_wait_for_completion(mcspi, &mcspi_dma->dma_rx_completion);
    +	ret = mcspi_wait_for_completion(mcspi, &mcspi->txrxdone);
    +
    +	/*
    +	 * Before disabling RX DMA we need to confirm whether DMA RX is complete.
    +	 * This polling completes on the first attempt itself in most cases.
    +	 */
    +	do {
    +		dma_status = dmaengine_tx_status(mcspi_dma->dma_rx, dma_rx_cookie,
    +						 &mcspi_dma_rxstate);
    +	} while (dma_status != DMA_COMPLETE);
    +
    +	omap2_mcspi_set_dma_req(spi, 1, 0);
     	if (ret || mcspi->target_aborted) {
     		dmaengine_terminate_sync(mcspi_dma->dma_rx);
    -		omap2_mcspi_set_dma_req(spi, 1, 0);
     		return 0;
     	}
     
    @@ -581,7 +561,6 @@ omap2_mcspi_txrx_dma(struct spi_device *spi, struct spi_transfer *xfer)
     {
     	struct omap2_mcspi	*mcspi;
     	struct omap2_mcspi_cs	*cs = spi->controller_state;
    -	struct omap2_mcspi_dma  *mcspi_dma;
     	unsigned int		count;
     	u8			*rx;
     	const u8		*tx;
    @@ -589,11 +568,10 @@ omap2_mcspi_txrx_dma(struct spi_device *spi, struct spi_transfer *xfer)
     	enum dma_slave_buswidth width;
     	unsigned es;
     	void __iomem		*chstat_reg;
    -	void __iomem            *irqstat_reg;
     	int			wait_res;
    +	int ret;
     
     	mcspi = spi_controller_get_devdata(spi->controller);
    -	mcspi_dma = &mcspi->dma_channels[spi_get_chipselect(spi, 0)];
     
     	if (cs->word_len <= 8) {
     		width = DMA_SLAVE_BUSWIDTH_1_BYTE;
    @@ -620,68 +598,36 @@ omap2_mcspi_txrx_dma(struct spi_device *spi, struct spi_transfer *xfer)
     	tx = xfer->tx_buf;
     
     	mcspi->target_aborted = false;
    -	reinit_completion(&mcspi_dma->dma_tx_completion);
    -	reinit_completion(&mcspi_dma->dma_rx_completion);
    -	reinit_completion(&mcspi->txdone);
    -	if (tx) {
    -		/* Enable EOW IRQ to know end of tx in target mode */
    -		if (spi_controller_is_target(spi->controller))
    -			mcspi_write_reg(spi->controller,
    -					OMAP2_MCSPI_IRQENABLE,
    -					OMAP2_MCSPI_IRQSTATUS_EOW);
    +	reinit_completion(&mcspi->txrxdone);
    +	mcspi_write_reg(spi->controller, OMAP2_MCSPI_IRQENABLE,	OMAP2_MCSPI_IRQSTATUS_EOW);
    +	if (tx)
     		omap2_mcspi_tx_dma(spi, xfer, cfg);
    -	}
     
    -	if (rx != NULL)
    +	if (rx)
     		count = omap2_mcspi_rx_dma(spi, xfer, cfg, es);
     
    -	if (tx != NULL) {
    -		int ret;
    -
    -		ret = mcspi_wait_for_completion(mcspi, &mcspi_dma->dma_tx_completion);
    -		if (ret || mcspi->target_aborted) {
    -			dmaengine_terminate_sync(mcspi_dma->dma_tx);
    -			omap2_mcspi_set_dma_req(spi, 0, 0);
    -			return 0;
    -		}
    -
    -		if (spi_controller_is_target(mcspi->ctlr)) {
    -			ret = mcspi_wait_for_completion(mcspi, &mcspi->txdone);
    -			if (ret || mcspi->target_aborted)
    -				return 0;
    -		}
    +	ret = mcspi_wait_for_completion(mcspi, &mcspi->txrxdone);
    +	omap2_mcspi_set_dma_req(spi, 0, 0);
    +	if (ret || mcspi->target_aborted)
    +		return 0;
     
    +	/* for TX_ONLY mode, be sure all words have shifted out */
    +	if (tx && !rx) {
    +		chstat_reg = cs->base + OMAP2_MCSPI_CHSTAT0;
     		if (mcspi->fifo_depth > 0) {
    -			irqstat_reg = mcspi->base + OMAP2_MCSPI_IRQSTATUS;
    -
    -			if (mcspi_wait_for_reg_bit(irqstat_reg,
    -						OMAP2_MCSPI_IRQSTATUS_EOW) < 0)
    -				dev_err(&spi->dev, "EOW timed out\n");
    -
    -			mcspi_write_reg(mcspi->ctlr, OMAP2_MCSPI_IRQSTATUS,
    -					OMAP2_MCSPI_IRQSTATUS_EOW);
    -		}
    -
    -		/* for TX_ONLY mode, be sure all words have shifted out */
    -		if (rx == NULL) {
    -			chstat_reg = cs->base + OMAP2_MCSPI_CHSTAT0;
    -			if (mcspi->fifo_depth > 0) {
    -				wait_res = mcspi_wait_for_reg_bit(chstat_reg,
    -						OMAP2_MCSPI_CHSTAT_TXFFE);
    -				if (wait_res < 0)
    -					dev_err(&spi->dev, "TXFFE timed out\n");
    -			} else {
    -				wait_res = mcspi_wait_for_reg_bit(chstat_reg,
    -						OMAP2_MCSPI_CHSTAT_TXS);
    -				if (wait_res < 0)
    -					dev_err(&spi->dev, "TXS timed out\n");
    -			}
    -			if (wait_res >= 0 &&
    -				(mcspi_wait_for_reg_bit(chstat_reg,
    -					OMAP2_MCSPI_CHSTAT_EOT) < 0))
    -				dev_err(&spi->dev, "EOT timed out\n");
    +			wait_res = mcspi_wait_for_reg_bit(chstat_reg, OMAP2_MCSPI_CHSTAT_TXFFE);
    +			if (wait_res < 0)
    +				dev_err(&spi->dev, "TXFFE timed out\n");
    +		} else {
    +			wait_res = mcspi_wait_for_reg_bit(chstat_reg, OMAP2_MCSPI_CHSTAT_TXS);
    +			if (wait_res < 0)
    +				dev_err(&spi->dev, "TXS timed out\n");
     		}
    +		if (wait_res >= 0 && (mcspi_wait_for_reg_bit(chstat_reg,
    +							     OMAP2_MCSPI_CHSTAT_EOT) < 0))
    +			dev_err(&spi->dev, "EOT timed out\n");
     	}
    +
     	return count;
     }
     
    @@ -1013,9 +959,6 @@ static int omap2_mcspi_request_dma(struct omap2_mcspi *mcspi,
     		mcspi_dma->dma_rx = NULL;
     	}
     
    -	init_completion(&mcspi_dma->dma_rx_completion);
    -	init_completion(&mcspi_dma->dma_tx_completion);
    -
     no_dma:
     	return ret;
     }
    @@ -1105,8 +1048,10 @@ static irqreturn_t omap2_mcspi_irq_handler(int irq, void *data)
     
     	/* Disable IRQ and wakeup target xfer task */
     	mcspi_write_reg(mcspi->ctlr, OMAP2_MCSPI_IRQENABLE, 0);
    -	if (irqstat & OMAP2_MCSPI_IRQSTATUS_EOW)
    -		complete(&mcspi->txdone);
    +	if (irqstat & OMAP2_MCSPI_IRQSTATUS_EOW) {
    +		complete_all(&mcspi->txrxdone);
    +		mcspi_write_reg(mcspi->ctlr, OMAP2_MCSPI_IRQSTATUS, OMAP2_MCSPI_IRQSTATUS_EOW);
    +	}
     
     	return IRQ_HANDLED;
     }
    @@ -1114,12 +1059,9 @@ static irqreturn_t omap2_mcspi_irq_handler(int irq, void *data)
     static int omap2_mcspi_target_abort(struct spi_controller *ctlr)
     {
     	struct omap2_mcspi *mcspi = spi_controller_get_devdata(ctlr);
    -	struct omap2_mcspi_dma *mcspi_dma = mcspi->dma_channels;
     
     	mcspi->target_aborted = true;
    -	complete(&mcspi_dma->dma_rx_completion);
    -	complete(&mcspi_dma->dma_tx_completion);
    -	complete(&mcspi->txdone);
    +	complete_all(&mcspi->txrxdone);
     
     	return 0;
     }
    @@ -1511,7 +1453,7 @@ static int omap2_mcspi_probe(struct platform_device *pdev)
     	status = platform_get_irq(pdev, 0);
     	if (status < 0)
     		goto free_ctlr;
    -	init_completion(&mcspi->txdone);
    +	init_completion(&mcspi->txrxdone);
     	status = devm_request_irq(&pdev->dev, status,
     				  omap2_mcspi_irq_handler, 0, pdev->name,
     				  mcspi);
    -- 
    2.44.1