This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM623: QSPI Communication Between AM623 and FPGA – Read Operation working but Writing

Part Number: AM623

Tool/software:

Hello,

We are working with a custom board that features an AM623 processor and an FPGA, connected via QSPI.

The AM623 is running Linux Kernel version 6.6.87, obtained from: https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/log/?h=ti-linux-6.6.y-cicd

Within the kernel, I examined TI's low-level OSPI driver located at: linux-kernel/drivers/spi/spi-cadence-quadspi.c

Based on this driver, I developed a higher-level driver for our FPGA. I'm using the spi_mem_exec_op() API from the Linux kernel to send and receive QSPI frames.

Each time I call spi_mem_exec_op(), it invokes a function from spi-cadence-quadspi.c that performs the actual data transfer over QSPI.

According to my tests, I am able to successfully read data from the FPGA in 4-4-4 mode using the following code:

static int fpga_qspi_read(struct spi_mem *mem)
{
    struct spi_mem_op op;
    u8 *rx_buf;
    int ret;

    // Allocate RX buffer
    rx_buf = devm_kzalloc(&mem->spi->dev, 16, GFP_KERNEL);
    if (!rx_buf)
        return -ENOMEM;
    
    op = (struct spi_mem_op)SPI_MEM_OP(
                                       SPI_MEM_OP_CMD(READ_OPCODE, 4),      // Command  2 clock
                                       SPI_MEM_OP_ADDR(3, 0x00AA55, 4),     // 3-byte address 6 clocks                                       
                                       SPI_MEM_OP_NO_DUMMY,                 
                                       SPI_MEM_OP_DATA_IN(2, rx_buf, 4)     // 16-byte read, 4-bit bus 4 clocks
                                       );

    if (!spi_mem_supports_op(mem, &op))
    {
        dev_err(&mem->spi->dev, "Controller does not support this Quad Read op\n");
        return -EOPNOTSUPP;
    }

    ret = spi_mem_exec_op(mem, &op);
    if (ret)
    {
        dev_err(&mem->spi->dev, "Quad read failed: %d\n", ret);
        return ret;
    }

    dev_info(&mem->spi->dev, "Read data:");
    print_hex_dump(KERN_INFO, "FPGA: ", DUMP_PREFIX_OFFSET, 16, 1, rx_buf, 16, true);

    return 0;
}

Below is the oscilloscope output, confirming that the communication works correctly and all signals appear as expected:

I’m also able to read 16 bytes at a time without any failures.

However, the issue arises with writing. Every time I attempt a write operation—whether in 4-4-4, 1-4-4, 1-1-4, or even 1-1-1 mode—it fails, returning an error that the operation is not supported.

This error message originates from the spi-cadence-quadspi.c driver, but I haven’t been able to understand the exact reason behind it.

In the device tree, the OSPI module is bound as follows:

		ospi0: spi@fc40000 {
			compatible = "ti,am654-ospi", "cdns,qspi-nor";
			reg = <0x00 0x0fc40000 0x00 0x100>,
			      <0x05 0x00000000 0x01 0x00000000>;
			interrupts = <GIC_SPI 139 IRQ_TYPE_LEVEL_HIGH>;
			cdns,fifo-depth = <256>;
			cdns,fifo-width = <4>;
			cdns,trigger-address = <0x0>;
			cdns,phase-detect-selector = <2>;
			clocks = <&k3_clks 75 7>;
			assigned-clocks = <&k3_clks 75 7>;
			assigned-clock-parents = <&k3_clks 75 8>;
			assigned-clock-rates = <166666666>;
			power-domains = <&k3_pds 75 TI_SCI_PD_EXCLUSIVE>;
			#address-cells = <1>;
			#size-cells = <0>;
			status = "disabled";
		};

So it matches with the compatible string "ti,am654-ospi" first.

Then on top of it I have our custom driver bound like:

&ospi0 {	
	status = "okay";
	
	fpga@0{
		compatible = "XYZ,fpga-qspi";
		reg = <0x0>;
		spi-tx-bus-width = <4>;
		spi-rx-bus-width = <4>;
		spi-max-frequency = <25000000>;
	};
};

And here is the write test function but fails:

static int fpga_qspi_write(struct spi_mem *mem)
{
    u8 tx_buf[16] = {
        0x00, 0x01, 0x02, 0x03,
        0x04, 0x05, 0x06, 0x07,
        0x08, 0x09, 0x0A, 0x0B,
        0x0C, 0x0D, 0x0E, 0x0F
    };

    int ret;

    struct spi_mem_op op = (struct spi_mem_op)SPI_MEM_OP(
                                        SPI_MEM_OP_CMD(0x55, 4),            // CMD
                                        SPI_MEM_OP_ADDR(3, 0x00AA55, 4),    // 3-byte address (1-bit lines)
                                        SPI_MEM_OP_NO_DUMMY,
                                        SPI_MEM_OP_DATA_OUT(16, tx_buf, 4)   // 16-byte write over 4-bit bus
                                       );

    
    if (!spi_mem_supports_op(mem, &op))
    {
        dev_err(&mem->spi->dev, "spi_mem_supports_op() for writing failed!\n");
        return -EOPNOTSUPP;
    }
    
    ret = spi_mem_exec_op(mem, &op);
    if (ret)
    {
        dev_err(&mem->spi->dev, "spi_mem_exec_op() failed for write operation: %d\n", ret);
        return ret;
    }

    return 0;
}

And finally here my questions:

1. Which transfer modes are supported by spi-cadence-quadspi.c?

2. Why do you think read operations succeed while similar write operations fail?

3. When comparing TI’s spi-cadence-quadspi driver in the TI repository versus the one in the mainline Linux kernel, which repository offers broader feature support? Shall we go with linux main stream instead of TI's downstream?

Thanks.

  • Hi Mehmt,

    However, the issue arises with writing. Every time I attempt a write operation—whether in 4-4-4, 1-4-4, 1-1-4, or even 1-1-1 mode—it fails, returning an error that the operation is not supported.

    In any of the write attempts, do you see any signaling coming out of the device at all (e.g., clock signal) on the scope?

    1. Which transfer modes are supported by spi-cadence-quadspi.c?

    That particular ti-linux-6.6.y kernel version should have the driver that includes the following commit, that specifically adds the SPI_TX_QUAD transfer mode:

    $ git show 7a78c89eb58bad042c55b7b0234300ab9f561919
    commit 7a78c89eb58bad042c55b7b0234300ab9f561919
    Author: Santhosh Kumar K <s-k6@ti.com>
    Date:   Wed Jul 17 11:56:01 2024 +0530
    
        spi: cadence-quadspi: Enable SPI_TX_QUAD
    
        Enable SPI_TX_QUAD in spi_controller->mode_bits to add support for
        the 1S-4S-4S operations.
    
        Signed-off-by: Santhosh Kumar K <s-k6@ti.com>
        Tested-by: Prasanth Babu Mantena <p-mantena@ti.com>
    
    diff --git a/drivers/spi/spi-cadence-quadspi.c b/drivers/spi/spi-cadence-quadspi.c
    index 97069f8d27847..c2f83d2c50988 100644
    --- a/drivers/spi/spi-cadence-quadspi.c
    +++ b/drivers/spi/spi-cadence-quadspi.c
    @@ -2801,7 +2801,7 @@ static int cqspi_probe(struct platform_device *pdev)
                    dev_err(&pdev->dev, "devm_spi_alloc_host failed\n");
                    return -ENOMEM;
            }
    -       host->mode_bits = SPI_RX_QUAD | SPI_RX_DUAL;
    +       host->mode_bits = SPI_RX_QUAD | SPI_TX_QUAD | SPI_RX_DUAL;
            host->mem_ops = &cqspi_mem_ops;
            host->mem_caps = &cqspi_mem_caps;
            host->dev.of_node = pdev->dev.of_node;

    2. Why do you think read operations succeed while similar write operations fail?

    I can't tell; we should do further experimentation. How about swapping your "XYZ,fpga-qspi" slave device for an existing quad-spi device, that is supported by the kernel, just for experimentation purposes with the APIs (and to see if you get data transmitted out). For example (there may be better starting points, but this is one)...

    • Use the `drivers/mtd/spi-nor/core.c` driver as a reference for your custom FPGA driver
    • Use a device tree compatible string for that driver that doesn't trigger the auto-detection. Something like "m25p05-nonjedec" should work (I haven't tried it, I'm just looking at the code)
    • See if you can get your 4-bit wide writes to work that way, just for testing purposes
    • The driver stack may require explicitly configuration for write operations through your FPGA driver. For example the spi_nor_create_write_dirmap() function called from that `drivers/mtd/spi-nor/core.c` driver may be important to mimic in your custom code
    3. When comparing TI’s spi-cadence-quadspi driver in the TI repository versus the one in the mainline Linux kernel, which repository offers broader feature support? Shall we go with linux main stream instead of TI's downstream?

    The driver from the TI tree is usually more feature-complete than the corresponding upstream driver; however over time we strive for those things to converge as much as possible (according to our "upstream first" philosophy)

    I don't think this will make a difference but nevertheless I'd recommend trying our latest ti-linux-6.12.y kernel tree as well just so you have all your bases covered.

    Regards, Andreas

  • Hi Andreas,

    >> In any of the write attempts, do you see any signaling coming out of the device at all (e.g., clock signal) on the scope?

    Nothing appears on the oscilloscope when writing. The issue occurs at the spi_mem_supports_op() API call, which is used just before spi_mem_exec_op().

    If I call spi_mem_exec_op() directly without checking whether the operation is supported, it returns an 'invalid parameters' error—and again, nothing shows up on the scope.

    The commit you mentioned, SPI_TX_QUAD, is present in our kernel version 6.6.87, which we obtained from TI’s downstream branch.

    Interestingly, I checked the latest mainline Linux kernel (version 6.16-rc2 as of today), and that commit does not appear to be included there.

    Based on this small patch, it seems we should stick with the TI downstream branch. Maybe we can try version 6.12 from your downstream, as you suggested.

    In our FPGA, we don’t intend to fully emulate a NOR flash, but rather to implement custom read and write operations in 4-4-4 mode. For example, we can define a custom command 1 to read between 1 and 4096 bytes, and a custom command 2 to write between 1 and 4096 bytes. We don’t need to follow the standard JEDEC commands, but we do adhere to the same frame format (command, address, data) as suggested by JEDEC.

    Will this be supported by the TI driver ( ti,am654-ospi ), or is it strictly limited to standard JEDEC commands? So far, reading appears to be supported, but writing is the main topic of concern in this thread.

    I'll be on holiday next week. Once I'm back, I'll try replacing the FPGA binding in the device tree with m25p05-nonjedec and let you know if I observe any write sequence activity on the scope.

    In the meantime, I have a few questions regarding the device tree OSPI bindings. In your k3-am62-main.dtsi, the OSPI compatible driver is defined as both "ti,am654-ospi", "cdns,qspi-nor". However, in spi-cadence-quadspi.c, both of these strings are listed in the compatibility table, each associated with different quirks. As far as I understand, Linux matches only the first compatible string in the device tree, which would render the second entry irrelevant, right? This matters because the hwcaps_mask used in the probe function is determined based on the quirks defined in those compatibility entries and they are different.

    And later in the probe function SPI_TX_QUAD support added only if hwcaps_mask has CQSPI_SUPPORTS_QUAD which is not the case for "cdns,qspi-nor" but for "ti,am654-ospi"

    Regarding your suggestion to check devm_spi_mem_dirmap_create(), we're not interested in Direct mode, which maps external flash memory into the Linux address space. Instead, we plan to handle read and write operations indirectly—frame by frame—using our custom commands, which are not defined by JEDEC standards, through our driver. I hope this is supported by your lower layer qspi driver.

    Thank you for your support, and I hope to see you next week.

    Best regards,
    MF

  • In the meantime, I have a few questions regarding the device tree OSPI bindings. In your k3-am62-main.dtsi, the OSPI compatible driver is defined as both "ti,am654-ospi", "cdns,qspi-nor". However, in spi-cadence-quadspi.c, both of these strings are listed in the compatibility table, each associated with different quirks. As far as I understand, Linux matches only the first compatible string in the device tree, which would render the second entry irrelevant, right?

    It'll pick the first item from the list in the DTS file and try to match it. If it fails, it'll pick the second. And so on. On the other hand, the order of the entries in the matching table in the driver code doesn't matter here.

    If I call spi_mem_exec_op() directly without checking whether the operation is supported, it returns an 'invalid parameters' error

    Can you try to trace this down where in the Kernel driver that comes from?

    Regards, Andreas

  • Hello Andreas,

    I’ve reviewed the Linux kernel source code and identified a restriction that affects my write test routine: the read or write buffer must not reside on the stack. If it does, the kernel immediately returns EINVAL and does not pass the request to the lower-level driver (spi-cadence-quadspi.c).

    After switching to dynamically allocated memory for the TX buffer instead of using a static array, I started observing write activity on the scope as well.

    This thread can be closed. Thank you for your help.