• State Resolved
  • Locked Locked
  • Replies 16 replies
  • Answers 1 answer
  • Subscribers 105 subscribers
  • Views 1409 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

J721EXSOMXEVM: SerDes Configuration for PCIe

Satish Lal Das2
Expert 1900 points
Part Number: J721EXSOMXEVM

Hello,

Based on our HW schematic, SerDes ports are shared between SGMII and PCIe. Since CPSW9g on Linux doesn't support SGMII connection we are planning to freeup/disable SerDes ports used for SGMII connection from Linux DTS file and enable it in  Ethernet firmware on MCU2_1. However, on Linux side, we want to retain SerDes's PCIe connection.

I have a few questions on PCIe mapping - 

1) How to disable SerDes Port 0, 1 & 4 from dts file and enable port 2 & 3 for PCIe2 & PCIe3 in Linux side?

2) How to enable "&pcie3_rc" in dts file?

These are the changes I made in my dts file but I don't see any pcie logs from "dmesg" , Also "lspci" doesn't show any bus.

Subject: [PATCH] reconfigure SerDes ports

---
 .../dts/mbrdna/k3-j721e-common-proc-board.dts | 41 +++++++++++--------
 1 file changed, 25 insertions(+), 16 deletions(-)

diff --git a/arch/arm64/boot/dts/mbrdna/k3-j721e-common-proc-board.dts b/arch/arm64/boot/dts/mbrdna/k3-j721e-common-proc-board.dts
index e2087b8cfcb9..9442011a7920 100644
--- a/arch/arm64/boot/dts/mbrdna/k3-j721e-common-proc-board.dts
+++ b/arch/arm64/boot/dts/mbrdna/k3-j721e-common-proc-board.dts
@@ -965,6 +965,7 @@
 		#phy-cells = <0>;
 		cdns,phy-type = <PHY_TYPE_PCIE>;
 		resets = <&serdes_wiz0 1>;
+		status = "disabled";
 	};
 };
 
@@ -975,6 +976,7 @@
 		#phy-cells = <0>;
 		cdns,phy-type = <PHY_TYPE_PCIE>;
 		resets = <&serdes_wiz1 1>, <&serdes_wiz1 2>;
+		status = "disabled";
 	};
 };
 
@@ -993,6 +995,7 @@
 	phys = <&serdes0_pcie_link>;
 	phy-names = "pcie_phy";
 	num-lanes = <1>;
+	status = "disabled";
 };
 
 &pcie1_rc {
@@ -1000,6 +1003,7 @@
 	phys = <&serdes1_pcie_link>;
 	phy-names = "pcie_phy";
 	num-lanes = <2>;
+	status = "disabled";
 };
 
 &pcie2_rc {
@@ -1038,33 +1042,38 @@
 	status = "disabled";
 };
 
-&usb_serdes_mux {
-	idle-states = <1>, <0>; /* USB0 to SERDES3, USB1 to SERDES1 */
-};
-
 &serdes_ln_ctrl {
-	idle-states = <SERDES0_LANE0_PCIE0_LANE0>, <SERDES0_LANE1_PCIE0_LANE1>,
-		      <SERDES1_LANE0_PCIE1_LANE0>, <SERDES1_LANE1_PCIE1_LANE1>,
-		      <SERDES2_LANE0_PCIE2_LANE0>, <SERDES2_LANE1_PCIE2_LANE1>,
-		      <SERDES3_LANE0_USB3_0_SWAP>, <SERDES3_LANE1_USB3_0>,
-		      <SERDES4_LANE0_EDP_LANE0>, <SERDES4_LANE1_EDP_LANE1>, <SERDES4_LANE2_EDP_LANE2>, <SERDES4_LANE3_EDP_LANE3>;
-};
-
-&serdes_wiz3 {
-	typec-dir-gpios = <&main_gpio1 3 GPIO_ACTIVE_HIGH>;
-	typec-dir-debounce-ms = <700>;	/* TUSB321, tCCB_DEFAULT 133 ms */
+	idle-states = <SERDES2_LANE0_PCIE2_LANE0>, <SERDES2_LANE1_PCIE2_LANE1>,
+		      <SERDES3_LANE0_PCIE3_LANE0>, <SERDES3_LANE1_PCIE3_LANE1>;
 };
 
 &serdes3 {
-	serdes3_usb_link: link@0 {
+	serdes3_pcie_link: link@0 {
 		reg = <0>;
 		cdns,num-lanes = <2>;
 		#phy-cells = <0>;
-		cdns,phy-type = <PHY_TYPE_USB3>;
+		cdns,phy-type = <PHY_TYPE_PCIE>;
 		resets = <&serdes_wiz3 1>, <&serdes_wiz3 2>;
 	};
 };
 
+&serdes4 {
+	status = "disabled";
+};
+
+&serdes_wiz0 {
+	status = "disabled";
+};
+
+&serdes_wiz1 {
+	status = "disabled";
+};
+
+&serdes_wiz4 {
+	status = "disabled";
+};
+
+
 &usbss0 {
 	ti,vbus-divider;
 	ti,usb2-only;
-- 
2.17.1

I have attached my DTS file just for your reference.

0880.k3-j721e-common-proc-board.dts.txt
Fullscreen
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2019 Texas Instruments Incorporated - http://www.ti.com/
*/
/dts-v1/;
#include "k3-j721e-som-p0.dtsi"
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/input/input.h>
#include <dt-bindings/sound/ti-mcasp.h>
#include <dt-bindings/net/ti-dp83867.h>
/ {
chosen {
stdout-path = "serial9:115200n8";
bootargs = "console=ttyS9,115200n8 earlycon=ns16550a,mmio32,0x02870000";
};
gpio_keys: gpio-keys {
compatible = "gpio-keys";
autorepeat;
pinctrl-names = "default";
pinctrl-0 = <&sw10_button_pins_default &sw11_button_pins_default>;
sw10: sw10 {
label = "GPIO Key USER1";
linux,code = <BTN_0>;
gpios = <&main_gpio0 0 GPIO_ACTIVE_LOW>;
};
sw11: sw11 {
label = "GPIO Key USER2";
linux,code = <BTN_1>;
gpios = <&wkup_gpio0 7 GPIO_ACTIVE_LOW>;
};
};
evm_12v0: fixedregulator-evm12v0 {
/* main supply */
compatible = "regulator-fixed";
regulator-name = "evm_12v0";
regulator-min-microvolt = <12000000>;
regulator-max-microvolt = <12000000>;
regulator-always-on;
regulator-boot-on;
};
vsys_3v3: fixedregulator-vsys3v3 {
/* Output of LMS140 */
compatible = "regulator-fixed";
regulator-name = "vsys_3v3";
regulator-min-microvolt = <3300000>;
regulator-max-microvolt = <3300000>;
vin-supply = <&evm_12v0>;
regulator-always-on;
regulator-boot-on;
};
vsys_5v0: fixedregulator-vsys5v0 {
/* Output of LM5140 */
compatible = "regulator-fixed";
regulator-name = "vsys_5v0";
regulator-min-microvolt = <5000000>;
regulator-max-microvolt = <5000000>;
vin-supply = <&evm_12v0>;
regulator-always-on;
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

We are using SDK 7.1. Could you please help me to enable PCIe bus on our board?

Update on Feb 3. 20201:  I forgot to mention but these changes were also added in dts file along with the aforementioned dts changes - 

Subject: [PATCH] turn off SerDes ports

---
 arch/arm64/boot/dts/mbrdna/k3-j721e-main.dtsi | 16 +++++-----------
 1 file changed, 5 insertions(+), 11 deletions(-)

diff --git a/arch/arm64/boot/dts/mbrdna/k3-j721e-main.dtsi b/arch/arm64/boot/dts/mbrdna/k3-j721e-main.dtsi
index 8675a4887cab..23a79eb3ccff 100644
--- a/arch/arm64/boot/dts/mbrdna/k3-j721e-main.dtsi
+++ b/arch/arm64/boot/dts/mbrdna/k3-j721e-main.dtsi
@@ -53,17 +53,11 @@
 		serdes_ln_ctrl: serdes_ln_ctrl@4080 {
 			compatible = "mmio-mux";
 			#mux-control-cells = <1>;
-			mux-reg-masks = <0x4080 0x3>, <0x4084 0x3>, /* SERDES0 lane0/1 select */
-					<0x4090 0x3>, <0x4094 0x3>, /* SERDES1 lane0/1 select */
-					<0x40a0 0x3>, <0x40a4 0x3>, /* SERDES2 lane0/1 select */
-					<0x40b0 0x3>, <0x40b4 0x3>, /* SERDES3 lane0/1 select */
-					<0x40c0 0x3>, <0x40c4 0x3>, <0x40c8 0x3>, <0x40cc 0x3>;
-					/* SERDES4 lane0/1/2/3 select */
-			idle-states = <SERDES0_LANE0_PCIE0_LANE0>, <SERDES0_LANE1_PCIE0_LANE1>,
-				      <SERDES1_LANE0_PCIE1_LANE0>, <SERDES1_LANE1_PCIE1_LANE1>,
-				      <SERDES2_LANE0_PCIE2_LANE0>, <SERDES2_LANE1_PCIE2_LANE1>,
-				      <MUX_IDLE_AS_IS>, <SERDES3_LANE1_USB3_0>,
-				      <SERDES4_LANE0_EDP_LANE0>, <SERDES4_LANE1_EDP_LANE1>, <SERDES4_LANE2_EDP_LANE2>, <SERDES4_LANE3_EDP_LANE3>;
+			mux-reg-masks = <0x40a0 0x3>, <0x40a4 0x3>, /* SERDES2 lane0/1 select */
+					<0x40b0 0x3>, <0x40b4 0x3>; /* SERDES3 lane0/1 select */
+
+			idle-states = <SERDES2_LANE0_PCIE2_LANE0>, <SERDES2_LANE1_PCIE2_LANE1>,
+				      <SERDES3_LANE0_PCIE3_LANE0>, <SERDES3_LANE1_PCIE3_LANE1>;
 		};
 
 		usb_serdes_mux: mux-controller@4000 {
-- 
2.17.1

Thank you,

Satish

  • 0
    TI__Mastermind 43751 points

    Satish, 

    Per our discussion just now and one of apps team members: 

    Regarding the disabling serdes from Linux, you can take reference from pdf attached in https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/latest/exports/docs/psdk_rtos/docs/user_guide/developer_notes_ethfw.html#enable-8-port-ethernet-with-linux-in-j721e-evm

     Here we disable serdes 0 from the Linux so you can do similar for all needed serdes’s.

    John

  • 0 in reply to John H. Gardner
    Expert 1900 points

    Hi John,

    Thank you for posting the link here. This is the very same document Arun shared with us around two days back. After reading those documents (or the link you shared) I raised the question here. 

    It would be good if we stick to my first question.

    Thank you,

    Satish

  • 0 in reply to Satish Lal Das2
    TI__Genius 9035 points

    Satish,

    Was PCIe working in your platform before you made the changes to disable certain dts nodes? Or is PCIe being enabled for the first time?

    The dts changes to free up the SERDES instances that are going to be used by EthFw look correct.

  • 0 in reply to Misael Lopez Cruz
    Expert 1900 points

    Hi Misael,

    Thank you for confirming on freeing up SerDes Lanes from dts file.

    We are in board bring up phase and this is the first time we are trying to get PCIe working on our board.

    Thank you,

    Satish

  • 0 in reply to Satish Lal Das2
    TI__Mastermind 37435 points

    Hi Satish,

    The following snippet in arch/arm64/boot/dts/ti/k3-j721e-main.dtsi

                    serdes_ln_ctrl: serdes_ln_ctrl@4080 {
                            compatible = "mmio-mux";
                            #mux-control-cells = <1>;
                            mux-reg-masks = <0x4080 0x3>, <0x4084 0x3>, /* SERDES0 lane0/1 select */
                                            <0x4090 0x3>, <0x4094 0x3>, /* SERDES1 lane0/1 select */
                                            <0x40a0 0x3>, <0x40a4 0x3>, /* SERDES2 lane0/1 select */
                                            <0x40b0 0x3>, <0x40b4 0x3>, /* SERDES3 lane0/1 select */
                                            <0x40c0 0x3>, <0x40c4 0x3>, <0x40c8 0x3>, <0x40cc 0x3>;
                                            /* SERDES4 lane0/1/2/3 select */
                            idle-states = <SERDES0_LANE0_PCIE0_LANE0>, <SERDES0_LANE1_PCIE0_LANE1>,
                                          <SERDES1_LANE0_PCIE1_LANE0>, <SERDES1_LANE1_PCIE1_LANE1>,
                                          <SERDES2_LANE0_PCIE2_LANE0>, <SERDES2_LANE1_PCIE2_LANE1>,
                                          <MUX_IDLE_AS_IS>, <SERDES3_LANE1_USB3_0>,
                                          <SERDES4_LANE0_EDP_LANE0>, <SERDES4_LANE1_EDP_LANE1>, <SERDES4_LANE2_EDP_LANE2>, <SERDES4_LANE3_EDP_LANE3>;
                    };

    And the following snippet in arch/arm64/boot/dts/ti/k3-j721e-common-proc-board.dts is responsible for configuring the SERDES lane function.

    &serdes_ln_ctrl {
            idle-states = <SERDES0_LANE0_PCIE0_LANE0>, <SERDES0_LANE1_PCIE0_LANE1>,
                          <SERDES1_LANE0_PCIE1_LANE0>, <SERDES1_LANE1_PCIE1_LANE1>,
                          <SERDES2_LANE0_PCIE2_LANE0>, <SERDES2_LANE1_PCIE2_LANE1>,
                          <SERDES3_LANE0_USB3_0_SWAP>, <SERDES3_LANE1_USB3_0>,
                          <SERDES4_LANE0_EDP_LANE0>, <SERDES4_LANE1_EDP_LANE1>, <SERDES4_LANE2_EDP_LANE2>, <SERDES4_LANE3_EDP_LANE3>;
    };

     

    So if you don't want Linux to configure the SERDES lanes 0 and 1, it should be removed in both k3-j721e-main.dts and k3-j721e-common-proc-board.dts. (remove it both in mux-reg-masks and idle-states)

    pcie2_rc is already configured our EVM, so the same configuration can be used for customer platform.

    pcie3 should be enabled (similar to pcie2_rc)

    &pcie2_rc {
        reset-gpios = <>;
        phys = <&serdes3_pcie_link>;
        phy-names = "pcie_phy";
        num-lanes = <2>;
    };

    Regards

    Vineet

  • 0 in reply to Vineet Roy
    Expert 1900 points

    Hi Vineet,

    I was able to free up SerDes Port 0,1 & 4 from A72 w/ Linux. Could you please help me to enable PCIe lanes by probing SerDes Port 2 & 3? I would appreciate some help on PCIe.

    Thanks,

    Satish

  • 0 in reply to Satish Lal Das2
    TI__Intellectual 1385 points

    Something like below  

     serdes_ln_ctrl: serdes_ln_ctrl@4080 {
                            compatible = "mmio-mux";
                            #mux-control-cells = <1>;
                            mux-reg-masks = <0x40a0 0x3>, <0x40a4 0x3>, /* SERDES2 lane0/1 select */
                                            <0x40b0 0x3>, <0x40b4 0x3>; /* SERDES3 lane0/1 select */
                    };

    &serdes_ln_ctrl {
            idle-states = <SERDES2_LANE0_PCIE2_LANE0>, <SERDES2_LANE1_PCIE2_LANE1>,
                          <SERDES3_LANE0_USB3_0_SWAP>, <SERDES3_LANE1_USB3_0>;
    };

    &serdes2 {
    serdes2_pcie_link: link@0 {
    reg = <0>;
    cdns,num-lanes = <2>;
    #phy-cells = <0>;
    cdns,phy-type = <PHY_TYPE_PCIE>;
    resets = <&serdes_wiz2 1>, <&serdes_wiz2 2>;
    };
    };

    &serdes3 {
    serdes3_pcie_link: link@0 {
    reg = <0>;
    cdns,num-lanes = <2>;
    #phy-cells = <0>;
    cdns,phy-type = <PHY_TYPE_PCIE>;
    resets = <&serdes_wiz3 1>, <&serdes_wiz3 2>;
    };
    };

    &pcie2_rc {
    reset-gpios = <&exp2 20 GPIO_ACTIVE_HIGH>; /* Modify Based on how your board is connected */
    phys = <&serdes2_pcie_link>;
    phy-names = "pcie_phy";
    num-lanes = <2>;
    };

    &pcie3_rc {
        reset-gpios = <>; /* Based on how your board is connected */
        phys = <&serdes3_pcie_link>;
        phy-names = "pcie_phy";
        num-lanes = <2>;
    };

  • 0 in reply to KISHON VIJAY ABRAHAM
    TI__Intellectual 1385 points

    Sorry, fixed the idle-states for PCIe

    &serdes_ln_ctrl {
            idle-states = <SERDES2_LANE0_PCIE2_LANE0>, <SERDES2_LANE1_PCIE2_LANE1>,
                          <SERDES3_LANE0_PCIE3_LANE0>, <SERDES3_LANE1_PCIE3_LANE1>;
    };

  • 0 in reply to KISHON VIJAY ABRAHAM
    Expert 1900 points

    Hello Kishon,

    Thank you for those commands. I made those changes LInux dts file. I also changed the "reset-gpio" according to our board i.e. GPIO_9 for PCIe2 & GPIO_18 for PCIe3.

    I still don't see any PCIe Lanes or any Kernel logs for PCIe. 

    In k3-j721e.dtsi, there is two-row for PCIe Core. Do you think this table needs an update to accommodate PCIe2 & PCIe3 registers?

    cbass_main: interconnect@100000 {
        compatible = "simple-bus";
        #address-cells = <2>;
        size-cells = <2>;
        ranges = <0x00 0x00100000 0x00 0x00100000 0x00 0x00020000>, /* ctrl mmr */
                       <0x00 0x00600000 0x00 0x00600000 0x00 0x00031100>, /* GPIO */
                       <0x00 0x00900000 0x00 0x00900000 0x00 0x00012000>, /* serdes */
                       <0x00 0x00A40000 0x00 0x00A40000 0x00 0x00000800>, /* timesync router */
                       <0x00 0x06000000 0x00 0x06000000 0x00 0x00400000>, /* USBSS0 */
                       <0x00 0x06400000 0x00 0x06400000 0x00 0x00400000>, /* USBSS1 */
                       <0x00 0x01000000 0x00 0x01000000 0x00 0x0af02400>, /* Most peripherals */
                       <0x00 0x30000000 0x00 0x30000000 0x00 0x0c400000>, /* MAIN NAVSS */
                       <0x00 0x0d000000 0x00 0x0d000000 0x00 0x01800000>, /* PCIe Core*/
                       <0x00 0x0e000000 0x00 0x0e000000 0x00 0x01800000>, /* PCIe Core*/
                       <0x00 0x10000000 0x00 0x10000000 0x00 0x10000000>, /* PCIe DAT */
                       <0x00 0x64800000 0x00 0x64800000 0x00 0x00800000>, /* C71 */
                       <0x44 0x00000000 0x44 0x00000000 0x00 0x08000000>, /* PCIe2 DAT */
                       <0x44 0x10000000 0x44 0x10000000 0x00 0x08000000>, /* PCIe3 DAT */
                       <0x4d 0x80800000 0x4d 0x80800000 0x00 0x00800000>, /* C66_0 */
                       <0x4d 0x81800000 0x4d 0x81800000 0x00 0x00800000>, /* C66_1 */
                       <0x4e 0x20000000 0x4e 0x20000000 0x00 0x00080000>, /* GPU */
                       <0x00 0x70000000 0x00 0x70000000 0x00 0x00800000>, /* MSMC RAM */

    Thanks,

    Satish

  • 0 in reply to Satish Lal Das2
    Expert 1900 points

    Here is the Boot log just in case if you want to take a look.

    kernellogpcie.txt
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    root@nova-slot5:~#
    root@nova-slot5:~# Flash: 0 Bytes
    MMC: sdhci@4f80000: 0, sdhci@4fb0000: 1
    Loading Environment from MMC... OK
    In: serial@2870000
    Out: serial@2870000
    Err: serial@2870000
    Reading on-board EEPROM at 0x50 failed 1
    Net: can't parse phy-handle port 1 (-2)
    can't find cpsw-phy-sel
    No ethernet found.
    Hit any key to stop autoboot: 0
    switch to partitions #0, OK
    mmc0(part 0) is current device
    SD/MMC found on device 0
    ** Unrecognized filesystem type **
    16654344 bytes read in 50 ms (317.7 MiB/s)
    93562 bytes read in 2 ms (44.6 MiB/s)
    ## Flattened Device Tree blob at 82000000
    Booting using the fdt blob at 0x82000000
    Loading Device Tree to 00000000fdda0000, end 00000000fdeb9fff ... OK
    Starting kernel ...
    [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x411fd080]
    [ 0.000000] Linux version 5.4.40-g66cf445b76 (oe-user@oe-host) (gcc version 9.2.1 20191025 (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10))) #1 SMP PREEMPT Sat Feb 13 08:09:58 UTC 2021
    [ 0.000000] Machine model: Texas Instruments K3 J721E SoC
    [ 0.000000] earlycon: ns16550a0 at MMIO32 0x0000000002870000 (options '')
    [ 0.000000] printk: bootconsole [ns16550a0] enabled
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a0000000, size 1 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-dma-memory@a0000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a0100000, size 15 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-memory@a0100000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a1000000, size 1 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-dma-memory@a1000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a1100000, size 31 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-memory@a1100000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a3000000, size 1 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-dma-memory@a3000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a3100000, size 15 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-memory@a3100000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a4000000, size 1 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-dma-memory@a4000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a4100000, size 15 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-memory@a4100000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a5000000, size 1 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-dma-memory@a5000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a5100000, size 15 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-memory@a5100000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a6000000, size 1 MiB
    [ 0.000000] OF: reserved mem: initialized node c66-dma-memory@a6000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a6100000, size 15 MiB
    [ 0.000000] OF: reserved mem: initialized node c66-memory@a6100000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a7000000, size 1 MiB
    [ 0.000000] OF: reserved mem: initialized node c66-dma-memory@a7000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a7100000, size 15 MiB
    [ 0.000000] OF: reserved mem: initialized node c66-memory@a7100000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a8000000, size 1 MiB
    [ 0.000000] OF: reserved mem: initialized node c71-dma-memory@a8000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000a8100000, size 15 MiB
    [ 0.000000] OF: reserved mem: initialized node c71-memory@a8100000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000fb000000, size 1 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-dma-memory@fb000000, compatible id shared-dma-pool
    [ 0.000000] Reserved memory: created DMA memory pool at 0x00000000fb100000, size 15 MiB
    [ 0.000000] OF: reserved mem: initialized node r5f-memory@fb100000, compatible id shared-dma-pool
    [ 0.000000] cma: Reserved 512 MiB at 0x00000000c0000000
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

  • 0 in reply to Satish Lal Das2
    TI__Intellectual 1385 points

    Hi,

    If the following two lines are present, it means pcie2 and pcie3 are already included.

    <0x00 0x0d000000 0x00 0x0d000000 0x00 0x01800000>, /* PCIe Core*/
    <0x00 0x0e000000 0x00 0x0e000000 0x00 0x01800000>, /* PCIe Core*/

     

    Can you check if the DTB file is properly updated

    cat /proc/device-tree/bus@100000/pcie@2930000/status

  • 0 in reply to KISHON VIJAY ABRAHAM
    Expert 1900 points

    Hi Kishon,

    In my case 7.1 was not working so I moved to SDK 7.0. In my case I don't see "status" option on my board. 

    Here is the options available in my case.

    root@nova-slot5:~# cat /proc/device-tree/interconnect@100000/pcie@2930000/                                                                                                                                         
#address-cells               cdns,max-outbound-regions    compatible                   interrupt-map-mask           name                         phys                         reg-names
#interrupt-cells             cdns,no-bar-match-nbits      device-id                    legacy-interrupt-controller/ num-lanes                    power-domains                reset-gpios
#size-cells                  clock-names                  dma-coherent                 max-link-speed               phandle                      ranges                       ti,syscon-pcie-ctrl
bus-range                    clocks                       interrupt-map                msi-map                      phy-names                    reg                          vendor-id

    Thanks,

    Satish

  • 0 in reply to Satish Lal Das2
    TI__Mastermind 23125 points

    Satish, 

    I checked the files you sent offline, and confirming that:

    1. your linux kernel config is almost identify to the sdk. so it should have PCIe enabled.

    2. your dts files looks fine for PCIe2 and PCIe3 (except for reset pin as we discussed)

    3. your bootlog IS missing any driver messages. Since the IO Expander hardware does not exist on your hardware, we would at least expect some I2C error message when the driver attempt to write to the I2C address. So along the same path Kishon asked above, can we check:

    a. do you also see similar files under /proc for PCIe2, as you also enabled PCIe port?

         cat /proc/device-tree/interconnect@100000/pcie@2920000/

        On my EVM I can see the status file. 

     b. as a sanity check,  can you replace the dtb with an older version or even the sdk stock version to see any difference in kernel log and content of /proc/device-tree/interconnect@100000/? As discussed, we expect the stock sdk to work with your hardware, as soon as the reset is fixed. The stock SDK is using internal refclk on PCIe to supply the PCIe on the SOC side, but not giving out refclk to the slot. 

    regards

    jian

  • 0 in reply to jian35385
    Expert 1900 points

    Hi Jian,

    I added the following code in dts file for PCIe2

    	reset-gpios = <&main_gpio0 9 GPIO_ACTIVE_HIGH>;

    With this change, I do see some PCIe logs from dmesg and lspci. Here is the logs -

    root@nova-slot5:~# dmesg | grep pci                                                                                                                                                                                
[    5.193772] j721e-pcie 2920000.pcie: host bridge /interconnect@100000/pcie@2920000 ranges:
[    5.202048] j721e-pcie 2920000.pcie:    IO 0x4400001000..0x4400010fff -> 0x00001000
[    5.209695] j721e-pcie 2920000.pcie:   MEM 0x4400011000..0x4407ffffff -> 0x00011000
[    5.217412] j721e-pcie 2920000.pcie: PCI host bridge to bus 0000:00
[    5.223667] pci_bus 0000:00: root bus resource [bus 00-0f]
[    5.229139] pci_bus 0000:00: root bus resource [io  0x0000-0xffff] (bus address [0x1000-0x10fff])
[    5.237991] pci_bus 0000:00: root bus resource [mem 0x4400011000-0x4407ffffff] (bus address [0x00011000-0x07ffffff])
[    5.248503] pci 0000:00:00.0: [104c:b00d] type 01 class 0x060400
[    5.254505] pci_bus 0000:00: 2-byte config write to 0000:00:00.0 offset 0x4 may corrupt adjacent RW1C bits
[    5.264138] pci_bus 0000:00: 2-byte config write to 0000:00:00.0 offset 0x4 may corrupt adjacent RW1C bits
[    5.273788] pci_bus 0000:00: 2-byte config write to 0000:00:00.0 offset 0xe8 may corrupt adjacent RW1C bits
[    5.283508] pci_bus 0000:00: 2-byte config write to 0000:00:00.0 offset 0x3e may corrupt adjacent RW1C bits
[    5.293236] pci_bus 0000:00: 2-byte config write to 0000:00:00.0 offset 0x92 may corrupt adjacent RW1C bits
[    5.302960] pci_bus 0000:00: 2-byte config write to 0000:00:00.0 offset 0xb2 may corrupt adjacent RW1C bits
[    5.312704] pci 0000:00:00.0: supports D1
[    5.316702] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    5.322434] pci_bus 0000:00: 2-byte config write to 0000:00:00.0 offset 0x84 may corrupt adjacent RW1C bits
[    5.333805] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    5.341798] pci_bus 0000:00: 2-byte config write to 0000:00:00.0 offset 0x3e may corrupt adjacent RW1C bits
[    5.351517] pci_bus 0000:00: 2-byte config write to 0000:00:00.0 offset 0x3e may corrupt adjacent RW1C bits
[    5.361237] pci_bus 0000:00: 2-byte config write to 0000:00:00.0 offset 0x3e may corrupt adjacent RW1C bits
[    5.372443] pci_bus 0000:01: busn_res: [bus 01-0f] end is updated to 01
[    5.379048] pci 0000:00:00.0: PCI bridge to [bus 01]
[    5.384169] pcieport 0000:00:00.0: PME: Signaling with IRQ 403
[    5.390109] pcieport 0000:00:00.0: AER: enabled with IRQ 403

    The output of lspci is as below -

    root@nova-slot5:~# lspci -vvv
00:00.0 Class 0604: 104c:b00d

    The output of "rescan" is as below -

    root@nova-slot5:~# echo 1 > /sys/bus/pci/rescan
[  606.780990] pci_generic_config_write32: 20 callbacks suppressed
[  606.780997] pci_bus 0000:00: 2-byte config write to 0000:00:00.0 offset 0x3e may corrupt adjacent RW1C bits
[  606.798018] pci_bus 0000:00: 2-byte config write to 0000:00:00.0 offset 0x3e may corrupt adjacent RW1C bits
[  606.807749] pci_bus 0000:00: 2-byte config write to 0000:00:00.0 offset 0x3e may corrupt adjacent RW1C bits
[  606.817478] pci_bus 0000:00: 2-byte config write to 0000:00:00.0 offset 0x3e may corrupt adjacent RW1C bits

    However, I still don't see any "status" file under "cat /proc/device-tree/interconnect@100000/pcie@2920000/"

    Similar to PCIe2 I will change the PCIe3 "reset-gpio" configuration for PCIe3 and send you the log. 

    It looks like it is able to scan the bus. What is your observation from these logs?

    Thanks,

    Satish

  • 0 in reply to Satish Lal Das2
    Expert 1900 points

    Hi Jian,

    I fixed "reset-gpio" field for PCIe3 and now I see both i.e. PCIe2 & PCIe3 is configured as Root complex. Here is the log

    root@nova-slot5:~# dmesg | grep pcie
[    5.194263] j721e-pcie 2920000.pcie: host bridge /interconnect@100000/pcie@2920000 ranges:
[    5.202539] j721e-pcie 2920000.pcie:    IO 0x4400001000..0x4400010fff -> 0x00001000
[    5.210186] j721e-pcie 2920000.pcie:   MEM 0x4400011000..0x4407ffffff -> 0x00011000
[    5.217903] j721e-pcie 2920000.pcie: PCI host bridge to bus 0000:00
[    5.384702] pcieport 0000:00:00.0: PME: Signaling with IRQ 403
[    5.390639] pcieport 0000:00:00.0: AER: enabled with IRQ 403
[    6.397942] j721e-pcie 2930000.pcie: host bridge /interconnect@100000/pcie@2930000 ranges:
[    6.406205] j721e-pcie 2930000.pcie:    IO 0x4410001000..0x4410010fff -> 0x00001000
[    6.413850] j721e-pcie 2930000.pcie:   MEM 0x4410011000..0x4417ffffff -> 0x00011000
[    6.421545] j721e-pcie 2930000.pcie: PCI host bridge to bus 0001:00
[    6.491371] pcieport 0001:00:00.0: PME: Signaling with IRQ 405
[    6.497297] pcieport 0001:00:00.0: AER: enabled with IRQ 405
root@nova-slot5:~# 
root@nova-slot5:~# lspci -vv
00:00.0 Class 0604: 104c:b00d
00:00.0 Class 0604: 104c:b00d

    Now since PCIe bus is up, I am thinking that we should now connect our J7 board to backplane and so the testing. I am fine closing this E2E thread. Let me know if you want me to check something else.

    Thanks,

    Satish

  • 0 in reply to Satish Lal Das2
    TI__Mastermind 23125 points

    Satish,

    Thanks for the updates. Glad PCIe ports are up. 

    I also tried on my board, and I did not see the status in the /proc if I don't have anything plugged in. 

    Kishon thinks the reason that we did not see the driver error was it might be endlessly returning -EPROBE_DEFER when the IO Expander was specified in the DT. 

    Please go ahead plug in a device and give it try. if you run into further issues to bring the EP up, pls file a new ticket. 

    regards

    Jian