This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

EVM6657 PCIe enumeration failure

Other Parts Discussed in Thread: TMDXEVMPCI

Our EVM6657 is not enumerating on 2 out of 3 host PC we've tried. I programmed the latest IBL from "mcsdk_2_01_02_06" (with the PLL lock workaround). The host where the board does enumerate only does so when I apply external power to the EVM.

This problem is all over the forum and earlier I have posted questions here:
    http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/273816/1123605.aspx
    http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/216539/1122702.aspx

The "solution" mostly ends up with "we changed host system". Something I am not comfortable with: there must be a good reason enumeration fails.

Here's some information regarding my setup:

I am running in Little Endian mode and I loaded the I2C EEPROM (address 0x51) with the latest IBL (from MCSDK mcsdk_2_01_02_06) with the PCIe PLL lock workaround. When I hookup an emulator I see the following values:

    PC                                                                  = 0x20B010B0 (In the Boot ROM. This is an IDLE instruction)
    DEVSTAT(@0x2620020)                           = 0x00011809
    BOOTMAGIC (0x8FFFFC)                          = 0x00000000
    PCIE_SERDES_STS (@0x0262015C)   = 0x00000001 (so PLL is LOCKed)

I tried first powering up the EVM and then the host in which case the PCIE_SERDES_STS changes from 0x00000209 (LOCK/LOSDTCT0/LOSDTCT1) to 0x00000001 (LOCK).


The only possible explanation I could find is that the x86 host uses spread spectrum clocking (SSC):
    http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/278885/982383.aspx

EDIT: The register SERDES_CFG0 (0x390) is set to 0x00062320, so RX_CDR = 100b = 0x4 = "Second order, low precision, threshold of 1. Best response to changes in frequency offset and fastest lock time, but lowest precision frequency offset matching. Suitable for use in systems with spread spectrum clocking".

So...I believe a host using SSC should not be a problem.

More links to similar issues:

    http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/222043.aspx
    http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/716/t/260354.aspx
    http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/123060/441967.aspx
    http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/196023/699565.aspx

In our final design we are planning to use the ROM boot loader (RBL) to boot over PCIe from a Marvel ARM SoC. However, for debugging we plan on using the x86 host system with an EVM6657 and it is important to get this to work reliably.

Any help is hugely appreciated!

Dirk

  • Here's an update:

    When the Host powers up, I can see the Transmitter being enabled on both lanes and subsequently the Receivers. Right after after that both are disabled Reg 0x388 goes from 0x1300 ==> 0x1330 ==> 0x1333 ==> 0x1300.

    Additionally the following PCIe registers change:

    - 0x1D0 (PMRST_IRQ_STATUS_RAW) gets set from 0x0 ==> 0x8, (LINK_RST_REQ)

    - 0x028 (ACT_STATUS) changes from 0 ==> 0x2 (OB_NOT_EMPTY), like it is trying to send something but fails.

    Also see the attached video:

    Please advice...

    Dirk

  • Dirk,

    Thanks very much for your efforts to debug this. Under the EVM manufacturer website: https://www.einfochips.com/index.php/partnerships/texas-instruments/ti-tms320c6657-evm#5-resources

    Failure to operate PCIe in an ATX chassis when using the TMDXEVMPCI adapter

    All

    production

    version EVMs are not detected when inserted in ATX chassis using TMDXEVMPCI adapter.

    Workaround:

    The standard IBL (created for the C6678 workaround) contains PCIe configuration writes that are needed to enable robust operation in an ATX computer chassis. The FPGA version v02 redirects the C6657 to boot from the IBL even when the DIP switches are programmed for PCIe end-point boot. These PCIe configuration writes will be needed in the customer application to enable C6657 operation in an ATX computer.

    The production EVM has the V3 FPGA, so it will boot from ROM directly and what you programed to IBL will NOT be loaded and run. You need to downgrade the FPGA to V2, it will boot from IBL. The PCIE workaround in IBL should work for PCIE enumeration for ATX PC with SSC.

    Let me know if this works for you.

    Regards, Eric

     

  • Thanks Eric, I'll try that. BTW, the link to this file on the eInfoChips site is not working. I sent a message to eInfoChips with a request to solve the issue.

    Dirk

  • While I wait for the actual FPGA file (if you have it, please post it here!), could you explain a bit more about the actual issue? I found a thread with your posts which describe the following issue.

    http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/203172.aspx?pi267198=1

    "The production 6657 EVM only uses RBL (NO IBL), due to some DSS clock setting issues in PCIE code, the 6657 EVM can’t be enumerated by a Linux or Windows PC using DSS clock. So the 6657 EVM + AMC-PCIE  adaptor card can't work in a PC enviornment unless the PC can be configured to generate narrow spectrum 100 MHz PCIE reference clock.

    Regards, Eric""

    I am rather confused....the older FPGA firmware starts the IBL, just like all the EVM docs state and what I have so far been assuming. You are saying the IBL is no longer used, for any boot mode but I2C. Is this correct? Also, what are these " some DSS clock setting issues in PCIE code" exactly? Which register is affected?

    The thread goes on and I find the following post:

    "The production 6657 EVM boot directly from RBL (Rom boot loader) in PCIE boot mode, there may be some issue in the PCIE serdes setting in RBL code when using with a PC, typically with DSS clock.

    For production 6678 EVM, it boots from IBL (intermediate boot loader) when in PCIE boot mode, the IBL code has some PCIE workaround for this DSS clock issue so 6678 EVM works fine with PC environment.  

    The EVM is powered from PCIe PCB edge finger connector of the TMDXEVMPCI, no external power supply required.

    Regards, Eric"

     

    Again I am confused: this states that the EVM6678 DOES have the I2C IBL boot followed by RBL boot of the selected flavor. So why is the EVM6657 different from the EVM6678? Especially since the EVM6657 does not boot over PCIe when used with the AMC-PCIE adapter?

    I am assuming the DSS clock fix mentioned in the thread above sets the registers "SERDES_CFGx" up differently. However, when looking at the various clock schemes the EVM6657 supports, the one set by the IBL seems to be identical to the one I see set in the PCIe registers with my XDS560. Also, I can see the ROMCode writing them out:

    ========== RBL:

    20b0e676:   400C                LDW.D1T1      *A4[2],A0 <== A0 = 0x22320
    20b0e678:   1507                MV.L2X        A10,B0 <== A10 = 0x21800390
    20b0e67a:   8441                ADD.L2        B0,4,B4
    20b0e67c:   EE600200            .fphead       n, l, W, BU, nobr, nosat, 1110011b
    20b0e680:   2C6E                NOP           2
    20b0e682:   0005                STW.D2T1      A0,*B4[0]
    20b0e684:   027A902A            MVK.S2        0xfffff520,B4
    20b0e688:   0210586A            MVKH.S2       0x20b00000,B4
    20b0e68c:   00100362            B.S2          B4
    20b0e690:   0200AE28            MVK.S1        0x015c,A4
    20b0e694:   01884162            ADDKPC.S2     0x20b0e6a0,B3,2 <== PC when the HW watch triggered.
    20b0e698:   02013168            MVKH.S1       0x2620000,A4

    ========== IBL:
    "C:\ti\mcsdk_2_01_02_06\tools\boot_loader\ibl\src\device\c665x\c665xinit.c"
    void iblPCIeWorkaround()
    {
        UINT32  v, flag_6678 = 0, flag_6670 = 0, flag_6657 = 0, MAGIC_ADDR;
        UINT32  i;
    
         /* Power up PCIe */
        devicePowerPeriph (TARGET_PWR_PCIE);
        for(i=0; i<1000; i++) asm (" NOP");
    
        DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_APP_SERDES_CFG0), 0x00062320);  /* ss clock */	
        DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_APP_SERDES_CFG1), 0x00022320);  /* ss clock */
    ...
    }

    PCIE_APP_SERDES_CFG0 = 0x00062320 Corresponds with:
    - [5] RX_LOS=1,
    - [8:6] RX_CDR=4h,
    - [12:9] RX_EQ=0001b,
    - [13] RX_ENOC=1,
    - [15:14] RX_LOOPBACK = 0,
    - [16] TX_INVPAIR=0,
    - [17] TX_CM=1,
    - [18] TX_MSYNC=1,
    - [20-19] TX_LOOPBACK=0.

    Am I looking at the wrong register? Isn't the SERDES_CFGx register the one wihch determines the PCIe clocking scheme? If RBL and IBL write the same values, what else is the IBL doing that fixes the PCIe boot issue?

     

    Please enlighten me :)

    Thanks,

    Dirk

     

  • The 6678 and 6670 production EVM DO use IBL then jump to RBL for PCIE workaround. The 6657 production EVM however uses RBL by default unless you downgrade the FPGA to let it boot from IBL. There is some business decision for those different approaches.

    The PCIE DSS issue is about setting of the SERDES_CFGx registers. I will check more since you mentioned RBL and IBL do the same.

    The download link is working, I just downloaded them from eInfochips website and uploaded. You need to use v02 FPGA image.

    Regards, Eric

    http://e2e.ti.com/cfs-file.ashx/__key/communityserver-discussions-components-files/639/5314.C6657-Gauss-EVM_5F00_FPGA_5F00_v02.bit

    http://e2e.ti.com/cfs-file.ashx/__key/communityserver-discussions-components-files/639/4532.C6657-Gauss-EVM_5F00_FPGA_5F00_v03.bit

    6332.FPGA_Update.zip

     

     

     

  • Hi Eric,

    Thanks for uploading the FPGA images. It's good to see eInfoChips responded quickly to the issue.

    I just downgraded the FPGA and tried again. Still no boot in PCIe mode, but the SERDES_CFGx fields did change to 0x000622A0 and 0x000222A0 for lane 0 and lane 1 respectively. So now the RX_CDR field is set to 2h, where before it was 4h.

    Note that these are the default register values: Are you sure the v2 FPGA image boots though the IBL first and that it is not the other way around (eg FPGA v3 image boots though IBL first and the through the RBL)?

    Here's another issue: I tried to modify the value of the SERDES_CFGx register in the IBL to make sure it actually gets written (tried it with both FPGA FW images)....it does not. Also, the PCIe Device ID which is programmed in the IBL by default is not the one I see in the actual VENDOR_DEVICE_ID register (@ 0x21801000). The one in the IBL sourcecode is "0xb005104c" (in c665xinit.c), but the actual value I see in my debugger is "0xb006104c". 

    I followed these steps to modify and program the IBL (I am using MinGW):

    1. In "c665xinit.c" I modified the code such that it writes "0x00062220" to the SERDES_CFG0 register.
    2. [MinGW] cd /c/ti/mcsdk_2_01_02_06/tools/boot_loader/ibl/src/make
    3. [MinGW] make clean
    4. [MinGW] make evm_c6657_i2c ENDIAN=little I2C_BUS_ADDR=0x51" 
    5. powerup the EVM
    6. [CCS] Launch an EVM6657 target
    7. [CCS] Connect to core 0
    8. [CCS] Load "C:\ti\mcsdk_2_01_02_06\tools\writer\eeprom\evmc6657l\bin\eepromwriter_evm6657l.out"
    9. Modified eepromwriter_input.txt:

      file_name = i2crom_0x51_c6657_le.bin
      bus_addr = 0x51
      start_addr = 0x0C000000
      swap_data = 0

    10. copied i2crom_0x51_c6657_le.bin from "C:\ti\mcsdk_2_01_02_06\tools\boot_loader\ibl\src\make\bin" to   "C:\ti\mcsdk_2_01_02_06\tools\writer\eeprom\evmc6657l\bin"
    11. [CCS] hit run and wait until the program reports the image was written successfully..

    Did I do something wrong here? Is the "start_addr" eepromwriter_input.txt file correct? This is the default value.

    This post seems relevant, but the solution was not posted. It mentions IBL configuration. I am not sure what they are referring to:

    http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/123060.aspx

    Thanks!

    - Dirk

  • Dirk,

    For FPGA v2, in PCIE boot mode, the Program Counter (PC) of DSP will be inside LL2, that is, it boot from IBL (loaded from I2C address 0x51 into LL2) and stay inside LL2 forever, will NOT jump back to RBL (0x20b0xxxx).

    For FPGA V3, it boot directly from RBL, no IBL is used, so PC should be inside ROM (0x20b0xxxx).

    You steps to build IBL and flash to EEPROM seems correct, but the PCIE register observed seems contradict to it. Do you know where is your PC in PCIE boot mode? LL2 or ROM?

    Regards, Eric 

  • Yeah, I was wondering about that. The PC is still in ROM and never seems to get to L2 (0x008...). My boot-switches are set to PCIe boot, and I tried some other combinations, but PC always ends up in ROM. It seems to be in some kind of loop (I don't have access to the HW today, but can send you more details tomorrow).

    What's a good way to debug this? Maybe my IBL is somehow wrong? Could you upload an IBL image which is sure to be working? Also, is the I2C address I am storing the image at correct? (0x0C000000).

    Edit: Just to clarify: When I run the I2C POST test with FPGA v3, It completes the test successfully (observed via UART). When I downgrade the FPGA image to v2 - without changing anything else - the POST test does not start and I can see the PC is still in ROM. Tomorrow I can give you the exact address in ROM if that helps. I'll also post the DEVSTAT register value here: maybe the FPGA is not setting the bootpins correctly?

    Thanks,

    Dirk

  • Dirk,

    I repeated the steps for downgrading FPGA and reflashing IBL, the 6657 enumerated in my Linux PC. Attached are the screenshoot of the sequence. Hope it helps! If you have PC in ROM, it looks some steps are wrong. When flash FPGA, be careful to follow the steps from eInfochips.
    By the way, what environment you re-build the IBL? a WinXP PC or Win 7 PC with Wingw?

    Regards, Eric

    7181.6657_FPGA_IBL_PCIE.docx

  • Thanks Eric, This is really great. I'll try this right now and get back to you with the results.

  • A typo:

    Put card back into “no boot” mode and into a Linux PC =========> PCIE boot mode

    When power up, PC should be in LL2

    Sorry for this!

    Regards, Eric

  • No problem! I figured as much. 

    The IBL <==> FPGA v2 issue is now solved. However, the EVM6657 still only enumerates on 1 out of 3 hosts I tried. Effectively it behaves identical to the original EVM6657 (which boots directly with the RBL (with FPGA v3 GW)).

    So my original issue is still not solved. Could you test in some more hosts and let me know what you find on your end? I have 2 EVM6657s and they behave identically, so this should be consistent. The only host here that works has a “x9dr3-ln4f+” motherboard and an Intel® C606/C602 Chipset. The hosts which don't work are a HP xw9300 and uses a "NVIDIA nForce CK804" chipset and a Dell Precision T3610 which uses a Intel® C602 chipset like the working host.... I am not sure why machines with identical chipsets would behave differently, but perhaps it is a matter of different BIOSes/configuration.

    What else can I do to figure out what is going on? I hooked up a PCIe analyzer earlier last week, but that didn't give much info. Results varied accross runs, but mostly it came up with "invalid 10-bit code". Really I don't want to get to this level of debugging myself....

    Back to the IBL issue: It looks like the issue I had is related to rebuilding the image....I had not tried the included image before. The image I built is slightly larger than the original (47884 bytes vs 47848 bytes) and it does not work. [EDIT: works: see next post]  am using MinGW to build the image and the build completes successfully..... I attached the build log. One thing I found a bit suspicious is that the makeflow seems to be setup to ignore errors like so:

        /bin/sh: q: command not found
        make[5]: [c64x.da] Error 127 (ignored)

    6712.build_2014_02_25_A.txt

    Some notes/comments:

    • The IBL you used sets the SERDES_CFGx exactly the same as the RBL (0x00062320).
    • Concerning the eepromwriter:  
      • The parameter "start_addr" does not do anything and is coerced to 0x0C000000 in the eepromwriter program. Even if it did work, it does not accept hex values.
      • The documentation states you have to load the binary into RAM. However, this step is redundant: what the eepromwriter program actually does is using CIO to open the file you specify in "eepromwriter_input.txt" and load this into RAM itself. [EDIT: see next post]
    • Minor, but still: The documentation states you should place the EVM in "noboot" mode before programming. This step is also redundant as the emulator bypasses any of the bootmode logic and manipulates the PC directly using JTAG. 
    • The RBL its Device ID is set to 0xb006, but the IBL Device ID is set to 0xb005. The example host code from TI "mcsdk_2_01_02_06\tools\boot_loader\examples\pcie\linux_host_loader" looks for 0xb005.... So this example will fail "out of the box". What is the reason for the ID change?

    Thanks Eric, I really appreciate your help on this. I'll try some more hosts in the meanwhile when I find some different models. Like I said earlier, the primary use case will not be a PC host. The Root Complex will be an ARM processor. We're using a PCIe switch in between the two (a PEX8603). It is critical that this works flawlessly and that is why I would really like to find out the root cause for the enumeration failure which I have seen on several hosts.

    Please advice,

    Dirk

  • Dirk,

    Sorry to know that 6657 PCIE still didn't work with all the machines after downgrading FPGA, we tested with a Dell Precision T3500 and Optiplex 790 machine, they both worked. Let me try to find some models you used.

    For building the IBL, what is your host PC, Win XP or Win 7? Thanks for so many comments, we will investigate.

    Regards, Eric

  • It looks like I made some false assumptions concerning EEPROM writer. I can now build the IBL and run it successfully. You DO have to load the IBL its .bin file into RAM AND make sure this .bin file is located in the same directory as the eepromwriter.out file. image into RAM. After looking into "eepromwriter.c" I found out that it gets the file size from the file on the disk, but, it loads the data from RAM and not from the file. I apologize for the confusion.

    The machines I tried this on are:

    1. [Works] Supermicro CSE745 with “x9dr3-ln4f+” motherboard and an Intel® C606/C602 Chipset.
    2. [Does not enumerate] HP xw9300 with "NVIDIA nForce CK804" chipset
    3. [Does not enumerate]  Dell Precision T3610 with Intel® C602 chipset.
    4. [Does not even boot!] GIGABYTE GA-MA790GP-UD4H with AMD 790GX Chipset and AMD SB750 South bridge.

    The last machine in the list did not even want to boot into the BIOS, presumably due to enumeration failure. This test was only performed using FPGA GW v3. I'll try it with the FPGA v2 Gateware and the latest IBL next.

    I'll try to get access to the machines you mention for a 1 on 1 comparison. In the meanwhile I am working in parallel on getting the boot-loader going with the working setup.

    Thanks for all your help!

    Dirk

  • OK, I just confirmed this works on an Optiplex 790, at least with the FPGA V2 + latest IBL (I still have to try the FPGA v3 setup). Right away the card shows up in the BIOS "System Information" and lspci -n shows the card. I'll do some more tests.

    EDIT: BTW this uses the Intel Q65 Express

    Cheers,

    Dirk

  • These are my findings concerning the Optiplex 790:

    • EVM6657 FPGA v2 + latest IBL: 
      • Consistently boots (3 out of 3), but only when external power is applied.
    • EVM6657 FPGA v3 (stock EVM):
      • No boot, (5 out of 5) even with external power applied.

    The necessity for an external power supply can probably be attributed to the PCIe boot requirements (the minimum time between the PC's power rails being stable and the PCIe PERST# signal being deactivated is  Tpvperl = 100 ms).

    Find attached some screenshots with register values after the boot.1018.EVM6657 PCIe register values after boot.docx

    It's interesting to see that when the PCIe initializes correctly, it does so with only a single lane enabled: (see PCS_STATUS, 0x21800388 = 0x00001111). Do you see the same in your setup? 

    This is where I am at: I have 1 PC where both the stock EVM and the EVM with downgraded GW boot properly and a second PC (Optiplex 790) which only boots the EVM with downgraded GW. The whole thing feels "flaky".

    What is causing these boot issues? At this point I am out of ideas for debugging short of using a PCIe analyzer and doing a comprehensive study of the issue. This is not how I planned to spend my time....

    Please advice,

    Dirk

  • Dirk,

    "I have 1 PC where both the stock EVM and the EVM with downgraded GW boot properly", I thought this PC didn't use SSC, so RBL or IBL doesn't matter in this.

    "second PC (Optiplex 790) which only boots the EVM with downgraded GW", IBL works but not RBL, I thought this PC uses SSC. But we don't need an external power supply for 790 when we do the PCIE boot test (with IBL), everytime we succeed.

    For the lane enabled, I will check what we have in 790 PC. If you get GEN2 x1 lane, you have 5.0Gb/s throughput, this affects performance. Can you continue your work or this is blocking? What is the GEN1 or GEN2 or 1/2 lane when the 6657 is enumerated in your first PC? You can read that from DSP side PCIE register 0x21801080.

    Regards, Eric  

  • Thanks Eric,

    I attached some screenshots of the successful boot on the other PC (Supermicro):

    0184.EVM6657 PCIe register values after boot_Supermicro.docx 

    The value of LINK_STAT_CTRL (0x21801080) = 0x10220040, so LINK_SPEED = 0x02 which I believe corresponds with 5.0 GT/s (the PCIe manual does not define the meaning of values of this field). This is the same value as the Optiplex 790. The NEGOTIATED_LINK_WD is different however: 0x01 for the Optiples 790 and 0x02 for the Supermicro.

    I am not blocked, but the issue needs to be understood to prevent any issues in our final product.

    In parallel I am working on a driver for our application and I am able to load the provided "hello world" PCIe example in the Optiplex 790. In the Supermicro PC I could load the kernel module, but the EVM would never start the demo program. I tried manually triggering MSI_0 which would start execution, but somehow the program data written to the DDR was not the same as the "hello_world" program data and nothing was output on the UART. I did not spend a lot of time on this, so please do not attach to much value to this experiment.

    EDIT:

    Regarding the 790 you mention "I thought this PC uses SSC". How can you tell? Do you have a spec? This might help me in characterizing the issue better.

    You mention you are able to boot the 790 without applying external power to the EVM: is this with the default IBL (the one in "mcsdk_2_01_02_06\tools\program_evm\binaries\evm6657l") or did you rebuild the IBL yourself? I'd like to know why my system behaves differently than yours. This might not be a big deal considering the PCIe boot requirements (the EP needs to be ready 100ms after powerup from what I read) and perhaps the EVM just takes a bit longer. I have seen similar issues with large FPGAs where the configuration time is too long and they use a workaround to be able to have the PCIe ready on time (using a secondary smaller FPGA).

    Cheers,

    Dirk

  • Dirk,

    If the same card, without any changes, works on one system but not another. I would be interested in knowing the difference between the systems and see if I can find any clues there.

    Can you let me know the working/failed cases: the system's manufactor name and model name? If you are  using Linuxs, can you type in this command and also point out which PCIe slot you plugged in the EVM with.

    sudo dmidecode --type 9

    Thanks,

    David

  • Hi David,

    Thanks for your message. In this thread you'll find more info, but here's a list of the various model numbers and motherboards that go with it:

    The machines I tried this on are:

    1. [Works, 2x linkup] Supermicro CSE745 with “x9dr3-ln4f+” motherboard and an Intel® C606/C602 Chipset. 
    2. [Works, however, 1x linkup] Dell Optiplex 790 with Intel Q65 Express
    3. [Does not enumerate] HP xw9300 with "NVIDIA nForce CK804" chipset
    4. [Does not enumerate]  Dell Precision T3610 with Intel® C602 chipset.
    5. [Does not even boot!] GIGABYTE GA-MA790GP-UD4H with AMD 790GX Chipset and AMD SB750 South bridge.

    I did try various slots with machines which had multiple, but this never helped. The  slot I am using in the current machine (Optiplex 790) is Slot 1 (the only slot the card will fit into). Find attached a "dmidecode" listing. 

    $ sudo dmidecode --type 9
    # dmidecode 2.11
    SMBIOS 2.6 present.
    
    Handle 0x0023, DMI type 9, 17 bytes
    System Slot Information
    	Designation: Slot1
    	Type: x16 PCI Express 2
    	Current Usage: In Use
    	Length: Long
    	ID: 1
    	Characteristics:
    		3.3 V is provided
    		PME signal is supported
    	Bus Address: 0000:00:01.0
    
    Handle 0x0024, DMI type 9, 17 bytes
    System Slot Information
    	Designation: Slot2
    	Type: x4 PCI Express 2 x16
    	Current Usage: Available
    	Length: Short
    	ID: 2
    	Characteristics:
    		3.3 V is provided
    		PME signal is supported
    	Bus Address: 0000:00:1c.2
    
    Handle 0x0025, DMI type 9, 17 bytes
    System Slot Information
    	Designation: Slot3
    	Type: 32-bit PCI
    	Current Usage: Available
    	Length: Long
    	ID: 3
    	Characteristics:
    		5.0 V is provided
    		3.3 V is provided
    		PME signal is supported
    	Bus Address: 0000:00:1e.0
    
    Handle 0x0026, DMI type 9, 17 bytes
    System Slot Information
    	Designation: Slot4
    	Type: x1 PCI Express 2
    	Current Usage: Available
    	Length: Long
    	ID: 4
    	Characteristics:
    		3.3 V is provided
    		PME signal is supported
    	Bus Address: 0000:00:1c.0
    
    
    

    One thing I was hoping to get from Eric is confirmation that his card linkedup 2 lanes in the Optiplex 790. My card consistently enumerates, but always in 1x mode.

    Dirk

  • Dirk,

    Thanks for the info. I'd divide your issues into two scenarios:

    1) For the x1link up mode issue:

    If the PCIe device is connected directly to host PCIe slot without any PCIe switch such as PLX, I think it’s due to PCIe x2 mode is not a standard so not every PC can recognize x2 device, some motherboard only support x1, x4, x8, x16. In this case the x2 device will be downspeed to x1 mode.

    2) For not being enumerated issue:

    The best way is to get exactly same type of machine to try to reproduce your issue so we can debug it. However, I don't have those machines but I am trying.

    Let's take HP XW9300 for example, if you plug in another PCIe card, it works fine on this system, correct?

    best regards,

    David Zhou

  • Hi David,

    1) That seems like a valid explanation. Functionally the EVM with downgraded FPGA GW (v2) and the IBL works well: I can load my driver and receive MSIs, DMA data etc, so everything is functional. I am still unclear as to what the IBL exactly fixes: the SERDES_CFG registers contain the same values as an "original" EVM (so without IBL, FPGA GW v3).Please refer to an earlier post for more details.

    2) Please refer to the list in the post above. The HP XW9300 does not work: no enumeration takes place. This is a bit of an older machine, but it has a lot of chassis space which is great for larger cards like the EVM. A more recent machine which did not work for me is the "Dell Precision T3610".

    Thanks!

    Dirk

  • Dirk,

    For the not-being enumerated case, I am trying to get same machine as  you do and then investigate. I should be able to provide an update either late this week or early next week.

    best regards,

    David Zhou

  • Dirk,

    I've got a Gigabyte MB and was able to reproduce the issue. I'd keep you informed of findings.

    regards,

    David

  • Thanks for the update David. Let me know if you can find something.

    Cheers,

    Dirk

  • Hi! We have the same problem, but we use c6678. One hostPC enumerates PCIe, and the other doesn't. Both HostPC have equal motherboards Asus p9x79, and the equal BIOSes. Is there any update on your problem?
  • Hello,

    The Keystone I (6657/6670/6678) EVM is an AMC form factor card which provides multiple connectivity options for inter-connection with various systems, primarily for Telecom. In addition, it can be connected to a standard PCIE slot via an AMC to PCIE adaptor card. The manufacturer had tested the adaptor card with multiple platforms, however the adaptor card is not designed for 100% PCIe compatibility. The most significant compatibility issue is lack of PCIe Reset control on adaptor card. The PCIe Reset is not propagated to the SOC.  This can cause enumeration issues or boot-up issues.

    best regards,

    David Zhou

  • Hello, David. Thank you for the answer. We could see that reset on evm "awake" pcie, and we could see multimedia device. But hostPC in this case doesn't have resources for BARs. We tried to reload our PC and PC couldn't start up at all. Is there any way to apply correct reset? Thank you, Yuliya
  • Yuliya,

    Can you pxplain how do you did "reset" on EVM? If it is local reset by running program via PCIE link, the PCIE peripheral itself is not reset at all and the PCIE link should be there forever. There is no new enumeration process and what the BAR enumerated before should be still there after local reset, and I don’t understand why host PC doesn’t have resource for BARs.

    Maybe you did a warm or cold reset by pressing a button on the EVM? This is not the local reset in the topic.

    Regards, Eric

  • Hi Eric.

    We have found the way to fix the problem.  We connected electrically (using a simple scheme) reset from AMC connector with warm reset on EVM board (button). PCIe was detected by hostPC and hostPC had resources for BARs. This system works almost every time. Sometimes hostPC doesn't have resources for BARs. It stays not clear for me too. But now I can continue my work.

    We tried to apply reset pushing hard reset button, but hostPC didn't allocate resources for BARs. Probably because pcie wasn't detected by BIOS during power on PC. We also tried to connect hard reset on EVM with AMC reset, the result was the same as pushing hard reset button.

    Could you tell me, please, is there anything wrong in such system with our reset?

    Thank you for help! 

  • Good day! It's little too late and I'm not sure for 100 percent but I guess I've solved why EVM can't be enumerated on some computers! I have evm6678le and I've got sometimes PCIe malfunction with host PC when I've connected emulator through USB port on front panel of PC.  Using rear panel gets things better. Also at all it seems to look not good to connect board to PCIe port and at same time to USB because of bad ground in that case for high speed transactions in PCIe.