mwrnd/opencapi-to-pcie

PCIe to OpenCAPI-Compatible SlimSAS 8x Adapter PCB

Work-in-Progress - Third Revision gerbers completed. Second Revision is being tested. PCIe x8 using the OpenCAPI connector works but requires a high quality cable and uses a PCIe Lane to Transceiver Channel ordering that Vivado complains about.

PCIe to OpenCAPI-Compatible SlimSAS8x

The Open Coherant Accelerator Processor Interface (OpenCAPI) was a standard that had FPGA-based Advanced Accelerated Cable (AAC) Add-In cards such as ADM-PCIE-9H3, ADM-PCIE-9H7, ADM-PCIE-9V3, ADM-PCIE-9V5, BittWare XUP-VV4, BittWare XUP-VVH, and Nvidia Innova-2 Flex SmartNIC.

The OpenCAPI interface is based on PCI-Express and uses a SlimSAS 8X (SFF-8654) Connector. This adapter enables connecting an OpenCAPI FPGA board to a host using PCIe over a SlimSAS cable.

Testing

The innova2_xdma_opencapi project is designed to test the OpenCAPI-to-PCIe Adapter using an Innova2 Flex SmartNIC.

The Innova2 SmartNIC's XCKU15P FPGA does not have its Configuration Block in the same column as the OpenCAPI GTY transceivers so it is impossible to configure the FPGA within the PCIe Specification's 100ms time limit. Motherboard boot must be delayed to allow the FPGA to configure itself before PCIe devices are enumerated by the host system. This can be accomplished by toggling the POWER button, then pressing and holding the RESET button for a second before releasing it. Or, connect a capacitor across the reset pins of an ATX motherboard's Front Panel Header:

Using a 3M 8ES8-1DF21-0.75 cable:

PCIe Link Status is usually excellent:

Using an SFPCables.com SFF-8654 to SFF-8654 8i cable:

PCIe Link Status is downgraded:

I am working on a third revision of the OpenCAPI-to-PCIe adapter to improve signal integrity.

Additional OpenCAPI Signals

Additional useful signals from the OpenCAPI connector are routed to a 6x1 0.1" Header. The pinout matches a TC74 I2C Temperature Sensor. Note 3.3V is from the PCIe connector. PRE is a Presence Detect pin which is connected to GND via a 50-Ohm resistor on the OpenCAPI AAC Add-In card. RST is connected to PCIe/OpenCAPI RESET.

The innova2_xdma_opencapi project features the ability to test a TC74Ax-3.3VAT in an OpenCAPI-to-PCIe Adapter.

PCB Layout

4-Layer PCB. Inner 2 layers are GND planes. Differential pairs are matched to a length of 61mm +/- 1mm both inter-pair and intra-pair (N-to-P).

Schematic

Design Notes

Refer to the ADM-PCIE-9V5 User Manual (Pg15-19of38) for the OpenCAPI pinout. Useful High Speed Design presentation.

Only a single component is required for the adapter, a U10A474200T/U10A474240T SlimSAS 8x Right-Angle SMD Connector. A SlimSAS 8x Cable such as the 3M 8ES8-1DF21(Datasheet) is required to use the adapter with an OpenCAPI FPGA Board.

Resistor R1 is shorted to connect nPRSNT1 to nPRSNT2_x8. The trace can be scratched off and nPRSNT1 can be connected to nPRSNT2_x1 or nPRSNT2_x4 using 28AWG to change the PCIe width.

Second Revision

The Second Revision of this project worked successfully with the First Release of the innova2_xdma_opencapi project but required a non-standard GTY Channel to PCIe Lane mapping.

The Second Revision used the OpenCAPI pinout from the ADM-PCIE-9V5 User Manual (Pg15-19of38):

Second Revision PCB Layout

4-Layer PCB. Inner 2 layers are GND planes. Differential pairs are matched to a length of 65mm +/- 1mm both inter-pair and intra-pair (N-to-P).

Second Revision Schematic

PCB Stackup

I am using values from JLCPCB.

Trace Impedance Control

OpenCAPI uses 85ohm impedance cables. I played with the values until I got the loosest differential pair coupling that is manufacturable with larger tolerances.