Giter Site home page Giter Site logo

Windows Platform Development Kit

The Windows Platform Development Kit (WPDK) enables applications based on the Storage Performance Development Kit (SPDK) to build and run as native Windows executables, bringing the power and performance of NVMe and NVMe-oF to Windows. It provides header files and a lightweight library that implement the required POSIX/Linux functionality.

The scope of the project is limited to supporting the Storage Performance Development Kit. Unlike Cygwin, it is not intended to be a generic POSIX emulation library. Functionality is mapped as closely as possible to existing Windows semantics with the minimum of emulation.

rtegrity is leading the ongoing development of the project, which was contributed by DataCore as part of the work to build OpenEBS Mayastor, a high speed Container Attached Storage solution for OpenEBS.

Recent News

The Windows Driver Unit Test Framework (WDUTF) has been added to WPDK, to enable the unit testing of Windows kernel drivers. This has been a key tool in DataCore's test and development process for many years.

For more details, see the WDUTF Repository where the code and documentation can be found!

In this ReadMe

Documentation

Further information about the design and implementation can be found in the project documentation.

Source Code

The source code can be obtained using:

git clone https://github.com/wpdk/wpdk

Getting Started

The project can be built and tested independently, but it is intended to be used in conjunction with SPDK. SPDK requires a POSIX-like environment for configuration, compiling and testing. Determining the best way to achieve this is still work in progress, but currently the recommended option is cross-compilation under Linux / WSL.

A Getting Started guide is available:

For more advanced use, the following guides are available, but these are almost certainly not the documentation that you want to follow!

Runtime Prerequisites

A system running Windows build 17063 or later is required to support AF_UNIX sockets which are used for SPDK configuration.

Running SPDK on Windows depends upon the work being done by the DPDK community to add support for Windows. Please follow Running DPDK Applications in the Getting Started Guide for Windows to grant 'lock pages in memory' privileges and to load the 'virt2phys' driver which can be found in the dpdk-kmods repository.

Note: It is possible to run an iSCSI or NVMe over TCP target stack without the 'virt2phys' driver, but this currently requires a patch to SPDK. The 'windows' branch of wpdk/spdk already includes this, or it can be obtained using:

# https://review.spdk.io/gerrit/c/spdk/spdk/+/6697
git apply ../wpdk/scripts/patches/spdk-rfc-when-virt2phys-is-unavailable.diff

Access to physical NVMe disks requires loading the 'netuio' and 'virt2phys' drivers from the dpdk-kmods repository. See the README files in the repository for build and installation instructions. Additional information can be found in Using a Physical NVMe Disk.

Current Status

The project is at an alpha stage:

  • All of the SPDK source compiles, apart from spdk_top which requires libcurses.
  • All of the SPDK Unit Tests pass.
  • The iSCSI target can serve storage.
  • The NVMe over TCP target can serve storage.
  • The SPDK stack can attach to a physical NVMe disk and issue I/O.
  • Unit tests exist for the majority of the WPDK functionality.

Limitations

  • There are still a few areas that are currently unimplemented, or where quick hacks have been applied. These are indicated in the code with WPDK_UNIMPLEMENTED and HACK.

  • Where new functionality has been implemented (epoll, aio, etc) the initial focus has been on functional correctness rather than performance.

  • Currently only x64 builds are supported.

  • Python 3.9 does not support AF_UNIX for sockets on Windows which means that the SPDK ./scripts/rpc.py does not work. Currently, the best workaround is to use WSL to access the socket file which can be located in either /mnt/c/tools/msys64/var/tmp or %TEMP%.

Known Issues

  • SPDK applications will fail to start if DPDK cannot allocate large pages of memory. If memory is available but allocations are failing, try rebooting to reduce fragmentation.

Unit Tests

Unit tests are available for libcunit and WPDK. Once built, these can be run with:

.\build\bin\test_cunit.exe
.\build\bin\wpdk_unit.exe

The file .\autotest.bat is provided as a convenience.

Examples

The generated SPDK binaries in spdk\build\bin can be used to create example target stacks.

# 64MB loopback iSCSI RAM disk
.\iscsi_tgt.exe -c wpdk\test\iscsi\iscsi_loopback.json

# Physical NVMe disk presented over iSCSI
.\iscsi_tgt.exe -c wpdk\test\iscsi\iscsi_nvme.json

# 50MB NVMe over TCP RAM disk (replace LOCAL-IP in nvmf_tgt.json)
.\nvmf_tgt.exe -c wpdk\test\iscsi\nvmf_tgt.json"

Contributing

Contributions are welcome and needed! It is expected that the process will be modelled on the SPDK Development Guidelines. In these initial stages of the project, please email the maintainers directly.

Please join the SPDK community and tell us how you are using SPDK on Windows. For real-time discussions, the SPDK Slack contains a Windows channel.

Dependencies

The WPDK repository currently contains the following forked projects:

  • SPDK - contains experimental changes for Clang support on Windows in the 'windows' branch. This is only needed with the advanced guides in Getting Started.
  • DPDK - temporarily required in conjunction with the SPDK fork until fixes are upstreamed.
  • CUnit - a convenience project for build purposes (no development).

Acknowledgements

rtegrity is leading the ongoing development of the Windows Platform Development Kit, which was contributed by DataCore.

Portions of the code are based on work done by the DPDK community to add support for Windows.

Core Maintainers

The core maintainers primary responsibility is to provide technical oversight for the WPDK Project. The current list includes:

Windows Platform Development Kit's Projects

cunit icon cunit

A Unit Testing Framework for C

spdk icon spdk

Storage Performance Development Kit

wdutf icon wdutf

Windows Driver Unit Test Framework

wpdk icon wpdk

Windows Platform Development Kit

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.