performantdata/clang-scala.g8

Giter8 template for combined Clang/Scala projects

A Giter8 template for combined Clang/Scala projects

This template generates a skeleton project that contains both Clang and Scala code. The goal is to make all of this work together in a natural way. It's aimed at Scala and Java developers who aren't up-to-date on C++ environment setup, and aims to get them started with a working example.

Specific goals (not all achieved yet) are:

• The project's directory structure supports multiple languages in a natural way. I should be able to mix Scala Native and Java HotSpot tests into the project in order to build and test linkages to those environments. Adding tests for other language environments, like Python, may also become desirable. (GraalVM may obviate the need for all these variations in the future.) The emphasis, though, is on developing mostly in C++ and testing against a small amount of Scala/Java/etc. code.

• It's possible to run builds and tests equally well from the command line or the IDE, with sbt controlling the whole.

• cmake at the command line builds, tests and packages all the Clang code.

• sbt at the command line builds, tests and packages everything.

• The IDE shows Doxygen documentation when I hover over terms.

• I can use either libstdc++ or libc++, as needed.

OS configuration

The definitive instructions for setting up the operating system's environment are found in the continuous integration tests, located in the GitHub Workflow for this project. Instructions are only given for Debian 10 currently.

Debian 10

Prepare Aptitude per the sbt download and backports instructions. I added backports in order to pick up a 3.14+ CMake.

Sbt, CMake, Ninja, Clang and Doxygen are installed in the usual Debian way. It's probably useful to pick up a few extras, too:

sudo aptitude install doxygen-gui doxygen-doc openjdk-11-jdk swig sbt
sudo aptitude -t buster-backports install cmake-qt-gui cmake-doc clang-8 clang-8-doc libc++1-8 ninja-build

running it

Giter8

An instance of this Giter8 template is created in the usual way:

sbt new performantdata/clang-scala.g8

The below assumes that you've changed into the created directory, the one with the name that you entered.

cmake

Though the CMake files theoretically support a range of CMake versions, they've actually only been tested with 3.16.3.

The top-level CMakeLists.txt file is in the src/ directory, so in the root directory, invoke:

export CC=/usr/bin/clang
export CXX=/usr/bin/clang++
cmake -S src -B target/cmake -G Ninja
cmake --build target/cmake
(cd target/cmake; ctest -VV)

setting the environment as described here. You may need to adjust the compiler paths from the above example. (Placing the environment settings in your login script may help keep from forgetting to set them, but then they affect every process. Your choice.)

sbt

TBD

directory structure

The directory layout may be unfamiliar to a typical C/C++ coder, who might have seen something more like this. It instead follows the standard sbt structure, which has been in use in the Java world for decades.

• It extends it by adding directories src/main/cpp/ and src/test/cpp/ for C/C++ source code for the product and the tests, respectively. There is a corresponding src/main/include/ for the distributed include files. The top-level CMakeLists.txt file is in the src/ directory.
• Build outputs are placed into target/cmake/ (instead of the usual build/).

You can think of it as two overlapping trees: one for the sbt build, and one for the CMake build (which the sbt build calls).

IDE configuration

This is discussed on a separate page.

further reading

The following tools are commonly used in C++ development. This is an opinionated list.

• Clang

  Some use GCC rather than Clang, but that's not the direction that I'm going. Future versions of this project will interface with OpenJDK's new GraalVM, which requires LLVM IR.

• SWIG

  Some use Java tools like javac -h to generate a JNI API from Java code. The advantage of SWIG is that it works from a C/C++ header-like syntax, generating the APIs for other languages. So it can not only generate a JNI API, but also APIs for Python, etc. from the same specification.

  Eventually, as GraalVM matures, I would hope that other languages would adopt it as their runtime. Then there would be no need for SWIG. But that's years away.

• CMake

  If you're new to CMake, the tutorial is an obvious place to start, but I found that reading about its build system in parallel helps. The language description is also enlightening on basic syntax. For a more gradual description via examples, you may find CMake Cookbook (2018) online via your local library.

• Ninja

  If you're new to Ninja, just think of it as an intermediate representation for build scripting. Or as UNIX Makefiles, stripped down for fast execution. You shouldn't need to work with Ninja code directly.

• Catch2

  Some use Googletest rather than Catch2, but I prefer the BDD style.

• Doxygen

related work

Similar instructions to these can be found in the book Advanced C++ (2019), which may be available online through your library. (I started on this project well before discovering this book.)

Mizux Seiha maintains a project demonstrating a polyglot CMake + SWIG project. It addresses CI, portable CMake, and SWIG code generation for Java/Python/.NET with language-specific tests and packaging, but doesn't go so far as to make it a template for new projects. My project doesn't care about CMake portability, since I expect high-performance code to run only on Linux. It's also intended for non-trivial development in the targeted scripting languages in an IDE, for providing a portable fallback implementation should a native code version be unavailable.

clang-scala.g8's Issues

deploy the compiled code

We need a deployment process for the native or LLVM IR libraries and JARs generated by the build process. It should be possible to upload a package of such contents to an Ivy or Maven repository. If only Coursier is supported, that's OK for now. There is apparently no standard defined for this yet. Required features:

• An archive format that can hold JARs, native libraries (shared or static) for multiple architectures, Python wheels, and other language-specific archives.
• A packaging type that can be uploaded to/downloaded from an Ivy or Maven repository like Maven Central, Artifactory or Nexus. This likely requires a new packaging type and thus a Coursier plugin to support it. The standard Maven packaging types include only those Java-specific ones for which Maven was created: JAR, WAR, EAR, RAR, POM, etc. Most packaging types seem to be based on ZIP, so we'd probably go with that.

native-only

I asked on SO about a standard solely for native and LLVM IR libraries, but it only got 10 hits and no answers. I don't think that such a standard exists except for the OS-level packaging that I mentioned there, RPM and DEB and such. While such packages can be deployed locally, they seem like overkill for the

JAR overloading

Some people have sought ways to use the JAR format for deploying non-JAR resources; for example, packaging native libraries into the JAR (under META-INF). A JAR can be used as an archive like this,
The problem with this is that the loading

AAR

There's an Android Archive format that is capable of storing JARs and native libraries. I'm not sure whether it supports LLVM IR libraries. Its contents are very specific to Android's needs.

Prefab

Included in the AAR is the Prefab archive format. This is the part of the AAR that contains the native libraries. Prefab is also a tool that generates the CMake integration needed to consume the modules in a Prefab package. This may provide the

Nix