bitcoindevkit/bdk-kotlin

Kotlin jvm + android packages for bdk-ffi

bdk-kotlin's Issues

Proposition: migrate build scripts to Kotlin DSL

The Kotlin DSL is one of the two supported DSLs for Gradle. Because Kotlin is statically typed, it offers stronger guarantees and better tooling support (the IDEs can infer and work with the Kotlin DSL better than with the Groovy DSL).

I don't see downsides to moving to the Kotlin DSL other than the fact that it's a slightly newer DSL so not all devs are familiar with it. I do think it's gaining market share over the Groovy DSL however. Here are a few resources explaining the differences and recommending the Kotlin DSL:

1. Gradle Kotlin vs. Groovy DSL (side-by-side comparison)
2. 5 reasons to switch to the Gradle Kotlin DSL
3. Stack Overflow: Gradle: What is the benefit if I switch from Groovy to Kotlin?

I would be happy to work on the PR for that. I have good experience with the Kotlin DSL from using it in different projects.

Add support for armeabi-v7a on Android

We currently add the Rust target armv7-linux-androideabi in the readme and in the CI workflow for Android but we don't actually use it in the build script (nor in the Gradle plugin as of now).

Just opening an issue to keep track of this. I do think it might be a good idea to support armeabi-v7a but I need to do a bit more research.

This issue is almost a sister issue to #52.

Migrate build script logic into Gradle plugin

I have a MVP for a small Gradle plugin that would add the required tasks to build the native libraries and the bindings files, and would love some feedback on the approach. See PR #37.

One of the problems we currently have is that we build the bindings file independently of the native libraries, which creates problems if the version of the uniffi-bindgen cli tool used is not the same as the uniffi-crate dependency.

My current fix for that is to, as part of the plugin, pull the uniffi-rs repo at a specific tag and build the bindgen tool, which can be then used to generate the bindings. This is a bit of extra work because it requires a complete clone and build of the tool, but ensures that every build of the library will be consistent. The version of the cli tool is simply added as a config in the gradle.properties; this way we can simply keep both the Cargo.toml file and the gradle.properties files in sync and ensure the native libs and the bindings are built using the same version of uniffi-rs.

Remove compiler option to suppress warnings for using experimental unsigned types

The following block is no longer necessary as of Kotlin 1.5 (we're currently using 1.6.10):

allprojects {
    tasks.withType<org.jetbrains.kotlin.gradle.tasks.KotlinCompile>().configureEach {
        kotlinOptions.freeCompilerArgs += "-opt-in=kotlin.ExperimentalUnsignedTypes"
    }
}

See uniffi-rs docs here and related merged PR here for more info.

I can't assign myself but @notmandatory feel free to assign me this one.

Deprecate support for i686 on Android

Our currently supported architectures are arm64-v8a, x86_64, and i686. Looking at this, it really looks like the 32-bit architecture is not much in use anymore (and this thread is already 4 years old). Furthermore, LDK has dropped support for it as well (see this PR). The only use for it really would be developers who would run 32-bit computers and would require it for their emulators. Until we find such developers in the wild and they ask for it, I suggest we keep the build process lean and remove support for it.

The one architecture we used to support with bdk-jni and which I think we should bring in for bdk-android is armeabi-v7a, which is arm64-v8a's ancestor, but which is still in use today (as per the thread linked above), but I think this probably merits an issue of its own (see #53)

I have not added it in my Gradle plugin (#37) so far, and propose we simply do not add it. It's an easy add if ever we have users who would like it.

API docs missing `DerivationPath` class

API docs

Problem

It would be great to be able to generate library docs.

Description

I might have a working solution on this branch of my repo.

The workflow is as follows:

1. create a fresh bindings file
2. apply the patch docs.patch using patch jvm/src/main/kotlin/org/bitcoindevkit/bdk.kt docs.patch
3. generate the website using ./gradlew :jvm:dokkaHtml

The main question I have and the reason I have not put too much time in trying to port over all of the docs and polishing the current branch is that the patch does not seem to apply that easily, and I don't know to which extent the addition of a new API changes the bindings file. If the changes are extensive, I assume the patch will not work, and I'm still looking at a good way to fix that (like some interactive way using a GUI maybe), because at the moment when the patch fails it just... fails miserably but doesn't tell you much about why and where.

See the current state of the docs here:
bdk-android: https://thunderbiscuit.github.io/bdk-android-api-docs/
bdk-jvm: https://thunderbiscuit.github.io/bdk-jvm-api-docs/

Honestly they look pretty good! The question is whether changes will always completely break them or if the process can be streamlined well.

Use HTTPS instead of SSH to clone submodules

Problem

The current .gitmodules uses SSH to clone the bdk-ffi repo (i.e. [email protected]:bitcoindevkit/bdk-ffi.git)

Description

New contributers who aren't set up with SSH keys won't be authenticated by Github and will receive a Permission denied. Changing to HTTPS is the recommended way and lets you avoid the SSH key authentication.

Alternatives

No response

Update docs to reflect new bdk-ffi-bindgen tool

The workflow to build locally will change a bit, replacing the cli tool by the bdk-ffi-bindgen tool provided in the bdk-ffi submodule.

Add PGP key to repo and add docs on how to verify signature on libraries

Title says it all.

Nexus plugin doesn't work well with multi-project (that are not really multi-projects)

The core issue here is that the gradle-nexus.publish-plugin must be set at in the root gradle script, but our libraries are not really a "multi-project" gradle construct (they're their own thing, we just wanted them to share the test suite).

One of the issues this causes is that the plugin doesn't pick up the fact that the releases might be of type SNAPSHOT (defined in the specific gradle scripts) because it is defined at the project/root level, and therefore attempts to publish to a release repository, which rejects the release (because of course it's a snapshot). Attempting to pull the nexusPublishing block into the individual gradle scripts throws an error explaining that the plugin must be defined at the root level.

Other smaller nits are that it creates the bdk-kotlin namespace on the Nexus platform, when this is not really a valid construct (it's just the name of the repo).

Create GitHub actions workflow to run tests on PRs

A simple workflow would be a great start, and promote the growth of our test suite for both libraries.

Bump bdk-ffi version to latest (0.3.0)

Description

This should be relatively easy, mostly a question of fixing up the tests. The main difference is the introduction of Wallet instead of OnlineWallet and OfflineWallet.

Add version number to API docs website

I figured out how to do it. Opening this to not forget to PR it in.

Make signature optional for local maven publishing

I think forcing contributors to set up gpg signatures to simply test/deploy the library locally might be an unnecessary friction we could easily remove.

Failing to build

Hi there, I'm trying to build the project myself but I'm facing an issue.

First I updated uniffi in my Cargo.toml on the bdk-ffi submodule file to match the version installed in my CLI

[dependencies]
bdk = { version = "0.13", features = ["all-keys", "use-esplora-ureq"] }
uniffi_macros = "0.16.0"
uniffi = "0.16.0"
thiserror = "1.0"
anyhow = "=1.0.45" # remove after upgrading to next version of uniffi

[build-dependencies]
uniffi_build = "0.16.0"

Bu now I'm seeing this issue with ring dependency. I'm using NDK version 23.1.7779620

This is the console output I have

Build and test bdk-ffi library for local platform (darwin or linux)
~/workspace/bitcoin/bitcoindevkit/bdk-kotlin/bdk-ffi ~/workspace/bitcoin/bitcoindevkit/bdk-kotlin
    Finished release [optimized] target(s) in 0.03s
    Finished test [unoptimized + debuginfo] target(s) in 0.03s
     Running unittests (target/debug/deps/bdkffi-6bf461246c56dc6e)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Copy linux libs to jvm subproject
Generate kotlin bindings from bdk.udl to jvm subproject
aarch64,x86_64,i686
   Compiling ring v0.16.20
   Compiling secp256k1-sys v0.4.1
   Compiling semver-parser v0.10.2
   Compiling anyhow v1.0.45
   Compiling parking_lot_core v0.8.5
   Compiling getrandom v0.1.16
   Compiling socks v0.3.3
   Compiling crossbeam-utils v0.8.5
error: failed to run custom build command for `ring v0.16.20`

Caused by:
  process didn't exit successfully: `/home/caiof/workspace/bitcoin/bitcoindevkit/bdk-kotlin/bdk-ffi/target/release/build/ring-a763e9a8b7d8c1bd/build-script-build` (exit status: 101)
  --- stdout
  OPT_LEVEL = Some("3")
  TARGET = Some("aarch64-linux-android")
  HOST = Some("x86_64-unknown-linux-gnu")
  CC_aarch64-linux-android = None
  CC_aarch64_linux_android = None
  TARGET_CC = None
  CC = Some("aarch64-linux-android21-clang")
  CFLAGS_aarch64-linux-android = None
  CFLAGS_aarch64_linux_android = None
  TARGET_CFLAGS = None
  CFLAGS = Some("-D__ANDROID_API__=21")
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("false")

  --- stderr
  running "aarch64-linux-android21-clang" "-O3" "-DANDROID" "-ffunction-sections" "-fdata-sections" "-fPIC" "-D__ANDROID_API__=21" "-I" "include" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-DNDEBUG" "-c" "-o/home/caiof/workspace/bitcoin/bitcoindevkit/bdk-kotlin/bdk-ffi/target/aarch64-linux-android/release/build/ring-84f67e3140f20a93/out/aesv8-armx-linux64.o" "/home/caiof/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.20/pregenerated/aesv8-armx-linux64.S"
  <built-in>:368:9: error: '__ANDROID_API__' macro redefined [-Werror,-Wmacro-redefined]
  #define __ANDROID_API__ 21
          ^
  <built-in>:360:9: note: previous definition is here
  #define __ANDROID_API__ __ANDROID_MIN_SDK_VERSION__
          ^
  1 error generated.
  thread 'main' panicked at 'execution failed', /home/caiof/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.20/build.rs:656:9
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
The following warnings were emitted during compilation:

warning: In file included from <built-in>:365:
warning: <command line>:2:9: warning: '__ANDROID_API__' macro redefined [-Wmacro-redefined]
warning: #define __ANDROID_API__ 21
warning:         ^
warning: <built-in>:359:9: note: previous definition is here
warning: #define __ANDROID_API__ __ANDROID_MIN_SDK_VERSION__
warning:         ^
warning: 1 warning generated.
warning: In file included from <built-in>:365:
warning: <command line>:2:9: warning: '__ANDROID_API__' macro redefined [-Wmacro-redefined]
warning: #define __ANDROID_API__ 21
warning:         ^
warning: <built-in>:359:9: note: previous definition is here
warning: #define __ANDROID_API__ __ANDROID_MIN_SDK_VERSION__
warning:         ^
warning: depend/secp256k1/src/secp256k1.c:7:9: warning: 'SECP256K1_BUILD' macro redefined [-Wmacro-redefined]
warning: #define SECP256K1_BUILD
warning:         ^
warning: <command line>:3:9: note: previous definition is here
warning: #define SECP256K1_BUILD 1
warning:         ^
warning: 2 warnings generated.

error: build failed

Update readme

The example in the readme is outdated and should be updated to work with v0.8.0.

Error "Option::unwrap() on a None value" when broadcasting

Description

Getting this error: Broadcast error: called Option::unwrap() on a None value when broadcasting. I think the return type of the broadcast transaction has changed?

It's not preventing the broadcast however. Just opening this to not forget it.

Artifact

• bdk-jvm
• bdk-android

Platform

• ARM64 Android
• 64-bit x86 Android
• 32-bit x86 Android
• 64-bit x86 Linux (kernel 2.6.32+, glibc 2.11+)
• 64-bit x86 macOS (10.7+, Lion+)
• ARM64 macOS (11.0+, Big Sur+)

Publish sources and javadoc for bdk-android

Problem

It seems that sources and javadocs for bdk-android are not being generated and published to maven:
https://repo1.maven.org/maven2/org/bitcoindevkit/bdk-android/0.4.0.

Description

When using bdk-android as a library in your project it's not currently possible to look up source code and documentation of bdk-android because sources and javadocs are not being generated and published.

With AGP 7.1.0 publishing Android sources and javadocs is very simple, we can use that https://developer.android.com/studio/releases/gradle-plugin#build-variant-publishing.

Alternatives

No response

Hosting location of API docs

Just opening this so we don't forget to migrate the API docs from where they are now (in my private repos) to where they should live long-term.

cannot access com.sun.jna.Pointer

Description

I got a compilation error when using bdk-jvm 0.7.1 with the graalvm.

(java-compile) on project batch-payouts: Compilation failure
[ERROR] /Users/severinbuhler/git/lipa/batch-payouts/src/main/java/com/lipa/config/env/AbstractWalletEnvironment.java:[22,16] cannot access com.sun.jna.Pointer
[ERROR]   class file for com.sun.jna.Pointer not found
[ERROR] 
[ERROR] -> [Help 1]
[ERROR] 

After some further research, turns out graalvm doesn't natively support jna.

Solution

Add jna as a dependency to your pom.xml.

    <dependency>
      <groupId>com.sun.jna</groupId>
      <artifactId>jna</artifactId>
      <version>3.0.9</version>
    </dependency>

How to Reproduce

Use bdk-jvm with graalvm.

Expected Result

Should work without any additional dependencies. It's not a big issue though as the workaround is easy.

Version

• bdk-jvm 0.7.1

Artifact

• bdk-jvm
• bdk-android

Platform

• ARM64 Android
• 64-bit x86 Android
• 32-bit x86 Android
• 64-bit x86 Linux (kernel 2.6.32+, glibc 2.11+)
• 64-bit x86 macOS (10.7+, Lion+)
• ARM64 macOS (11.0+, Big Sur+)

Relevant log output

No response

Add changelog

Keeping a changelog is a good practice and all other bdk projects have one. We should add one to bdk-kotlin as well.

Question: do we want one for each of the library or one for both?

Create github actions pipeline to build and publish

Use actions-rs/cargo and cross-rs to build and package for all supported targets. See example "Building Rust binaries for different platforms".