Dependencies not downloaded until ktlint task is run with Gradle

[Drarok]

Dependencies not downloaded until ktlint task is run with Gradle

[shyiko]

Alright, I was able to reproduce aforementioned behavior on [email protected] (worked fine on gradle 2.4.1).
Solution is described at https://discuss.gradle.org/t/download-all-dependencies/6294/4, which boils down to:

1. add to build.gradle

allprojects {
    task dependenciesResolve {
        doLast {
            configurations.all { it.resolve() }
        }
    }
}

2. instead of gradle dependencies use gradle dependenciesResolve.

Tested on chibatching/docker-android-sdk docker image.

🎈

[shyiko]

@jeremymailen thank you for trying to help, I appreciate it 🙇‍♂️

[shyiko]

Hi @Drarok. Any chance you can share your build.gradle? The only case I can think of where ktlint would download anything is when using external 3rd party ruleset (-R flag). But even then caching ~/.m2 should work just fine.

FYI: You can use ktlint as a standalone binary (see https://ktlint.github.io/#getting-started) without the need for Maven or Gradle.

[Drarok]

I may have put things into the wrong gradle file, so here's both! Thanks for looking into this.

Project build.gradle:

// Top-level build file where you can add configuration options common to all sub-projects/modules.

buildscript {
    ext.kotlin_version = '1.1.2-3'
    repositories {
        jcenter()
        mavenCentral()
    }

    dependencies {
        classpath 'com.android.tools.build:gradle:2.3.2'
        classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"

        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files

        // The exception to this rule is compile-time dependencies, such as these:
        classpath 'de.felixschulze.gradle:gradle-hockeyapp-plugin:3.5'
        classpath 'com.github.shyiko:ktlint:0.6.1'
    }
}

allprojects {
    repositories {
        jcenter()
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}

App build.gradle:

apply plugin: 'com.android.application'
apply plugin: 'kotlin-android'
apply plugin: 'de.felixschulze.gradle.hockeyapp'

android {
    compileSdkVersion 25
    buildToolsVersion "25.0.0"
    defaultConfig {
        applicationId "<REDACTED>"
        minSdkVersion 19
        targetSdkVersion 25
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
}

configurations {
    ktlint
}

repositories {
    mavenCentral()
}

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
        exclude group: 'com.android.support', module: 'support-annotations'
    })
    compile "org.jetbrains.kotlin:kotlin-stdlib-jre7:$kotlin_version"
    compile 'com.android.support:appcompat-v7:25.3.1'
    compile 'com.android.support:recyclerview-v7:25.3.1'
    compile 'com.android.support:design:25.3.1'
    compile 'com.android.support:cardview-v7:25.3.1'
    testCompile 'junit:junit:4.12'
    ktlint 'com.github.shyiko:ktlint:0.6.1'
}

task ktlint(type: JavaExec) {
    main = "com.github.shyiko.ktlint.Main"
    classpath = configurations.ktlint
    args "src/**/*.kt"
}

check.dependsOn ktlint

task ktlintFormat(type: JavaExec) {
    main = "com.github.shyiko.ktlint.Main"
    classpath = configurations.ktlint
    args "-F", "src/**/*.kt"
}

hockeyapp {
    apiToken = "<REDACTED>"
}

[shyiko]

@Drarok Just to clarify: you are caching both ~/.gradle and ~/.m2?

[jeremymailen]

@Drarok what gradle target do you currently run in the dependencies section of your circle.yml to pull things to cache? Gradle of course is notoriously lazy/efficient about getting dependencies and of course CircleCI only caches after the dependencies step which is usually ahead of most people's gradle targets that do interesting things.

[Drarok]

I've just double checked, both are cached, yes.

Run: ./gradlew dependencies

Cache: Creating dependency cache from the following paths:
/home/ubuntu/.go_workspace
/home/ubuntu/.gradle
/home/ubuntu/.m2

Run: ./gradlew ktlint

Output (note the downloads from

> Connecting to Daemon> Configuring > 0/2 projects > root project > Resolving dependencies ':classpat> Configuring > 1/2 projects > :appNDK is missing a "platforms" directory.
If you are using NDK, verify the ndk.dir is set to a valid NDK directory.  It is currently set to /usr/local/android-sdk-linux/ndk-bundle.
If you are not using NDK, unset the NDK variable from ANDROID_NDK_HOME or local.properties to remove this warning.

> Configuring > 2/2 projects > Resolving dependencies ':app:_debugApk'> Configuring > 2/2 projectsIncremental java compilation is an incubating feature.
> Configuring > 2/2 projects > Resolving dependencies ':app:_releaseUnitTestApk> Configuring > 2/2 projects > Resolving dependencies ':app:_debugAndroidTestAp> Configuring > 2/2 projects> Building 0% > :app:ktlint > Resolving dependencies ':app:ktlint'> Building 0% > :app:ktlint:app:ktlint
Download https://jcenter.bintray.com/org/jetbrains/kotlin/kotlin-compiler-embeddable/1.1.0/kotlin-compiler-embeddable-1.1.0.jar
> Building 0% > :app:ktlint> Building 0% > :app:ktlint > 833 KB/25.88 MB downloaded> Building 0% > :app:ktlint > 1.87 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 3.06 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 4.50 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 6.25 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 7.14 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 8.92 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 9.64 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 10.87 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 11.70 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 12.78 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 13.99 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 15.26 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 15.92 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 16.50 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 17.23 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 17.77 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 18.54 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 20.03 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 21.31 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 23.57 MB/25.88 MB downloaded> Building 0% > :app:ktlint > 25.88 MB/25.88 MB downloaded
> Building 0% > :app:ktlintDownload https://jcenter.bintray.com/org/apache/httpcomponents/httpclient/4.3.5/httpclient-4.3.5.jar
> Building 0% > :app:ktlint
Download https://jcenter.bintray.com/org/apache/httpcomponents/httpcore/4.3.2/httpcore-4.3.2.jar
> Building 0% > :app:ktlintDownload https://jcenter.bintray.com/commons-codec/commons-codec/1.6/commons-codec-1.6.jar
> Building 0% > :app:ktlint> Building 0%
BUILD SUCCESSFUL

[jeremymailen]

It looks like it only triggers the download when the configurations ktlint is resolved. I don't think the classpath 'com.github.shyiko:ktlint:0.6.1' in the buildscript block is triggering a dependency download since it isn't part of an applied plugin? If you remove it I think you'll find your ktlint task still works 🤔.

[jeremymailen]

Well on 2nd take it looks like having classpath 'com.github.shyiko:ktlint:0.6.1' in buildscript does trigger a download of some dependencies on gradle dependencies but not a few of the transitive ones like kotlin-compiler-embeddable.

[Drarok]

Hmm, so am I stuck having it download its transitive dependencies, or can I force Gradle to download those somehow?

[shyiko]

Sorry @Drarok, got distracted at work. I'll try to provide an answer within 24h.

[Drarok]

Honestly, don't rush! I'm not back in work where this is is only a minor inconvenience until Tuesday anyway.

Have a good weekend! 😄

[shyiko]

@Drarok have a great weekend too ;)

[Drarok]

Finally got around to implementing this change, and my latest build didn't download anything extra, so everything's being cached as expected! 🎉

Thanks to everyone that helped.