ysb33r/ivypot-gradle-plugin

Ivypot is a Gradle plugin to help managing a local off-line repository for builds in restricted environments

ivypot-gradle-plugin's Issues

Plugin does not find 'aar' packages defined in local m2repository from android

I need to create offline re./pository for android project. It relies on eg. multidex support.
Running syncRemoteRepositories task fails with message:

[syncRemoteRepositoriesResolve] :::: WARNINGS
[syncRemoteRepositoriesResolve]                 module not found: com.android.support#multidex-instrumentation;1.0.1
[syncRemoteRepositoriesResolve]         ==== BintrayJCenter: tried
[syncRemoteRepositoriesResolve]           https://jcenter.bintray.com/com/android/support/multidex-instrumentation/1.0.1/multidex-instrumentation-1.0.1.pom
[syncRemoteRepositoriesResolve]           -- artifact com.android.support#multidex-instrumentation;1.0.1!multidex-instrumentation.jar:
[syncRemoteRepositoriesResolve]           https://jcenter.bintray.com/com/android/support/multidex-instrumentation/1.0.1/multidex-instrumentation-1.0.1.jar
[syncRemoteRepositoriesResolve]         ==== MavenRepo: tried
[syncRemoteRepositoriesResolve]           http://repo1.maven.org/maven2/com/android/support/multidex-instrumentation/1.0.1/multidex-instrumentation-1.0.1.pom
[syncRemoteRepositoriesResolve]           -- artifact com.android.support#multidex-instrumentation;1.0.1!multidex-instrumentation.jar:
[syncRemoteRepositoriesResolve]           http://repo1.maven.org/maven2/com/android/support/multidex-instrumentation/1.0.1/multidex-instrumentation-1.0.1.jar
[syncRemoteRepositoriesResolve]         ==== maven: tried
[syncRemoteRepositoriesResolve]           -- artifact com.android.support#multidex-instrumentation;1.0.1!multidex-instrumentation.jar:
[syncRemoteRepositoriesResolve]           file:////Users/filip/Library/Android/sdk/extras/android/m2repository/com/android/support/multidex-instrumentation/1.0.1/multidex-instrumentation-1.0.1.jar
[syncRemoteRepositoriesResolve]                 ::::::::::::::::::::::::::::::::::::::::::::::
[syncRemoteRepositoriesResolve]                 ::          UNRESOLVED DEPENDENCIES         ::
[syncRemoteRepositoriesResolve]                 ::::::::::::::::::::::::::::::::::::::::::::::
[syncRemoteRepositoriesResolve]                 :: com.android.support#multidex-instrumentation;1.0.1: not found
[syncRemoteRepositoriesResolve]                 ::::::::::::::::::::::::::::::::::::::::::::::
[syncRemoteRepositoriesResolve] 
[syncRemoteRepositoriesResolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
:app:syncRemoteRepositories FAILED

task configuration is as follow:

syncRemoteRepositories {
    repoRoot "$projectRootDir/libs-repository" //define location of offline repo

    repositories {
        jcenter()
        mavenCentral()
        maven {
            url "file:///" + System.getProperty("ANDROID_HOME") + "/extras/android/m2repository"
        }
    }

//    configurations 'compile', 'testCompile' //Restrict the configurations you would like to have added to the synchronisation process. If none are supplied, dependencies from all configurations will be added.

    includeBuildScriptDependencies = true
}

It seems that it tries to find multidex-instrumentation-1.0.1.jar instead of using multidex-instrumentation-1.0.1.aar

Add ability to cache arbitrary URLs

Download binaries from arbitrary URLs and sotre them in a relative path. This will help with testing whether binaries have to be downloaded from remote sites.

Cannot use pattern-based Ivy Resolves since Gradle 2.3

Compains about

Caused by: java.lang.ClassNotFoundException: org.gradle.api.internal.artifacts.repositories.layout.PatternRepositoryLayout

Support Google repositories

Run ivy proces in a separate JVM

This will resolve the

> Failed to apply plugin [id 'org.ysb33r.ivypot']
   > Could not create task ':testfixtures-offline-repo:syncRemoteRepositories'.
      > Could not create task of type 'OfflineRepositorySync'.
         > java.lang.ClassCastException: org.apache.xerces.parsers.XIncludeAwareParserConfiguration cannot be cast to org.apache.xerces.xni.parser.XMLParserConfiguration

issue.

Cannot resolve versions with Spring Boot Gradle Plugin

Hi,

There is a Spring Boot Gradle Plugin to manage Spring dependency versions: https://docs.spring.io/spring-boot/docs/current/gradle-plugin/reference/html/

Unfortunately it does not work with IvyPot:

$ ./gradlew syncRemoteRepositories

> Task :syncRemoteRepositories FAILED
[syncRemoteRepositoriesConfigure] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/ ::
[syncRemoteRepositoriesConfigure] :: loading settings :: file = /Users/blockchain/Development/digits-dataservice/Eclipse/ivypot-test/build/tmp/syncRemoteRepositories/ivysettings.xml
[syncRemoteRepositoriesResolve] ivy.instance reference an ivy:settings defined in an other classloader.  An new default one will be used in this project.
[syncRemoteRepositoriesResolve] :: problems summary ::
[syncRemoteRepositoriesResolve] :::: WARNINGS
[syncRemoteRepositoriesResolve]                 module not found: org.springframework.boot#spring-boot-starter-web;null
[syncRemoteRepositoriesResolve]         ==== MavenRepo: tried
[syncRemoteRepositoriesResolve]           http://repo1.maven.org/maven2/org/springframework/boot/spring-boot-starter-web/null/spring-boot-starter-web-null.pom
[syncRemoteRepositoriesResolve]           -- artifact org.springframework.boot#spring-boot-starter-web;null!spring-boot-starter-web.jar:
[syncRemoteRepositoriesResolve]           http://repo1.maven.org/maven2/org/springframework/boot/spring-boot-starter-web/null/spring-boot-starter-web-null.jar
[syncRemoteRepositoriesResolve]                 ::::::::::::::::::::::::::::::::::::::::::::::
[syncRemoteRepositoriesResolve]                 ::          UNRESOLVED DEPENDENCIES         ::
[syncRemoteRepositoriesResolve]                 ::::::::::::::::::::::::::::::::::::::::::::::
[syncRemoteRepositoriesResolve]                 :: org.springframework.boot#spring-boot-starter-web;null: not found
[syncRemoteRepositoriesResolve]                 ::::::::::::::::::::::::::::::::::::::::::::::

Here's the build.gradle:

buildscript {
    ext {
        springBootVersion = '2.0.1.RELEASE'
    }
    repositories {
        mavenLocal()
        mavenCentral()
        maven { url 'https://plugins.gradle.org/m2/' }
    }
    dependencies {
        classpath "org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}"
    }
}

plugins {
  id 'org.ysb33r.ivypot' version '0.7'
}

apply plugin: 'java'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'

group = 'com.digits'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = 1.8

repositories {
    mavenLocal()
    mavenCentral()
}

dependencies {
    compile "org.springframework.boot:spring-boot-starter-web" // :2.0.1.RELEASE"
}

syncRemoteRepositories {
  repoRoot "${buildDir}/repo"

  repositories {
    mavenCentral()
  }

  addAllProjects()
}

When I change the dependencies to

dependencies {
    compile "org.springframework.boot:spring-boot-starter-web:2.0.1.RELEASE"
}

it works.

Maven credentials are not working

It seems that the maven credentials are not used when accessing a maven repo:

plugins {
   id 'org.ysb33r.ivypot' version '0.6'
}

syncRemoteRepositories {
  repoRoot file('../gradle_offline_repo').absolutePath
  repositories {
    maven {
      url 'http://...'
        credentials {
          username 'user'
          password '...'
      }
    }
  }
}

dependencies {                          
  compile 'org.eclipse.etrice:org.eclipse.etrice.generator.java:1.1.2'
}

When running gradle sync (Gradle 4.3.1, Windows10 64bit, Java 1.8) the dependency cannot be resolved. Here is the debug output of gradle:

10:21:32.044 [DEBUG] [org.gradle.internal.progress.DefaultBuildOperationExecutor] Build operation 'Task :syncRemoteRepositories' started
10:21:32.046 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter] Starting to execute task ':syncRemoteRepositories'
10:21:32.151 [INFO] [org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter] Putting task artifact state for task ':syncRemoteRepositories' into context took 0.103 secs.
10:21:32.152 [DEBUG] [org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter] Determining if task ':syncRemoteRepositories' is up-to-date
10:21:32.152 [DEBUG] [org.gradle.api.internal.changedetection.state.CacheBackedTaskHistoryRepository] Implementation for task ':syncRemoteRepositories': org.ysb33r.gradle.ivypot.OfflineRepositorySync_Decorated@a413994fb1226014355981f30b8ff143
10:21:32.152 [DEBUG] [org.gradle.api.internal.changedetection.state.CacheBackedTaskHistoryRepository] Action implementations for task ':syncRemoteRepositories': [org.ysb33r.gradle.ivypot.OfflineRepositorySync_Decorated@a413994fb1226014355981f30b8ff143]
10:21:32.153 [DEBUG] [org.gradle.api.internal.changedetection.state.CacheBackedTaskHistoryRepository] Snapshotting property repoRoot (OUTPUT) for task ':syncRemoteRepositories'
10:21:32.155 [DEBUG] [org.gradle.api.internal.changedetection.state.CacheBackedTaskHistoryRepository] No discovered inputs for task ':syncRemoteRepositories'
10:21:32.155 [INFO] [org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter] Up-to-date check for task ':syncRemoteRepositories' took 0.002 secs. It is not up-to-date because:
  Task has failed previously.
10:21:32.155 [DEBUG] [org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter] Ensuring directory exists for property repoRoot (OUTPUT) at C:\Dev\Tools\svn\hsl\utilities\build\gradle_offline_repo
10:21:32.156 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter] Executing actions for task ':syncRemoteRepositories'.
10:21:32.156 [DEBUG] [org.gradle.internal.progress.DefaultBuildOperationExecutor] Build operation 'Execute exec for :syncRemoteRepositories' started
10:21:32.398 [QUIET] [system.out] [syncRemoteRepositoriesConfigure] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/ ::
10:21:32.402 [QUIET] [system.out] [syncRemoteRepositoriesConfigure] :: loading settings :: file = C:\Dev\Tools\svn\hsl\utilities\build\gradle_offline_script\build\tmp\syncRemoteRepositories\ivysettings.xml
10:21:32.500 [QUIET] [system.out] [syncRemoteRepositoriesResolve] ivy.instance reference an ivy:settings defined in an other classloader.  An new default one will be used in this project.
10:21:33.039 [DEBUG] [sun.net.www.protocol.http.HttpURLConnection] sun.net.www.MessageHeader@61905d005 pairs: {HEAD /artifactory/maven/org/eclipse/etrice/org.eclipse.etrice.generator.java/1.1.2/org.eclipse.etrice.generator.java-1.1.2.pom HTTP/1.1: null}{User-Agent: Apache Ivy/2.2.0}{Host: <host>}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
10:21:33.059 [DEBUG] [sun.net.www.protocol.http.HttpURLConnection] sun.net.www.MessageHeader@202e7a839 pairs: {null: HTTP/1.1 401 Unauthorized}{Date: Tue, 17 Apr 2018 08:21:29 GMT}{Server: Artifactory/5.8.4}{X-Frame-Options: SAMEORIGIN}{X-Artifactory-Id: 83e49a5e9efcf080:6e81e18d:162486af6e3:-8000}{WWW-Authenticate: Basic realm="Artifactory Realm"}{Content-Type: application/json;charset=ISO-8859-1}{Keep-Alive: timeout=2, max=100}{Connection: Keep-Alive}
10:21:33.077 [DEBUG] [sun.net.www.protocol.http.HttpURLConnection] sun.net.www.MessageHeader@391bfea65 pairs: {HEAD /artifactory/maven/org/eclipse/etrice/org.eclipse.etrice.generator.java/1.1.2/org.eclipse.etrice.generator.java-1.1.2.jar HTTP/1.1: null}{User-Agent: Apache Ivy/2.2.0}{Host: <host>}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
10:21:33.098 [DEBUG] [sun.net.www.protocol.http.HttpURLConnection] sun.net.www.MessageHeader@4808f7d09 pairs: {null: HTTP/1.1 401 Unauthorized}{Date: Tue, 17 Apr 2018 08:21:29 GMT}{Server: Artifactory/5.8.4}{X-Frame-Options: SAMEORIGIN}{X-Artifactory-Id: 83e49a5e9efcf080:6e81e18d:162486af6e3:-8000}{WWW-Authenticate: Basic realm="Artifactory Realm"}{Content-Type: application/json;charset=ISO-8859-1}{Keep-Alive: timeout=2, max=99}{Connection: Keep-Alive}
10:21:33.227 [QUIET] [system.out] [syncRemoteRepositoriesResolve] :: problems summary ::
10:21:33.227 [QUIET] [system.out] [syncRemoteRepositoriesResolve] :::: WARNINGS
10:21:33.229 [QUIET] [system.out] [syncRemoteRepositoriesResolve]               module not found: org.eclipse.etrice#org.eclipse.etrice.generator.java;1.1.2
10:21:33.230 [QUIET] [system.out] [syncRemoteRepositoriesResolve]       ==== maven: tried
10:21:33.230 [QUIET] [system.out] [syncRemoteRepositoriesResolve]         http://<host>/artifactory/maven/org/eclipse/etrice/org.eclipse.etrice.generator.java/1.1.2/org.eclipse.etrice.generator.java-1.1.2.pom
10:21:33.230 [QUIET] [system.out] [syncRemoteRepositoriesResolve]         -- artifact org.eclipse.etrice#org.eclipse.etrice.generator.java;1.1.2!org.eclipse.etrice.generator.java.jar:
10:21:33.230 [QUIET] [system.out] [syncRemoteRepositoriesResolve]         http://<host>/artifactory/maven/org/eclipse/etrice/org.eclipse.etrice.generator.java/1.1.2/org.eclipse.etrice.generator.java-1.1.2.jar
10:21:33.231 [QUIET] [system.out] [syncRemoteRepositoriesResolve]               ::::::::::::::::::::::::::::::::::::::::::::::
10:21:33.231 [QUIET] [system.out] [syncRemoteRepositoriesResolve]               ::          UNRESOLVED DEPENDENCIES         ::
10:21:33.231 [QUIET] [system.out] [syncRemoteRepositoriesResolve]               ::::::::::::::::::::::::::::::::::::::::::::::
10:21:33.232 [QUIET] [system.out] [syncRemoteRepositoriesResolve]               :: org.eclipse.etrice#org.eclipse.etrice.generator.java;1.1.2: not found
10:21:33.232 [QUIET] [system.out] [syncRemoteRepositoriesResolve]               ::::::::::::::::::::::::::::::::::::::::::::::
10:21:33.232 [QUIET] [system.out] [syncRemoteRepositoriesResolve]
10:21:33.234 [DEBUG] [org.gradle.internal.progress.DefaultBuildOperationExecutor] Completing Build operation 'Execute exec for :syncRemoteRepositories'
10:21:33.234 [DEBUG] [org.gradle.api.internal.changedetection.state.CacheBackedTaskHistoryRepository] Snapshotting property repoRoot (OUTPUT) for task ':syncRemoteRepositories'
10:21:33.237 [DEBUG] [org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter] Removed task artifact state for {} from context.
10:21:33.237 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter] Finished executing task ':syncRemoteRepositories'
10:21:33.237 [DEBUG] [org.gradle.internal.progress.DefaultBuildOperationExecutor] Completing Build operation 'Task :syncRemoteRepositories'
10:21:33.238 [null] [org.gradle.internal.progress.DefaultBuildOperationExecutor]
10:21:33.238 [DEBUG] [org.gradle.internal.progress.DefaultBuildOperationExecutor] Build operation 'Task :syncRemoteRepositories' completed
10:21:33.238 [INFO] [org.gradle.execution.taskgraph.DefaultTaskPlanExecutor] :syncRemoteRepositories (Thread[Task worker for ':',5,main]) completed. Took 1.193 secs.
10:21:33.238 [DEBUG] [org.gradle.internal.work.DefaultWorkerLeaseService] Worker lease root.1.6 completed (1 worker(s) in use)
10:21:33.238 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Task worker for ':': released lock on root.1.6
10:21:33.238 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Task worker for ':': released lock on :
10:21:33.238 [DEBUG] [org.gradle.execution.taskgraph.DefaultTaskPlanExecutor] Task worker [Thread[Task worker for ':',5,main]] finished, busy: 1.193 secs, idle: 0.002 secs
10:21:33.239 [DEBUG] [org.gradle.internal.progress.DefaultBuildOperationExecutor] Completing Build operation 'Run tasks'
10:21:33.240 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
10:21:33.240 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] FAILURE: Build failed with an exception.
10:21:33.241 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
10:21:33.241 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * What went wrong:
10:21:33.241 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Execution failed for task ':syncRemoteRepositories'.
10:21:33.241 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] > impossible to resolve dependencies:
10:21:33.241 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]           resolve failed - see output for details
10:21:33.241 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
10:21:33.241 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * Try:
10:21:33.241 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Run with --stacktrace option to get the stack trace.
10:21:33.241 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
10:21:33.241 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * Get more help at https://help.gradle.org
10:21:33.241 [ERROR] [org.gradle.internal.buildevents.BuildResultLogger]
10:21:33.241 [ERROR] [org.gradle.internal.buildevents.BuildResultLogger] BUILD FAILED in 3s
10:21:33.241 [LIFECYCLE] [org.gradle.internal.buildevents.BuildResultLogger] 1 actionable task: 1 executed

The maven repo is provided by an artifactory server and I can see following requests made:

20180417073746|1|REQUEST|ip_xxx|non_authenticated_user|HEAD|/maven/org/eclipse/etrice/org.eclipse.etrice.generator.java/1.1.2/org.eclipse.etrice.generator.java-1.1.2.pom|HTTP/1.1|401|0
20180417073746|0|REQUEST|ip_xxx|non_authenticated_user|HEAD|/maven/org/eclipse/etrice/org.eclipse.etrice.generator.java/1.1.2/org.eclipse.etrice.generator.java-1.1.2.jar|HTTP/1.1|401|0

But when I re-use the exact same repository block from above to resolve the dependencies online, it works. The second request now contains the user apparently:

20180417073506|1|REQUEST|ip_xxx|non_authenticated_user|HEAD|/maven/org/eclipse/etrice/org.eclipse.etrice.generator.java/1.1.2/org.eclipse.etrice.generator.java-1.1.2.pom|HTTP/1.1|401|0
20180417073506|7|REQUEST|ip_xxx|user|HEAD|/maven/org/eclipse/etrice/org.eclipse.etrice.generator.java/1.1.2/org.eclipse.etrice.generator.java-1.1.2.pom|HTTP/1.1|200|0

Type of Ivy repository

Hello,

With Gradle 2.12 and IvyPot 0.3.7, unlike in https://github.com/ysb33r/ivypot-gradle-plugin#usage I have to specify the Ivy type to use it, like that :

repositories {
  ivy {
    url '/path/to/repo'
    layout 'ivy'
  }
}

Does IvyPot use an old default value?
Does the documentation need to be updated?

Thank you.

Documentation for Multi-project is out-of-date

https://github.com/ysb33r/ivypot-gradle-plugin#multi-project-support

A quick check of the source code indicates that the offlineRepoDir variable mentioned in the documentation does not exist in the source code.

$ grep -R offlineRepoDir *
README.adoc:        offlineRepoDir = "${rootProject.projectDir}/repo"

I suspect repoRoot may have been what was intended?

Add support for repositories.flatDir

syncRemoteRepositories {
  repositories {
    flatDir {
     // as for usual flatDir definitions
   }
}

0.3 Release Checklist

• gradleTest added?
• Integration test for jCenter ?
• Example for using repository inside initscript

When using two OfflineRepositorySync tasks, the build fails

Errors out with Overriding previous definition of ivy:settings with the id 'ivy.instance' is not allowed when using override-'notallowed'

Don't mix cache data with artifacts

Curretnly the cache data sit in the top folder of repoRoot. This should be written elsewhere.

Make default layout on disk to be 'gradle'

Current default layout is ivy. Make the default layout to be gradle.

Investigate implementation of metadatasupplier for org.ysb33r.gradle.ivypot.internal.IvyRepository

Initial implementation will just throw an exception. Please register your interest here, if this make sense for your use-case.

NOTE: Only needed when plugin is built with Gradle 4.0+

Transitive dependencies are not copied to local repository

Simplify multi-project support

This example by @Galmor shows the pain that one has to go through in 0.5 to load in dependencies from every subproject.

In the end of day the following code needs to be rolled-up into the plugin

ext {
  thisProject = project.name
  addConfigurationsFor = { Project forProj ->
    if( forProj != project ) {
      forProj.configurations.each { Configuration c->
        logger.info "${forProj.name}: ${c.name}"
        String configName = "${forProj.name}_${c.name}_deps"
        project.configurations.create configName
        project.syncRemoteRepositories.configurations configName
        c.dependencies.each { Dependency d ->
          if (!(d instanceof ProjectDependency)) {
            logger.info "Found ${d} in ${forProj.name}. Adding to ${configName}"
            project.dependencies.add configName, d
          }
        }
      }
      forProj.childProjects.each { String projName, Project p ->
        addConfigurationsFor(p)
      }
    }
  }
}

addConfigurationsFor(rootProject)

• Add method that will roll up all external module dependencies for all other subprojects
• Add method that will roll up all external module dependenices for on subproject
• Provide docs that describe a good practice for multi-project builds.

Unable to create task extending org.ysb33r.gradle.ivypot.OfflineRepositorySync

Hello,

I'm getting error Unable to load class org.ysb33r.gradle.ivypot.OfflineRepositorySync due to missing dependency org/gradle/api/internal/DynamicObject when trying to extend OfflineRepositorySync class using gradle 3.x. I have found that the solution would be to release a new version using gradle 3.x. Here is the source discussion https://discuss.gradle.org/t/missing-dependency-org-gradle-api-internal-dynamicobject-prevents-extending-plugin-task/19472/2

Thx for resolution.

Non-transitive requests are not honoured

If a dependency is marked like

compile 'org:artifact:1.2.3', {
  transitive = false
}

then ivyport still resolves the transitive dependencies. This setting should be honoured

Building offlline fails because it is unable to find aapt2 ( com.android.tools.build:aapt2:3.3.2-5309881)

I'm struggling to get the plugin to work with my project. My project is a simple test project with a single app. Running ./gradlew syncRemoteRepositories succeeds and creates a repo directory under my project (myapp/repo) but when I run ./gradlew assembleDebug --offline I see the error message below. The error is correct because the location does not exist in my offline repo - the only 'aapt2' folder is this one file:/myapp/repo/com.android.tools.build/aapt2-proto/0.3.1/

Any ideas on how I can fix this because I've been fighting with it for last few days. I've also pasted my build.gradle below

* What went wrong:
Could not resolve all files for configuration ':app:_internal_aapt2_binary'.
> Could not find com.android.tools.build:aapt2:3.3.2-5309881.
  Searched in the following locations:
    - file:/home/android/Android/SDK/extras/m2repository/com/android/tools/build/aapt2/3.3.2-5309881/aapt2-3.3.2-5309881.pom
    - file:/home/android/Android/SDK/extras/m2repository/com/android/tools/build/aapt2/3.3.2-5309881/aapt2-3.3.2-5309881-linux.jar
    - file:/home/android/Android/SDK/extras/google/m2repository/com/android/tools/build/aapt2/3.3.2-5309881/aapt2-3.3.2-5309881.pom
    - file:/home/android/Android/SDK/extras/google/m2repository/com/android/tools/build/aapt2/3.3.2-5309881/aapt2-3.3.2-5309881-linux.jar
    - file:/home/android/Android/SDK/extras/android/m2repository/com/android/tools/build/aapt2/3.3.2-5309881/aapt2-3.3.2-5309881.pom
    - file:/home/android/Android/SDK/extras/android/m2repository/com/android/tools/build/aapt2/3.3.2-5309881/aapt2-3.3.2-5309881-linux.jar
    - file:/myapp/repo/com.android.tools.build/aapt2/3.3.2-5309881/ivy-3.3.2-5309881.xml
    - file:/myapp/repo/com.android.tools.build/aapt2/3.3.2-5309881/aapt2-3.3.2-5309881-linux.jar
  Required by:
      project :app

// Top-level build file where you can add configuration options common to all sub-projects/modules.
buildscript {
    repositories {
        if(gradle.startParameter.isOffline()) {
            println "OFFLINE: using local cache: ${rootProject.projectDir}/repo"
            ivy { url file("${rootProject.projectDir}/repo").toURI() }
        } else {
            jcenter()
            google()
            mavenCentral()
            mavenLocal()
            maven { url "https://plugins.gradle.org/m2/" }
      }
    }


    dependencies {
        classpath 'com.android.tools.build:gradle:3.3.2'
        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files
    }
}

plugins {
    id "org.ysb33r.ivypot" version "0.9"
}

allprojects {
    repositories {

        if(gradle.startParameter.isOffline()) {
            println "OFFLINE: allprojects using local cache: ${rootProject.projectDir}/repo"
            ivy { url file("${rootProject.projectDir}/repo").toURI() }
        } else {
            jcenter()
            google()            
            mavenCentral()
            mavenLocal()
            maven { url "https://plugins.gradle.org/m2/" }
      }
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}

ext {
    thisProject = project.name
    addConfigurationsFor = { Project forProj ->
        if( forProj != project ) {
            forProj.configurations.each { Configuration c->
                logger.info "${forProj.name}: ${c.name}"
                String configName = "${forProj.name}_${c.name}_deps"
                project.configurations.create configName
                project.syncRemoteRepositories.configurations configName
                c.dependencies.each { Dependency d ->
                    logger.info "Found ${d} in ${forProj.name}. Adding to ${configName}"
                    project.dependencies.add configName, d
                }
            }
            forProj.childProjects.each { String projName, Project p ->
                addConfigurationsFor(p)
            }
        }
    }
}


addConfigurationsFor(rootProject)

syncRemoteRepositories {
    repoRoot = "${rootProject.projectDir}/repo"

    repositories {
            jcenter()
            google()
            mavenCentral()
            mavenLocal()
            maven { url "https://plugins.gradle.org/m2/" }
    }

    onlyIf { !gradle.startParameter.isOffline() }
    includeBuildScriptDependencies = true
}

Investigate use of the Artifact Query API

This Gist by Benjamin Muschko - https://gist.github.com/bmuschko/e4e04abca105c397145e - provides an idea to create a local Maven repo. It would be worthwhile to investigate to see if it will work within the contexct of Ivypot.

Crash in gradle 6.0.1 during configuration

Running under gradle 6.0.1 generates exception during configuration:

2019-12-08T11:36:12.744-0500 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Caused by: java.lang.UnsupportedOperationException 2019-12-08T11:36:12.744-0500 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] at java_util_Map$put.call(Unknown Source) 2019-12-08T11:36:12.744-0500 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] at org.ysb33r.gradle.ivypot.OfflineRepositorySync.<init>(OfflineRepositorySync.groovy:61) 2019-12-08T11:36:12.744-0500 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] at org.ysb33r.gradle.ivypot.OfflineRepositorySync_Decorated.<init>(Unknown Source) 2019-12-08T11:36:12.744-0500 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] at org.gradle.internal.instantiation.generator.AsmBackedClassGenerator.newInstance(AsmBackedClassGenerator.java:200) 2019-12-08T11:36:12.744-0500 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] at org.gradle.internal.instantiation.generator.AbstractClassGenerator$GeneratedClassImpl$GeneratedConstructorImpl.newInstance(AbstractClassGenerator.java:402) 2019-12-08T11:36:12.744-0500 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] at org.gradle.internal.instantiation.generator.DependencyInjectingInstantiator.doCreate(DependencyInjectingInstantiator.java:64) 2019-12-08T11:36:12.744-0500 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] at org.gradle.internal.instantiation.generator.DependencyInjectingInstantiator.newInstance(DependencyInjectingInstantiator.java:55) 2019-12-08T11:36:12.744-0500 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] at org.gradle.api.internal.project.taskfactory.TaskFactory$1.call(TaskFactory.java:72) 2019-12-08T11:36:12.744-0500 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] ... 169 more 2019-12-08T11:36:12.744-0500 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]

this is caused by the following:
inputs.properties.put( ... )

Guess: inputs.properties is no longer a writable map

Add support for includeBuildScriptDependencies

Only add buildscript.dependencies.classpath when includeBuildScriptDependencies is true

Order of repositories are not honoured

When writing something like

syncRemoteRepositories {
    repositories {
        mavenCentral()
        jcenter()
    }
}

then the Bintray entry is written before the Maven Central entry in ivysettings.xml

Added repositories.ivy support

Need to be able to do

syncRemoteRepositories {
  repositories {
    ivy {
     // as for usual Ivy repo definitions
   }
}

Cannot resolve artifacts from Maven repositories that use 302 redirects

This is an inherent issue in Ivy itself. I am not sure whether it can be solved at all using Ivy itself.

Typo in code, or desired behaviour?

Should it be assign operation or substaction?

instantiator - DirectInstantiator.INSTANCE

https://github.com/ysb33r/ivypot-gradle-plugin/blob/master/src/main/groovy/org/ysb33r/gradle/ivypot/OfflineRepositorySync.groovy#L290

Investigate implementation of AuthencationContainer for org.ysb33r.gradle.ivypot.internal.RepositoryTraits

The initial implementaiton will just throw an exception. If you find a use-case please register your interest here.

NOTE: Only needed when plugin is built with Gradle 4.0+

Error when trying to resolve dependency with null values

I'm not sure where this dependency is coming from but the ivypot plugin is throwing an error while trying to resolve a null dependency. This is the error:

:syncRemoteRepositories
[syncRemoteRepositoriesConfigure] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/ ::
[syncRemoteRepositoriesConfigure] :: loading settings :: file = /root/projects/cognizant-ipas/ML_DHF_IPAS/build/tmp/syncRemoteRepositories/ivysettings.xml
[syncRemoteRepositoriesResolve] ivy.instance reference an ivy:settings defined in an other classloader.  An new default one will be used in this project.
[syncRemoteRepositoriesResolve] :: problems summary ::
[syncRemoteRepositoriesResolve] :::: WARNINGS
[syncRemoteRepositoriesResolve]                 module not found: null#unspecified;null
[syncRemoteRepositoriesResolve]         ==== BintrayJCenter: tried
[syncRemoteRepositoriesResolve]           https://jcenter.bintray.com/null/unspecified/null/unspecified-null.pom
[syncRemoteRepositoriesResolve]           -- artifact null#unspecified;null!unspecified.jar:
[syncRemoteRepositoriesResolve]           https://jcenter.bintray.com/null/unspecified/null/unspecified-null.jar
[syncRemoteRepositoriesResolve]         ==== MavenRepo: tried
[syncRemoteRepositoriesResolve]           http://repo1.maven.org/maven2/null/unspecified/null/unspecified-null.pom
[syncRemoteRepositoriesResolve]           -- artifact null#unspecified;null!unspecified.jar:
[syncRemoteRepositoriesResolve]           http://repo1.maven.org/maven2/null/unspecified/null/unspecified-null.jar
[syncRemoteRepositoriesResolve]         ==== maven: tried
[syncRemoteRepositoriesResolve]           https://plugins.gradle.org/m2/null/unspecified/null/unspecified-null.pom
[syncRemoteRepositoriesResolve]           -- artifact null#unspecified;null!unspecified.jar:
[syncRemoteRepositoriesResolve]           https://plugins.gradle.org/m2/null/unspecified/null/unspecified-null.jar
[syncRemoteRepositoriesResolve]         ==== maven2: tried
[syncRemoteRepositoriesResolve]           http://developer.marklogic.com/maven2/null/unspecified/null/unspecified-null.pom
[syncRemoteRepositoriesResolve]           -- artifact null#unspecified;null!unspecified.jar:
[syncRemoteRepositoriesResolve]                 ::::::::::::::::::::::::::::::::::::::::::::::
[syncRemoteRepositoriesResolve]                 ::          UNRESOLVED DEPENDENCIES         ::
[syncRemoteRepositoriesResolve]                 ::::::::::::::::::::::::::::::::::::::::::::::
[syncRemoteRepositoriesResolve]                 :: null#unspecified;null: not found
[syncRemoteRepositoriesResolve]                 ::::::::::::::::::::::::::::::::::::::::::::::
:syncRemoteRepositories FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':syncRemoteRepositories'.
> impossible to resolve dependencies:
        resolve failed - see output for details

This it the exception

Caused by: : impossible to resolve dependencies:
        resolve failed - see output for details
        at org.apache.ivy.ant.IvyResolve.doExecute(IvyResolve.java:318)
        at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293)
        at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
        at org.ysb33r.gradle.ivypot.OfflineRepositorySync.ivyInstall(OfflineRepositorySync.groovy:208)
        at org.ysb33r.gradle.ivypot.OfflineRepositorySync$_exec_closure4.doCall(OfflineRepositorySync.groovy:193)
        at org.ysb33r.gradle.ivypot.OfflineRepositorySync.exec(OfflineRepositorySync.groovy:192)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:75)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.doExecute(DefaultTaskClassInfoStore.java:136)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:129)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:118)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:623)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:606)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
        ... 68 more
Caused by: resolve failed - see output for details
        at org.apache.ivy.ant.IvyResolve.doExecute(IvyResolve.java:251)
        ... 82 more

Specifying ivy repositories with only a URL does not work

The workaround is currently to also supply an artifactUrls line with exacylt the same URL

kotlinx-html-jvm from jcenter causes the plugin to fail

The following build file will cause the error below:

plugins {
    id 'org.ysb33r.ivypot' version '0.6'
}
apply plugin: 'java'
syncRemoteRepositories {
    repoRoot 'c:\\temp\\test'
    repositories {
        jcenter()
    }
}

dependencies {
    compile("org.jetbrains.kotlinx:kotlinx-html-jvm:0.6.8")
}

[syncRemoteRepositoriesConfigure] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/ :: [syncRemoteRepositoriesConfigure] :: loading settings :: file = C:\percentile\dev\Scratch\build\tmp\syncRemoteRepositories\ivysettings.xml [syncRemoteRepositoriesResolve] ivy.instance reference an ivy:settings defined in an other classloader. An new default one will be used in this project. [syncRemoteRepositoriesResolve] :: problems summary :: [syncRemoteRepositoriesResolve] :::: WARNINGS [syncRemoteRepositoriesResolve] module not found: org.jetbrains.kotlin#kotlin-stdlib;${kotlin.version} [syncRemoteRepositoriesResolve] ==== BintrayJCenter: tried [syncRemoteRepositoriesResolve] https://jcenter.bintray.com/org/jetbrains/kotlin/kotlin-stdlib/${kotlin.version}/kotlin-stdlib-${kotlin.version}.pom [syncRemoteRepositoriesResolve] -- artifact org.jetbrains.kotlin#kotlin-stdlib;${kotlin.version}!kotlin-stdlib.jar: [syncRemoteRepositoriesResolve] https://jcenter.bintray.com/org/jetbrains/kotlin/kotlin-stdlib/${kotlin.version}/kotlin-stdlib-${kotlin.version}.jar [syncRemoteRepositoriesResolve] :::::::::::::::::::::::::::::::::::::::::::::: [syncRemoteRepositoriesResolve] :: UNRESOLVED DEPENDENCIES :: [syncRemoteRepositoriesResolve] :::::::::::::::::::::::::::::::::::::::::::::: [syncRemoteRepositoriesResolve] :: org.jetbrains.kotlin#kotlin-stdlib;${kotlin.version}: not found [syncRemoteRepositoriesResolve] :::::::::::::::::::::::::::::::::::::::::::::: [syncRemoteRepositoriesResolve]

Incorrect declaration of task property

This code has no effect:

        inputs.properties.put('project configurations', {
            this.projectConfigurations
        })

https://github.com/ysb33r/ivypot-gradle-plugin/blob/master/src/main/groovy/org/ysb33r/gradle/ivypot/OfflineRepositorySync.groovy#L60-L62

Please use this instead:

        inputs.property('project configurations', {
            this.projectConfigurations
        })

Use in build.gradle.kts (i.e. Kotlin-DSL for Gradle)

This is a multi-project.

$ gw clean --stacktrace
Using gradle at '/home/fred/git/face_applications/gradlew' to run buildfile '/home/fred/git/face_applications/build.gradle.kts':

Starting a Gradle Daemon (subsequent builds will be faster)
Configuration on demand is an incubating feature.

FAILURE: Build failed with an exception.

* Where:
Build file '/home/fred/git/face_applications/build.gradle.kts' line: 7

* What went wrong:
An exception occurred applying plugin request [id: 'org.ysb33r.ivypot', version: '0.10']
> Failed to apply plugin [id 'org.ysb33r.ivypot']
   > Could not create task ':syncRemoteRepositories'.
      > Could not create task of type 'OfflineRepositorySync'.
         > java.lang.UnsupportedOperationException (no error message)

settings.gradle.kts

pluginManagement {
    plugins {
        kotlin("multiplatform").version("1.3.61")
        kotlin("jvm").version("1.3.61")
        kotlin("kapt").version("1.3.61")

        id("org.gradle.kotlin.embedded-kotlin").version("1.3.3")
        id("org.gradle.kotlin.kotlin-dsl").version("1.3.3")
        id("org.ysb33r.ivypot").version("0.10")
    }
    repositories {
        gradlePluginPortal()
        mavenCentral()
        jcenter()
    }
}
...
}

build.gradle.kts

plugins {
    id("java-library")
    id("maven-publish")
    kotlin("jvm") version "1.3.41"
    id("distribution")
    id ( "org.ysb33r.ivypot")
}
...
allprojects {
...
    tasks {
        named<org.ysb33r.gradle.ivypot.OfflineRepositorySync>("syncRemoteRepositories") {
            repoRoot("${buildDir}/offlineRepo")
            repositories {
                jcenter()
            }
        }
    }
...
}

sync/build.gradle.kts

plugins {
    id ( "org.ysb33r.ivypot")
}
val offlineRepoDir by extra { "${rootProject.projectDir}/repo" }
...
tasks {
    named<org.ysb33r.gradle.ivypot.OfflineRepositorySync>("syncRemoteRepositories") {
        val offlineRepoDir : String? by extra

        addAllProjects()
...
}
...

No support for dependency classifier

I'd like to have this plug-in fetch artifacts other than jars. For example the following line causes problems. I am trying to download a zip distribution of apache karaf which is a dependency of my project.

dependencies {
karaf 'org.apache.karaf:apache-karaf:4.2.2@zip'
}

Output:
[syncRemoteRepositoriesResolve] :::: WARNINGS
[syncRemoteRepositoriesResolve] module not found: org.apache.karaf#apache-karaf;4.2.2
[syncRemoteRepositoriesResolve] ==== MavenRepo: tried
[syncRemoteRepositoriesResolve] http://repo1.maven.org/maven2/org/apache/karaf/apache-karaf/4.2.2/apache-karaf-4.2.2.pom
[syncRemoteRepositoriesResolve] -- artifact org.apache.karaf#apache-karaf;4.2.2!apache-karaf.jar:
[syncRemoteRepositoriesResolve] http://repo1.maven.org/maven2/org/apache/karaf/apache-karaf/4.2.2/apache-karaf-4.2.2.jar
[syncRemoteRepositoriesResolve] ::::::::::::::::::::::::::::::::::::::::::::::

Allow for option to change the patterns used within target repository

Ivypot fails to resolve the correct version of a plugin

Ivypot resolves the following dependency:
org.sonatype.sisu/sisu-guice/3.1.3/sisu-guice-3.1.3-no_aop.jar

and then when attempting to build with the offline repo Gradle complains about this version missing:
org.sonatype.sisu/sisu-guice/3.1.0/sisu-guice-3.1.0-no_aop.jar

Code snapshot:
https://github.com/lukecwik/incubator-beam/tree/ivypot

Repro (after cloning the above repo):

./gradlew :sync:syncRemoteRepositories
./gradlew --offline