Giter Site home page Giter Site logo

ismaeldivita / change-tracker-plugin Goto Github PK

View Code? Open in Web Editor NEW
112.0 6.0 5.0 662 KB

A Gradle plugin to help analyse the dependency between modules and run tasks only on modules impacted by specific set of changes.

License: MIT License

Kotlin 100.00%
gradle-plugin gradle android modules test

change-tracker-plugin's Introduction

Change Tracker Plugin

A Gradle plugin to help analyse the dependency between modules and run tasks only on modules impacted by specific set of changes.

The idea behind this plugin is to optimize running verification tasks on large modularized projects. There is no need to run these tasks on all modules every time we apply a change. Instead, we could run them only on modules affected by the changes and their dependents.


Take the following project structure as an example:



If you apply changes to the :profile-service module, you only need to run your verification tasks on :profile-service:profile-feature, and :app. It's safe to skip the tasks on all the other modules since they don't depend on :profile-service and can't be affected by these changes.

Setup

Apply the plugin and the configuration to your root project build.gradle

plugins {
    id 'com.ismaeldivita.changetracker' version '0.7.4'
}

// ...

changeTracker {
    tasks = ['lint','testDebugUnitTest']
    whitelist = [':app']
    blacklist = [':network',':featureA']
    reevaluate = [':sharedTest']
    branch = "master"
    remote = "origin"
}
  • tasks: List of the tasks the plugin will need to create.
  • branch: Name of the branch that should be used to extract the diff.
  • whitelist (optional): List of modules that should always run.
  • blacklist (optional): List of modules that should never run.
  • reevaluate (optional): List of modules that will trigger the task for all modules
  • remote (optional): Name of the remote repository.

Usage

The plugin will generate new tasks on the root project for each task provided on the configuration with the following name ${taskName}ChangedModules.

Taking as an example the configuration above the plugin will generate two new tasks lintChangedModules and testDebugUnitTestChangedModules.

To run your task:

./gradlew testDebugUnitTestChangedModules

You can override the default branch used for the comparison when running your command. This is useful when you're using the plugin on pull requests and each pull request may have different base branches.

./gradlew testDebugUnitTestChangedModules -Pbranch=dev

Notes

  • This plugin will assume you use GIT as your VCS.
  • Any changes to the root project or buildSrc will trigger the task for all modules.
  • Test configurations will not be tracked since dependency cycles could be created for test purposes and this is not supported by this plugin. If you have a shared test library project and want to trigger the tasks on their dependents check the reevaluate configuration.

License

MIT License

Copyright (c) 2019 Ismael Di Vita

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

change-tracker-plugin's People

Contributors

ismaeldivita avatar vtuan10 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

change-tracker-plugin's Issues

Compatibility with gradle 7

After updating the gradel version of my project to gradle 7, build using tasks from this plugin fail with the following message:

FAILURE: Build failed with an exception.

* What went wrong:
A problem was found with the configuration of task ':changedModules' (type 'ChangedModulesTask').
  - Type 'com.ismaeldivita.changetracker.ChangedModulesTask' property 'group' is missing an input or output annotation.
    
    Reason: A property without annotation isn't considered during up-to-date checking.
    
    Possible solutions:
      1. Add an input or output annotation.
      2. Mark it as @Internal.
    
    Please refer to https://docs.gradle.org/7.0.1/userguide/validation_problems.html#missing_annotation for more details about this problem.

Affected Projects listing all modules

Hi,

Thanks for this project, I think it's going to make a huge difference to our CI environment.

At the minute running changedModules lists all the modules in the app as affected.
Is there anyway to debug what is going on? I can see logger.isInfoEnabled but not sure how to toggle that.

The plugin always detekt all modules as changes

I've integrated the plugin and has problem in the next condition:

  • all changes are committed
  • current branch is develop
  • Configuration:
changeTracker {
    // List of tasks the plugin will need to create
    tasks = ['lintDebug','testDebugUnitTest', 'detekt']

    // List of modules that should always run
    whitelist = []

    // List of modules that should never run.
    blacklist = []

    // List of modules that will trigger the task for all modules
    reevaluate = []

    // Name of the branch that should compare to the current one to extract the diff
    // Branch can be specified dynamically via Gradle property 'branch'. E.g. -Pbranch=dev
    branch = "develop"
}

Task changedModules always return all modules as changed. What is incorrect?

Changed 3 modules and got java.lang.StackOverflowError

I've changed 3 modules and got such a problem. Could @ismaeldivita you take a look?

> Task :changedModules FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':changedModules'.
> java.lang.StackOverflowError (no error message)
* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':changedModules'.
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:207)
	at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:263)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:205)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:186)
	at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:114)
	at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
	at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:62)
	at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
	at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
	at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
	at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
	at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:41)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:370)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:357)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:350)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:336)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
Caused by: java.lang.StackOverflowError
	at kotlin.collections.CollectionsKt__IterablesKt.collectionSizeOrNull(Iterables.kt:33)
	at kotlin.collections.SetsKt___SetsKt.plus(_Sets.kt:119)
	at com.ismaeldivita.changetracker.project.ProjectDependents.getAllDependents(ProjectDependents.kt:16)
	at com.ismaeldivita.changetracker.project.ProjectDependents.getAllDependents(ProjectDependents.kt:16)
	at com.ismaeldivita.changetracker.project.ProjectDependents.getAllDependents(ProjectDependents.kt:16)
	at com.ismaeldivita.changetracker.project.ProjectDependents.getAllDependents(ProjectDependents.kt:16)
	at com.ismaeldivita.changetracker.project.ProjectDependents.getAllDependents(ProjectDependents.kt:16)
	at com.ismaeldivita.changetracker.project.ProjectDependents.getAllDependents(ProjectDependents.kt:16)
	at com.ismaeldivita.changetracker.project.ProjectDependents.getAllDependents(ProjectDependents.kt:16)

Logging diffs

Hi,

No rush at all, but if you do another release, do you think it would be possible to add more logging for the diffs found?
We're having trouble identifying what is changing in our root gradle in CI

cheers

Branch not found on CI

I have a master branch and I'm trying to test this plugin on the CI. I branched off of master into a new branch called a and installed the plugin in branch a.

in branch a:

changeTracker {
    tasks = ['testDebugUnitTest']
    branch = "a"
}

Then I branched off of branch a into branch b to test the changedModules task. In this branch I just made a small change in one of the modules. I pushed both branches and triggered a CI build on branch b and ran the command gradlew changedModules on CI. The branches look like the following:

------  master
             \
               a ----- commit (install the plugin)
                            \
                             b ------ commit (minor change in a module)

The CI build failed with the following error:

Caused by: org.gradle.api.GradleException: Branch  a not found

I've tried adding remote in the config block and tried other variations of the branch name like origin/a or remotes/origin/a, etc, but nothing works. Locally, if I'm on branch b and run ./gradlew changedModules, then it runs fine and prints out a nice report. Not sure what's going on in the CI. Somehow the branch name is the issue. Any ideas?

Let reevaluate option take paths

Sometimes there are buildsystem, config or buildSrc directories (not modules) that contains dependency versions for all libs. Changes in them should behave like the reevaluate option. Having module names in reevaluate is great but it'd be awesome if we can also include paths there as well.

Something like this https://github.com/dropbox/AffectedModuleDetector#installation

reevaluate = ["config/", ":core", ":sharedTest"]

They could be different options as well

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.