Giter Site home page Giter Site logo

jenkinsci / custom-distribution-service Goto Github PK

View Code? Open in Web Editor NEW
21.0 8.0 16.0 2.07 MB

Archived in https://github.com/jenkins-infra/helpdesk/issues/3126. The project wasn't completed. The main idea behind the project is to build a customizable jenkins distribution service :package: that could be used to build tailor-made jenkins distributions

Home Page: https://customize.jenkins.io/pluginList

License: MIT License

HTML 0.78% JavaScript 35.71% Java 55.78% Dockerfile 2.03% Shell 0.24% SCSS 5.46%
service packaging jenkins spring-boot custom

custom-distribution-service's Introduction

Jenkins Custom Distribution Service

Gitter Codacy Badge GitHub license

Overview

The main idea behind the project is to build a customizable Jenkins distribution service that could be used to build tailor-made Jenkins distributions. The service would provide users with a simple interface to select the configurations they want to build the instance with eg: plugins, authorization matrices etc. Furthermore it would include a section for sharing community created distros so that users can find and download already built Jenkins war/configuration files to use out of the box.

QuickStart

You can spin up the entire web application using a docker compose file command: The front-end runs on localhost:3001 and the backend runs on localhost:8080.

Build the containers:

docker-compose up -d --build

Run the containers:

docker-compose up

Run without docker

Prerequisites

a) maven

b) npm

You can run the backend spring boot server from the root directory using:

mvn spring-boot:run

You can run the front end using the following:

cd frontend/
npm install
npm start

Environment Variable configuration

We currently have two environment variables

a) REACT_APP_API_URL: This variable points to the base API_URL to which the service makes calls to.If you decide to run the backend on another port this variable should be set. The default value is set to http://localhost:8080

b) REACT_APP_GITHUB_COMMUNITY_URL: This variable points to the URL at which you would like to store your community configurations on github. It needs to be in the format of: https://api.github.com/repos/{repo-owner}/{repo-name}/contents/{path-to-folder}

The default configuration is:

https://api.github.com/repos/sladyn98/custom-distribution-service-community-configurations/contents/configurations

How to change ports and run without Docker

In order to change the ports on which spring-boot runs you need to execute the following command. For example if you need to run the spring boot server on port 8081 these are the commands you would want to run.

  • For UNIX
SERVER_PORT=8081 mvn spring-boot:run

Once you have started the spring boot server the next thing is to configure the front-end environment file so that the react server knows where to find the backend server.

  • The file that needs to be changed is the .env inside the frontend folder.

  • Inside this file this line needs to be changed REACT_APP_API_URL=INSERT_NEW_PORT_HERE for eg: If the backend is running on port 8081

REACT_APP_API_URL=http://localhost:8081/

How to change ports and run with Docker

In order to change the docker port the following file need to changed

  • Inside the docker-compose.yml the line:
ports:
      - "8080:8080" # Forward the exposed port 8080 on the container to port 8080 on the host machine

needs to be changed to the port you want to run it at.

Eg: The port on the left indicates the port that needs to be exposed on the host machine, so it needs to be changed according to our requirement. So in order to run on port 8081 we need to make the following change.

 ports:
      - "8081:8080"

Once you have started the spring boot server the next thing is to configure the front-end environment file so that the react server knows where to find the backend server.

  • The file that needs to be changed is the .env.docker inside the frontend folder.

  • Inside this file this line needs to be changed REACT_APP_API_URL=INSERT_NEW_PORT_HERE for eg: If the backend is running on port 8081

REACT_APP_API_URL=http://localhost:8081/

How to change Community-Configurations URL

For Docker Environment

  • The file that needs to be changed is the .env.docker inside the frontend folder.

  • Inside this file this line needs to be changed REACT_APP_GITHUB_COMMUNITY_URL=ENTER_URL_HERE

For Non-Docker Environment

  • The file that needs to be changed is the .env inside the frontend folder.

  • Inside this file this line needs to be changed REACT_APP_GITHUB_COMMUNITY_URL=ENTER_URL_HERE

Documentation

You can find more documentation about how to use the service here:

Useful Links

a) Project page

b) GSoC Proposal

c) Design Document

custom-distribution-service's People

Contributors

baymac avatar dependabot[bot] avatar dheerajodha avatar halkeye avatar kwhetstone avatar linuxsuren avatar markewaite avatar martinda avatar sladyn98 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

custom-distribution-service's Issues

Add ability to add plugins to the custom configuration

This is one of the main components for the service that once the user adds a plugin to the configuration we would be doing the following

  • Pick up the latest version from the update center

  • Cache this data as the user keeps adding plugins

Cannot read property "bundle" of null

URL: http://localhost:3001/generatePackage

Error:

×
TypeError: Cannot read property 'bundle' of null
editor.componentDidMount
src/app/src/components/PackageGeneration/Editor.js:39
  36 | 
  37 |  componentDidMount() {
  38 |   this.setState({code: localStorage.getItem("packageConfigYAML")})
> 39 |   this.setState({title: JSON.parse(localStorage.getItem("packageConfigJSON"))["bundle"]["title"]})
     | ^  40 |   this.setState({description: JSON.parse(localStorage.getItem("packageConfigJSON"))["bundle"]["description"]})
  41 | }
  42 | 
View compiled
▼ 19 stack frames were expanded.
commitLifeCycles
src/app/node_modules/react-dom/cjs/react-dom.development.js:19814
commitLayoutEffects
src/app/node_modules/react-dom/cjs/react-dom.development.js:22803
HTMLUnknownElement.callCallback
src/app/node_modules/react-dom/cjs/react-dom.development.js:188
invokeGuardedCallbackDev
src/app/node_modules/react-dom/cjs/react-dom.development.js:237
invokeGuardedCallback
src/app/node_modules/react-dom/cjs/react-dom.development.js:292
commitRootImpl
src/app/node_modules/react-dom/cjs/react-dom.development.js:22541
unstable_runWithPriority
src/app/node_modules/react-dom/node_modules/scheduler/cjs/scheduler.development.js:653
runWithPriority$1
src/app/node_modules/react-dom/cjs/react-dom.development.js:11039
commitRoot
src/app/node_modules/react-dom/cjs/react-dom.development.js:22381
finishSyncRender
src/app/node_modules/react-dom/cjs/react-dom.development.js:21807
performSyncWorkOnRoot
src/app/node_modules/react-dom/cjs/react-dom.development.js:21793
(anonymous function)
src/app/node_modules/react-dom/cjs/react-dom.development.js:11089
unstable_runWithPriority
src/app/node_modules/react-dom/node_modules/scheduler/cjs/scheduler.development.js:653
runWithPriority$1
src/app/node_modules/react-dom/cjs/react-dom.development.js:11039
flushSyncCallbackQueueImpl
src/app/node_modules/react-dom/cjs/react-dom.development.js:11084
flushSyncCallbackQueue
src/app/node_modules/react-dom/cjs/react-dom.development.js:11072
discreteUpdates$1
src/app/node_modules/react-dom/cjs/react-dom.development.js:21893
discreteUpdates
src/app/node_modules/react-dom/cjs/react-dom.development.js:806
dispatchDiscreteEvent
src/app/node_modules/react-dom/cjs/react-dom.development.js:4168
▲ 19 stack frames were expanded.
This screen is visible only in development. It will not appear if the app crashes in production.
Open your browser’s developer console to further inspect this error.  Click the 'X' or hit ESC to dismiss this message.

Steps to reproduce:

git checkout cb2e9ed6193698e059375a592bbde2f66651a0d7
mvn package
docker-compose up -d --build
docker-compose up

Visit the localhost:3001:

  • Click Plugins
  • Select a few plugins
  • Click Package Generation (do not click submit plugins)

I suspect I need to click "Submit plugins", but users do unpredictable things, and no matter what, the app should not crash.

Support Pull Request Creation for community configurations

Some of the tasks we need to go through before we start with this feature are:

  • Choose a repository to upload these configurations.

  • Repository review discussion
    a) Who reviews the code that gets merged.
    b) Guidelines (What serves as a community distribution to get that particular configuration merged in)

  • The decision to use a bot i.e Jenkins Custom Distribution Service bot to make the pull requests because we would need some sort of access token to make a pull request to the repository.

Add feature to search plugins.

Search bar to search plugins, I am assuming this would be happening on the client side, but would require some more research.

Download community config file and corresponding war file

Once the service has pulled in the configuration files we would want to be able to download those configuration file and also generate the corresponding war files. This should be a case of connecting the separate features on the front-end, since the functionalities for doing the heavy lifting have already been completed.

Dynamically show the list of plugins as they are selected

I need to select multiple plugins, but I have so many of them that I do not remember which ones I clicked. There is no visual feedback of the plugins I have picked. Is it possible to dynamically show the list of plugins as they are selected, and before they are submitted?

Prototype for JCasC Creation

Conduct more research maybe on how we can integrate JCasC into this, during the discussion for phase 1 Rick suggested that we keep this as a separate project. Needs more research in my opinion

User Docs + Developer Docs Guide

  • Purpose of the tool: Target Audience and why would you want to use it

  • How to use the tool

  • Architecture discussion

  • How to create custom packages (Batch Creating requirements)

Fix name in pom.xml

The name is currently Custom-Jenkins-Distribution service and should be custom-distribution service.

Add linter to React code

For standardized code, it's good practice to use a linter. You want to use ESLint, and extend the standard template. This not only keeps your coding consistent in the project, it keeps other commits consistent too.

You also will have to add jest and es6 to the environment. Pick whatever coding spaces, indents, and other style conventions

After adding your linter, make it a yarn task so you can run it automatically though GitHub actions, and fix locally before you deploy to the system. I like to use the following in my scripts section of package.json:
"lint": "eslint --ext .ts,.js --ignore-path .gitignore .",
and
"lint-fix": "eslint --fix --ext .ts,.js --ignore-path .gitignore .",

Hosting the Service

The service is now self-hosted do we want to proceed with hosting on jenkins infra ?

Add Code coverage to the repository

Would be good to have code coverage on our repository, likely jacoco so I will try adding it to the pipeline step. Would love to hear more suggestions

Fix Docker Compose

The Docker Compose currently throws which needs to be fixed :
org.springframework.beans.factory.BeanDefinitionStoreException: Failed to parse configuration class [com.org.jenkins.custom.jenkins.distribution.service.CustomJenkinsDistributionServiceApplication]; nested exception is java.io.FileNotFoundException: class path resource [com/org/jenkins/Custom/Jenkins/Distribution/Service/UpdateCenterService.class] cannot be opened because it does not exist

Unit - Testing for all features

Write a test plan. It should cover the list of tests ( Unit and Integration Tests)

a) Services need testing.

b) War generator needs testing (error in maven surefire)

c) Front - End Tests

d) Test the dockerfile (Test Containers) : Stretch Goal

Cleanup Tasks

a) Unused variables

b) Generate War file errors should be properly handled if the configuration file format is incorrect

Compulsory fields for packager-config.yml

Some fields could be optional while generating the packager-config.yml. It would be a good idea to make some fields mandatory and make some optional if they are not required in the UI form.
Examples:

  • System Settings - Might not be required

  • Dockerfile Settings - Might not be required

[EPIC] Render home page

  • Render individual plugin cards [#6]

  • Add feature to search plugins [#8]

  • Add feature to add plugins to the custom configuration [#10]

Pull community configurations from the main repository

The service should be able to pull in the configurations from the main repo and render the front end using that response. The raw github api is very powerful in that regard, so I guess if we are using the Github api this should be fairly simple. Since the configurations would just be config-name.yml we could pick the configuration name easily.

Add feature to filter plugins.

We could go with something similar to the plugin site in maybe building the ability to filter plugins based on most relevant, maybe most used etc.

[EPIC] Implement Custom War Generation

  • Implement generation of packager-config.yml

  • Implement Editor to show the generated packager-config.yml

  • Implement generation of WAR using CWP libraries

  • Implement ability to download generated WAR package.

Update Readme

  • Add Project page links [jenkins.io]

  • Add gitter chat link and CI badges.

  • Add design doc and proposal links

  • Add a quick How to Run guide

Make the docker-compose be able to run without having to build components locally

The Dockerfile for the backend requires that you build the app locally before being able to use the tool. This means you'll have to update the backend to be built as part of an intermediate container. The frontend container should also be built like this too so people don't have to install node and other tools simply to run a tool.

Allow users to use a custom update center

Eventually we want to be able to support users using their own custom update centers, this could be done using a config file where the user enters the custom URL and the service can just pull in the URL. The current implementation just makes use of the static custom update center URL.This ticket has been created in response to this [comment]
#31 (review)

Prepare for Jenkins Phase 1 Presentation

As part of the Jenkins GSoC milestones, there's a series of presentations and short demos. It's a great time to talk about what your project is, what you've accomplished during coding phase 1. If there's a lot of research you had to do for the project, include that in this phase 1 presentation too. We can go over this on the Tuesday meeting before the demo to give pointers and give you a chance to practice.

Note: this is something we'll copy to all other milestones, and is one of the last tasks for coding phase 1.

What you need:

  • Create a short slide presentation to give about your work in Phase 1
  • Prepare a demo of the custom distribution service as it exists at the end of Phase 1
  • Optional: blog post about GSoC experience
  • Share the demo with interested stakeholders

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.