Giter Site home page Giter Site logo

renaudcerrato / appengine-maven-repository Goto Github PK

View Code? Open in Web Editor NEW
213.0 17.0 54.0 154 KB

Free Private Maven repositories hosted on Google App-Engine, backed by Google Cloud Storage and deployed in less than 5 minutes.

Home Page: https://medium.com/@renaudcerrato/free-private-maven-repositories-on-google-app-engine-in-5-minutes-a55b3388e1fd

Java 98.13% Mustache 1.87%
maven jersey2 appengine-java google-cloud-storage appengine

appengine-maven-repository's Introduction

appengine-maven-repository

Private Maven repositories hosted on Google App-Engine, backed by Google Cloud Storage, supporting HTTP Basic authentication and minimalistic user access control deployed in less than 5 minutes.

Why ?

Private Maven repositories shouldn't cost you an arm and a leg, nor requires you to become a Linux Sys-Admin to setup, and should ideally be zero maintenance and costs nothing.

Thanks to Google App-Engine's free quotas, you'll benefits for free of:

  • 5GB of storage
  • 1GB of daily incoming bandwidth
  • 1GB of daily outgoing bandwidth
  • 20,000+ storage ops per day

Moreover, no credit card is required to benefits of the quotas above.

Installation

Prerequisites

Create a new Project

First of all, you'll need to go to your Google Cloud console to create a new App Engine application:

As soon as your project is created, a default Google Cloud storage bucket has been automatically created for you which provides the first 5GB of storage for free.

Setup the Google Cloud SDK

Follow the official documentation to install the latest Google Cloud SDK. As a shorthand, you'll find below the Ubuntu/Debian instructions:

$ export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)"
$ echo "deb http://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
$ curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
$ sudo apt-get update && sudo apt-get install google-cloud-sdk

Do not forget to install the app-engine-java component. If you installed the Google Cloud SDK using the instructions above:

$ sudo apt-get install google-cloud-sdk-app-engine-java

As a last step, configure the gcloud command line environment and select your newly created App Engine project when requested to do so:

$ gcloud init
$ gcloud auth application-default login

Configuration

Clone (or download) the source code:

$ git clone https://github.com/renaudcerrato/appengine-maven-repository.git

Update WEB-INF/users.txt to declare users, passwords and permissions:

# That file declares your users - using basic authentication.
# Minimalistic access control is provided through the following permissions: write, read, or list.
# Syntax is:
# <username>:<password>:<permission>
# (use '*' as username and password to declare anonymous users)
admin:l33t:write
john:j123:read
donald:coolpw:read
guest:guest:list

The list permission allows users to list the content of the repository but prohibits downloads. The read permission allows downloads (and implies list). The write permission allows uploads (and implies read).

Anonymous users are supported by using "*" for both username and password. For example, *:*:read will allow anonymous read access.

Deployment

Once you're ready to go live, just push the application to Google App-Engine:

$ cd appengine-maven-repository
$ ./gradlew appengineDeploy

And voilà! Your private Maven repository is hosted and running at the following address:

https://<your-project-id>.appspot.com

Usage

You'll find some usage examples in the examples folder. There's absolutely no extra steps required to fetch and/or deploy Maven artifacts to your repository: simply use your favorite Maven tools as you're used to do.

An example deploying artifacts using the maven plugin for Gradle:

apply plugin: 'java'
apply plugin: 'maven'

...

uploadArchives {
    repositories {
        mavenDeployer {
            repository(url: "https://<your-project-id>.appspot.com") {
                authentication(userName: "admin", password: "s3curepa55w0rd")
            }
            pom.version = "1.0.0"
            pom.artifactId = "test"
            pom.groupId = "com.example"
        }
    }
}

Using the above plugin, deploying artifacts to your repository is as simple as:

$ ./gradlew upload

In the other way, accessing password protected Maven repositories using Gradle only requires you to specify the credentials closure:

repositories {
    ...
    maven {
        credentials {
            username 'user'
            password 'YouCantGuess'
        }
        url "https://<your-project-id>.appspot.com"
    }
}

Ensure you do NOT commit credentials with your code. With Gradle, you can achieve this by amending the above examples using the approach specified here of moving your creds to ~/.gradle/gradle.properties and only referring to the variable names within your build.

Limitations

Google App-Engine HTTP requests are limited to 32MB - and thus, any artifacts above that limit can't be hosted.

File Management (Google Cloud File Storage)

The provided examples only provide a few gradle files that upload an installed local maven artifact. Sometimes there might be need to perform more file functions not available from gradle such as deletion. The documentation includes a button that launches a browser based file browser which presents a browser interface to perform file management.

License

Copyright 2018 Cerrato Renaud

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

appengine-maven-repository's People

Contributors

achuinard avatar gautamjain avatar jefimm avatar lbaeumer avatar matthewcosgrove avatar mp1 avatar peschmae avatar renaudcerrato avatar slartus avatar zed-kun avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

appengine-maven-repository's Issues

Respond to _ah/start?

I deployed this awesome repo to a new App Engine app yesterday and it worked for the first time. Then, this morning, the app stopped working, and I saw this in the logs:

2019-11-13 17:34:02.397 CST
The request failed because the instance could not start successfully
2019-11-13 17:34:02.397 CST
Process terminated because it failed to respond to the start request with an HTTP status code of 200-299 or 404.

I don't understand why it worked on deploy then stopped. It could have something to do with the instance scaling setup I added to appengine-web.xml.

<basic-scaling>
        <max-instances>1</max-instances>
        <idle-timeout>3m</idle-timeout>
    </basic-scaling>

It scales the app down to 0 instances when not in use.

I added an _ah/start controller and will make a PR...I just don't get why 1) the app worked and 2) no one has dealt with this before? Especially because it was reproducible locally just using ./gradlew appengineRun.

Add a gradle module example

Hi guys,
Thanks for your great library.
I am done deploy my repo as tutorial in ReadMe.
But I don't what's next as well as how to put my module gradle into appegine.
Could you please show a full example which has gradle module code in your project (where to put it and how to config)?
And show how to include this module into main project with private repo
I think it would be make sense for newbie like me, because I just a android developer, and I don't have any experience with Google Cloud
Thank you!

Automatic Cleanup of Snapshots

It would be nice if the app would keep only a certain specified number of latest snapshots and delete the rest. I am looking at adding it. I will send a PR when done.

Not working with compile 'com.google.appengine.tools:appengine-gcs-client:0.6'

Hello, first of all, thanks for the project.
I've got dependency retrieving exception while launching ./gradlew appengineDeploy to deploy.:

Caused by: org.gradle.internal.resolve.ModuleVersionNotFoundException: Could not find androidx.annotation:annotation:1.1.0.
Searched in the following locations:
  - https://jcenter.bintray.com/androidx/annotation/annotation/1.1.0/annotation-1.1.0.pom
  - https://jcenter.bintray.com/androidx/annotation/annotation/1.1.0/annotation-1.1.0.jar
Required by:
    project : > com.google.appengine.tools:appengine-gcs-client:0.6 > com.google.apis:google-api-services-storage:v1-rev68-1.21.0 > com.google.api-client:google-api-client:1.30.8

Problem solved by moving to:
'com.google.appengine.tools:appengine-gcs-client:0.8'

Initial Setup

just wanted to report my findings as I tried to follow the main instructions

  1. after creating my project in google cloud console, updating the users.txt and appengine-web.xml and running ./gradlew appengineUpdate I got the following error:
WARNING: Error posting to URL: https://appengine.google.com/api/appversion/getresourcelimits?app_id=bleachr-maven-repo&version=v1&
404 Not Found
This application does not exist (project_id=u'bleachr-maven-repo'). To create an App Engine application in this project, run "gcloud app create" in your console.
This is try #0

Could this have changed on google's side?
As soon as your project is created, a default Google Cloud storage bucket has been automatically created for you

I ended up signing up for a 'free trial' (not sure if I'm going to get billed after or not), enabling the google console shell and finally running the command gcloud app create

After this the ./gradlew appengineUpdate command worked!

Anonymous access

First, thanks for sharing this nice project – really appreciated.

Is there currently a way to allow anonymous read access to the repository?

Thanks!

Error: Not Found on landing page

First of all, nice project. It's a great idea for a frictionless way to get a Maven repo up and running. Especially when the best practise approach of using Nexus/Artifactory is cost prohibitive and cumbersome for personal projects.

Following the README page instructions, once the app is deployed, the login is fine via HTTP basic.

Without anything yet deployed to the Maven repo, the first landing page you see after login just says 'Error: Not Found'

I would suggest either

  1. This is expected behaviour and should be documented in the README
  2. There is an issue which requires a fix

Unique release artifact version

it would be nice to have an option to refuse artifacts that has the same version of an artifact already uploaded

Example:

  1. on the repository i have version 0.6
  2. I try to upload a new artifact, with verion 0.6
  3. the recpository refuses my upload, forcinjg me to bump version of the artifact

Build failed with an exception.

Configure project :
WARNING: You are a using release candidate 2.0.0-rc5. Behavior of this plugin has changed since 1.3.5. Please see release notes at: https://github.com/GoogleCloudPlatform/app-gradle-plugin.
Missing a feature? Can't get it to work?, please file a bug at: https://github.com/GoogleCloudPlatform/app-gradle-plugin/issues.
ERROR: (gcloud.app.deploy) The current Google Cloud project [h1r4-maven-repo] does not contain an App Engine application. Use gcloud app create to initialize an App Engine application within the project.

Task :appengineDeploy FAILED

FAILURE: Build failed with an exception.

  • What went wrong:
    Execution failed for task ':appengineDeploy'.

com.google.cloud.tools.appengine.AppEngineException: com.google.cloud.tools.appengine.operations.cloudsdk.process.ProcessHandlerException: com.google.cloud.tools.appengine.AppEngineException: Non zero exit: 1

  • Try:
    Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

  • Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.9/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 13s
6 actionable tasks: 2 executed, 4 up-to-date

Coupling user to specific packagename

I really like the simplicity of this project, only one thing; I really wish I could lock a user to a certain packagename when the listing appears, to make a simple grouping setup for different users

Migrate to JDK11

JDK8 on Appengine will be demised by end of Jan 2024. This project should be migrated to JDK11 in order to support further deployments.

Latest app engine default url requires version prefix

@renaudcerrato I also had to follow the manual steps from #6 for creating the app engine. It looks like app engine has changed how the default url works too.
I'm getting 404s at https://projectname.appspot.com
But found https://v1-dot-projectname.appspot.com in the console, which works when defining them in the maven repository url's. (Find at https://console.cloud.google.com/appengine/versions?project=<my-project-name>&serviceId=default)
It might be worth mentioning in the readme to verify the opening the url in a browser, results in a login popup? 👍

Doesn't seem to work with sbt

I couldn't get this project to work properly with sbt.
I'm fairly certain that it has something to do with the Basic Auth, because you are forced to define a realm for the credentials in sbt, but this project doesn't return any.

curl https://maven.company.com -vv returns WWW-Authenticate: Basic instead of WWW-Authenticate: Basic realm="some-realm-name"

I have already commented an open issue on sbt describing the problem (link)

However, i do believe that it would be fairly simple to just return a realm in this application.
Unfortunately, i wasn't able to do it myself, because i lack knowledge about JAX-RS.

Remove authorization key in local

Hi @renaudcerrato ,
Sorry to bother you, but I can't find the way to remove authorization key that generated after run
/gradlew appengineUpdate
I want to change the different google account to authorize (also change new app engine app), but it always use the old key for request
I tried to revoke permission from my google account, now I get 401 error when run
/gradlew appengineUpdate again
Do you know where the location of the authorization key? Or do you have any idea?
Thank!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.