Giter Site home page Giter Site logo

davidchen322 / bigdata-interop Goto Github PK

View Code? Open in Web Editor NEW

This project forked from googleclouddataproc/hadoop-connectors

0.0 1.0 0.0 2.52 MB

Libraries and tools for interoperability between Hadoop-related open-source software and Google Cloud Platform.

License: Apache License 2.0

Java 100.00%

bigdata-interop's Introduction

bigdata-interop

Libraries and tools for interoperability between Hadoop-related open-source software and Google Cloud Platform.

Google Cloud Storage connector for Hadoop

The Google Cloud Storage connector for Hadoop enables running MapReduce jobs directly on data in Google Cloud Storage by implementing the Hadoop FileSystem interface. For details, see the README in the /gcs/ folder.

Building

The Google Cloud Storage (GCS) connector is built with Maven 3 (as of 2014-05-23, version 3.2.1 has been tested). To build the connector for Hadoop 1, run the following commands from the main directory:

mvn -P hadoop1 package

To build the connector with support for Hadoop 2 & YARN, run the following commands from the main directory:

mvn -P hadoop2 package

In both cases the GCS connector JAR can be found in gcs/target/.

bigdata-interop's People

Contributors

dennishuo avatar mooman219 avatar pmkc avatar dansedov avatar laraschmidt avatar laurenhstephens avatar ml8 avatar cushon avatar gk5885 avatar jimmc avatar peihe avatar kluever avatar chimerasaurus avatar azurezyq avatar xardazz avatar galabar001 avatar lindner avatar lukesandberg avatar lukecwik avatar medb avatar gildea avatar dorireuv avatar dimo414 avatar dhalperi avatar adamderusha avatar chamikaramj avatar aman-ebay avatar ravwojdyla avatar

Watchers

David Chen avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.