mroark1m / hadoop-lzo-packager Goto Github PK
View Code? Open in Web Editor NEWThis project forked from toddlipcon/hadoop-lzo-packager
Packaging utilities for GPL compression libraries in Hadoop
License: GNU General Public License v2.0
This project forked from toddlipcon/hadoop-lzo-packager
Packaging utilities for GPL compression libraries in Hadoop
License: GNU General Public License v2.0
Overview =================== This project is a convenient mechanism of packaging the hadoop-gpl-compression project from Google Code. It has three basic steps: 1) Perform an svn export of the most recent revision of hadoop-gpl-compression 2) Create and build an RPM 3) Create an build a Debian package Requirements: ==================== - subversion - java (preferably sun's JDK) - JAVA_HOME must be set in your environment - appropriate package building tools and lzo libs for your platform - yum install rpm-build lzo-devel (RedHat based) - apt-get install dev-scripts liblzo2-dev (Debian based) - ant version 1.7.0 or greater (RedHat will require some fiddling[1]) When you try to build for your platform, build dependency errors will also inform you of any other packages you may need to install (eg lzo2 devel packages, ant, etc) [1]: First you'll need to install "ant" and "ant-nodeps" with yum. Then You'll need to download the binary Apache Ant distribution from their website, and extract the tarball somewhere. You'll have to set ANT_HOME in your environment to point to the newly archived directory. You'll also have to put $ANT_HOME/bin in your path, before /usr/bin. Running "ant" on the command line should run $ANT_HOME/bin/ant, not /usr/bin/ant. Usage: ==================== To build packages, simply run the included shell script. ./run.sh We recommend you run this on the same platform as your tasktrackers so as to be sure the built libraries are compatible. Various options are available, to get help do: ./run.sh -h If you would like to skip building debian or rpm, you can do: ./run.sh --no-rpm or ./run.sh --no-deb If you'd like to check out a particular revision, you can do: ./run.sh --svn-rev=46 If the downloads fail because of certificate problems, you can do: WGET_OPTS=--no-check-certificate ./run.sh If the build fails and you find a file build/master then you have a version of wget which does not use the filename from the redirected URL. You can work around it with: WGET_OPTS=--trust-server-names=on ./run.sh Or with both options: WGET_OPTS="--no-check-certificate --trust-server-names=on" ./run.sh There are some other variables that can be overridden - simply look at the top section of run.sh to learn what they are. After running the script, you should be able to find debs in the build/deb directory and RPMs in the build/topdir/RPMS directory. Contributing =================== To contribute to this project, please clone its repository from http://github.com/toddlipcon/hadoop-lzo-packager/ and commit patches to your github repository. When you would like to submit your contribution for inclusion, send a Pull Request to the Cloudera repository.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.