krishnachaitanyareddy/director-aws-plugin

Cloudera Director - Amazon Web Services integration

Amazon Web Services (AWS) plugin for Cloudera Director

• Introduction
• Getting started
• Building the plugin
• Running the live tests
• Implementation details
• Important notice

Introduction

This project defines an open source plugin implementing the Cloudera Director Service Provider Interface (Director SPI) for Amazon Web Services (AWS).

The plugin implements a compute provider that allocates Amazon Elastic Compute Cloud (EC2) instances, and a database server provider that allocates Amazon Relational Database Service (RDS) instances.

Getting started

Because the AWS plugin ships with Cloudera Director, these instructions are primarily geared towards plugin authors, who may be interested in using this plugin as an example or a base for their own plugins. They assume familiarity with git, github, maven, and other software development technologies; AWS concepts and terminology; and some familiarity with Cloudera Director and the Director SPI.

Additional details can be found in the source code, in the AWS Plugin javadoc, and in the Cloudera Director User Guide.

Prior to using the plugin, you will need to have AWS credentials.

Building the plugin

You can build the plugin with mvn clean install. Maven will need access to the Director SPI, either via the Cloudera Repository referenced in pom.xml, or via a local build.

Running the live tests

By default, the unit tests do not require communication with AWS. To run live tests that allocate and destroy EC2 and RDS instances, you must specify a system property (mvn -Dtest.aws.live=true clean install), and arrange for your AWS credentials to be available to the default credential provider chain via one of the methods described here.

Implementation details

The plugin follows the implementation pattern described in the Director SPI documentation.

The major classes are:

• The launcher (AWSLauncher)
• The cloud provider (AWSProvider)
• The resource providers (EC2Provider and RDSProvider)
• The corresponding resource templates (EC2InstanceTemplate and RDSInstanceTemplate)
• The corresponding resources (EC2Instance and RDSInstance)

The configuration properties are implemented as nested enums of configuration property tokens. A representative example is EC2InstanceTemplate.EC2InstanceTemplateConfigurationPropertyToken.

The display properties are implemented as nested enums of display property tokens. A representative example is EC2Instance.EC2InstanceDisplayPropertyToken.

Validation is implemented via composite validators, each containing a default validator and a custom validator. A representative custom validator example is EC2InstanceTemplateConfigurationValidator.

Localization uses the default localization support provided by the Director SPI.

Logging uses the SLF4J logging API, as required by the Director SPI.

Additional filesystem configuration is specified in HOCON and loaded using the typesafe config library.

Important notice

Copyright © 2015 Cloudera, Inc. Licensed under the Apache License, Version 2.0.

Cloudera, the Cloudera logo, and any other product or service names or slogans contained in this document are trademarks of Cloudera and its suppliers or licensors, and may not be copied, imitated or used, in whole or in part, without the prior written permission of Cloudera or the applicable trademark holder.

Hadoop and the Hadoop elephant logo are trademarks of the Apache Software Foundation. Amazon Web Services, the "Powered by Amazon Web Services" logo, Amazon Elastic Compute Cloud, EC2, Amazon Relational Database Service, and RDS are trademarks of Amazon.com, Inc. or its affiliates in the United States and/or other countries. All other trademarks, registered trademarks, product names and company names or logos mentioned in this document are the property of their respective owners. Reference to any products, services, processes or other information, by trade name, trademark, manufacturer, supplier or otherwise does not constitute or imply endorsement, sponsorship or recommendation thereof by us.

Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Cloudera.

Cloudera may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Cloudera, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. For information about patents covering Cloudera products, see http://tiny.cloudera.com/patents.

The information in this document is subject to change without notice. Cloudera shall not be liable for any damages resulting from technical errors or omissions which may be present in this document, or from use of this document.