cartalla/scale-out-computing-on-aws

Scale-Out Computing on AWS is a solution that helps customers deploy and operate a multiuser environment for computationally intensive workflows.

Scale-Out Computing on AWS - HPC made easy 🌴

Table of Contents

• 📖 Documentation
• 🚀 How to install Scale-Out Computing on AWS
  • 1-Click installer
  • Custom Build
• ✏️ File Structure

📖 Documentation

https://awslabs.github.io/scale-out-computing-on-aws/

🚀 How to install Scale-Out Computing on AWS

1-Click installer

Visit https://aws.amazon.com/solutions/scale-out-computing-on-aws

⚠️1-Click installer is great for PoC or demos. For production workload, it’s recommended to go with "Custom Build" instead.

Custom Build

1. Clone this git repository

   git clone https://github.com/awslabs/scale-out-computing-on-aws

2. Run the following command to create your build (support Python2 and Python3):

   STACK_NAME=<your-stack-name>
   python source/manual_build.py \
       --region <region> \
       --bucket <deployment-bucket> \
       [ --client-ip <CIDR-with-access-to-VPC> ] \
       [ --prefix-list-id <prefix-list-with-access-to-VPC> ] \
       --ssh-keypair <keypair> \
       --username <username> \
       --stack-name $STACK_NAME \
       --password <ssm-parameter-with password> \
       --id $STACK_NAME \
       --create

3. Output will be created under source/dist/<build_id>

4. Upload source/dist/<build_id> folder to your own S3 bucket

5. Launch CloudFormation and use scale-out-computing-on-aws.template as base template

Refer to https://awslabs.github.io/scale-out-computing-on-aws/install-soca-cluster/ for installation instructions.

✏️ File Structure

Scale-Out Computing on AWS project consists in a collection of CloudFormation templates, Shell scripts and Python code.

.
├── scale-out-computing-on-aws.template                 [ Soca Primary Template ]
├── soca
│   ├── cluster_analytics                               [ Scripts to ingest cluster/job data into ELK ]
│   ├── cluster_hooks                                   [ Scheduler Hooks ]
│   ├── cluster_logs_management                         [ Scripts to manage cluster log rotation ]
│   ├── cluster_manager                                 [ Scripts to control Soca cluster ]
│   └── cluster_web_ui                                  [ Web Interface ]
│   └── cluster_node_bootstrap                          [ Script to configure compute nodes]
├── scripts
│   ├── config.cfg                                      [ List of all packages to install ]
│   ├── Scheduler.sh                                    [ Configure Schedule Node ]
│   └── SchedulerPostReboot.sh                          [ Post Reboot Scheduler Node actions ]
└── templates
    ├── Analytics.template                              [ Manage ELK stack for your cluster ]
    ├── ComputeNode.template                            [ Manage simulation nodes ]
    ├── Configuration.template                          [ Centralize cluster configuration ]
    ├── Network.template                                [ Manage VPC configuration ]
    ├── Scheduler.template                              [ Manage Scheduler host ]
    ├── Security.template                               [ Manage ACL, IAM and SGs ]
    ├── Storage.template                                [ Manage backend storage ]
    └── Viewer.template                                 [ Manage DCV sessions ]

---

Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.