Welcome to TrinityX!
TrinityX is the new generation of ClusterVision's open-source HPC platform. It is designed from the ground up to provide all services required in a modern HPC system, and to allow full customization of the installation. Also it includes optional modules for specific needs, such as an OpenStack cloud and Docker on the compute nodes.
The full documentation is available in the doc
subdirectory. See the instructions how to build it below.
In standard configuration TrinityX provides the following services to the cluster:
- Luna, our default super-efficient node provisioner https://github.com/clustervision/luna
- OpenLDAP
- SLURM or OpenPBS
- Telegraf, InfluxDB and Grafana
- Infrastructure services such as NTP, DNS, DHCP
- and more
It will also set up:
- NFS-shared home and application directories
- OpenHPC applications and libraries
- environment modules
- rsyslog
- and more
Install CentOS Minimal on your controller(s). It is recommended to put
/var/lib/influxdb
on it's own filesystem.Configure network interfaces that will be used in the cluster, e.g public, provisioning and MPI networks
Configure passwordless authentication to the controller itself or/and for both controllers in the HA case
Setup luna repository:
# curl https://updates.clustervision.com/luna/1.2/centos/luna-1.2.repo > /etc/yum.repos.d/luna-1.2.repo
Enable the EPEL repository:
# yum install epel-release
Install
git
,ansible
andluna-ansible
:# yum install git ansible luna-ansible
Clone TrinityX repository into your working directory and go to the site directory:
# git clone http://github.com/clustervision/trinityX # cd trinityX/site
Check out mitogen into the current directory (trinityX/site):
# git clone https://github.com/dw/mitogen.git
Based on whether you're installing a single-controller or a high-availability (HA) setup, you might want to update the configuration files:
group_vars/all
You might also want to check if the default firewall parameters in the same file apply to your situation:
firewalld_public_interfaces: [eth0] firewalld_trusted_interfaces: [eth1] firewalld_public_tcp_ports: [22, 443]
Note: In the case of an HA setup you will most probably need to change the default name of the shared block device set by
shared_fs_device
.Install
OndrejHome.pcs-modules-2
from the ansible galaxy:# ansible-galaxy install OndrejHome.pcs-modules-2
Configure
hosts
file to allow ansible to address controllers.
Example for non-HA setup:
[controllers] controller ansible_host=10.141.255.254Example for HA setup:
[controllers] controller1 ansible_host=10.141.255.254 controller2 ansible_host=10.141.255.253
Start TrinityX installation:
# ansible-playbook controller.yml
Note: If errors are encoutered during the installation process, analyze the error(s) in the output and try to fix it then re-run the installer.
Note: By default, the installation logs will be available at
/var/log/trinity.log
Create a default OS image:
# ansible-playbook compute.yml
Now you have your controller(s) installed and the default OS image created!
Now, if you want to tailor TrinityX to your needs, you can modify the ansible playbooks and variable files.
Descriptions to configuration options are given inside controller.yml
and group_vars/*
. Options that might be changed include:
- Controller's hostnames and IP addresses
- Shared storage backing device
- DHCP dynamic range
- Firewall settings
You can also choose which components to exclude from the installation by modifying the controller.yml
playbook.
The OpenHPC project provides a framework for building, managing and maintain HPC clusters. This project provides packages for most popular scientific and HPC applications. TrinityX can integrate this effort into it's ecosystem. In order to enable this integration set the flag enable_openhpc
in group_vars/all
to true
.
Currently when OpenHPC is enabled standart environment modules and pdsh from TrinityX gets disabled, slurm
is used from TrinityX's repositories.
To build the full set of the documentation included with TrinityX:
Install
git
:# yum install git
Clone TrinityX repository into your working directory and go to the directory containing the documentation:
# git clone http://github.com/clustervision/trinityx # cd trinityX/doc
Install
pip
, e.g. from EPEL repository:# yum install python34-pip.noarch
Install
sphinx
andRinohtype
:# pip3.4 install sphinx Rinohtype
Build the PDF version of the TrinityX guides:
# sphinx-build -b rinoh . _build/
If everything goes well, the documentation will be saved as _build/TrinityX.pdf
To contribute to TrinityX:
- Get familiar with our code guidelines
- Clone TrinityX repository
- Commit your changes in your repository and create a pull request to the
dev
branch in ours.