openflighthpc / flight-cloud Goto Github PK

View Code? Open in Web Editor NEW

3.0 3.0 2.0 1.4 MB

Cloud orchestration tool

License: Eclipse Public License 2.0

Ruby 97.63% Shell 2.37%

aws azure

flight-cloud's People

Contributors

Stargazers

Watchers

Forkers

vlj91 frankfanslc

flight-cloud's Issues

Fix "method_lines" issue in lib/cloudware/aws.rb

Method machines has 32 lines of code (exceeds 25 allowed). Consider refactoring.

https://codeclimate.com/github/alces-software/cloudware/lib/cloudware/aws.rb#issue_5a57e4b98d1a7c0001000033

Remove the original class

The original Domain, Machine, Aws/Aws2 and Azure class are being replaced with there Provider versions. As such they should be removed

Destroying Domain on Azure Fails

When trying to destroy a domain (resource group) on Azure the following error appears:

[vagrant@controller ~]$ cloudware domain destroy --name crafty-caribou
[OK]  Checking domain exists
error: {oying domain crafty-caribou
  "message": "Long running operation failed with status 409",
  "request": {
    "base_uri": "https://management.azure.com",
    "path_template": "https://management.azure.com/subscriptions/d1e964ef-15c7-4b27-8113-e725167cee83/operationresults/eyJqb2JJZCI6IlJFU09VUkNFR1JPVVBERUxFVElPTkpPQi1DUkFGVFk6MkRDQVJJQk9VLVVLU09VVEgiLCJqb2JMb2NhdGlvbiI6InVrc291dGgifQ?api-version=2017-05-10",
    "method": "get",
    "path_params": null,
    "skip_encoding_path_params": null,
    "query_params": null,
    "skip_encoding_query_params": null,
    "headers": {
      "Content-Type": "application/json;charset=utf-8",
      "Accept": "application/json",
      "accept-language": "en-US",
      "x-ms-client-request-id": "3a30a87f-1371-4129-a842-6c4af0f77a99"
    },
    "body": null,
    "middlewares": [
      [
        "MsRest::RetryPolicyMiddleware",
        {
          "times": 3,
          "retry": 0.02
        }
      ],
      [
        "cookie_jar"
      ]
    ],
    "log": null
  },
  "response": {
    "body": "{\"Error\":{\"Code\":\"ResourceGroupDeletionBlocked\",\"Target\":null,\"Message\":\"Deletion of resource group 'crafty-caribou' failed as resources with identifiers 'Microsoft.Network/virtualNetworks/network' could not be deleted. The provisioning state of the resource group will be rolled back. The tracking Id is '719b2d3d-1117-4f0e-a349-05331165380d'. Please check audit logs for more details.\",\"Details\":[{\"Code\":null,\"Target\":\"/subscriptions/d1e964ef-15c7-4b27-8113-e725167cee83/resourceGroups/crafty-caribou/providers/Microsoft.Network/virtualNetworks/network\",\"Message\":\"{\\\"error\\\":{\\\"code\\\":\\\"InUseSubnetCannotBeDeleted\\\",\\\"message\\\":\\\"Subnet prv is in use by /subscriptions/d1e964ef-15c7-4b27-8113-e725167cee83/resourceGroups/crafty-caribou-master1/providers/Microsoft.Network/networkInterfaces/crafty-cariboumaster1prv/ipConfigurations/crafty-cariboumaster1-prv and cannot be deleted.\\\",\\\"details\\\":[]}}\",\"Details\":null,\"Innererror\":null}],\"Innererror\":null}}",
    "headers": {
      "cache-control": "no-cache",
      "pragma": "no-cache",
      "content-type": "application/json; charset=utf-8",
      "expires": "-1",
      "x-ms-ratelimit-remaining-subscription-reads": "14998",
      "x-ms-request-id": "2dddb76b-b8b6-4cfb-a742-7e4356f56b05",
      "x-ms-correlation-request-id": "2dddb76b-b8b6-4cfb-a742-7e4356f56b05",
      "x-ms-routing-request-id": "UKWEST:20180430T085004Z:2dddb76b-b8b6-4cfb-a742-7e4356f56b05",
      "strict-transport-security": "max-age=31536000; includeSubDomains",
      "x-content-type-options": "nosniff",
      "date": "Mon, 30 Apr 2018 08:50:04 GMT",
      "connection": "close",
      "content-length": "959"
    },
    "status": 409
  }
}
	from /opt/cloudware/vendor/ruby/2.4.0/gems/ms_rest_azure-0.9.0/lib/ms_rest_azure/azure_service_client.rb:207:in `get_async_with_custom_deserialization'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/ms_rest_azure-0.9.0/lib/ms_rest_azure/azure_service_client.rb:157:in `update_state_from_location_header'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/ms_rest_azure-0.9.0/lib/ms_rest_azure/azure_service_client.rb:50:in `block in get_long_running_operation_result'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/safe_task_executor.rb:24:in `block in execute'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `block in synchronize'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/safe_task_executor.rb:19:in `execute'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/timer_task.rb:309:in `execute_task'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/safe_task_executor.rb:24:in `block in execute'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `block in synchronize'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/safe_task_executor.rb:19:in `execute'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/ivar.rb:170:in `safe_execute'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/scheduled_task.rb:285:in `process_task'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/timer_set.rb:168:in `block (2 levels) in process_tasks'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `run_task'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in `block (3 levels) in create_worker'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `loop'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `block (2 levels) in create_worker'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `catch'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `block in create_worker'

This seems to indicate that the resources are being deleted in an order not supported by Azure (network being deleted before resources that are on that network).

Review race condition in domain templates

When the following commands are run in quick succession:

fc domain create race-condition-check1
fc domain create race-condition-check2
fc domain create race-condition-check3
fc domain create race-condition-check4
fc domain create race-condition-check5

Some of the domains will work, however others will fail. The logs from a failed build are included at the end of this message. The command is using the default domain template. A possible fix from slack: https://alces.slack.com/archives/C82EXS7J4/p1527596025000533

However we should also check the other domain templates to see if they also have the issue.

2018-05-29 | Status | Type | Logical ID | Status Reason
-- | -- | -- | -- | --
13:06:44 UTC+0100ROLLBACK_COMPLETEAWS::CloudFormation::Stackrace-condition-check113:06:43 UTC+0100DELETE_COMPLETEAWS::EC2::VPCNetwork13:06:27 UTC+0100DELETE_IN_PROGRESSAWS::EC2::VPCNetwork13:06:26 UTC+0100DELETE_COMPLETEAWS::EC2::SubnetPriSubnet13:06:25 UTC+0100DELETE_COMPLETEAWS::EC2::InternetGatewayInternetGateway13:06:11 UTC+0100DELETE_COMPLETEAWS::EC2::RouteTableRouteTable13:06:10 UTC+0100DELETE_IN_PROGRESSAWS::EC2::SubnetPriSubnet13:06:10 UTC+0100DELETE_IN_PROGRESSAWS::EC2::RouteTableRouteTable13:06:10 UTC+0100DELETE_IN_PROGRESSAWS::EC2::InternetGatewayInternetGateway13:06:09 UTC+0100DELETE_COMPLETEAWS::EC2::SubnetRouteTableAssociationPriSubnetRouteTableAssocation13:06:09 UTC+0100DELETE_COMPLETEAWS::EC2::VPCGatewayAttachmentInternetGatewayAttachment13:05:54 UTC+0100DELETE_IN_PROGRESSAWS::EC2::SubnetRouteTableAssociationPriSubnetRouteTableAssocation13:05:53 UTC+0100DELETE_COMPLETEAWS::EC2::RouteRouteInternetGateway13:05:53 UTC+0100DELETE_IN_PROGRESSAWS::EC2::VPCGatewayAttachmentInternetGatewayAttachment13:05:25 UTC+0100ROLLBACK_IN_PROGRESSAWS::CloudFormation::Stackrace-condition-check1The following resource(s) failed to create: [PriSubnetRouteTableAssocation, InternetGatewayAttachment, RouteInternetGateway]. . Rollback requested by user.13:05:24 UTC+0100CREATE_FAILEDAWS::EC2::SubnetRouteTableAssociationPriSubnetRouteTableAssocationResource creation cancelled13:05:24 UTC+0100CREATE_FAILEDAWS::EC2::VPCGatewayAttachmentInternetGatewayAttachmentResource creation cancelled13:05:24 UTC+0100CREATE_IN_PROGRESSAWS::EC2::SubnetRouteTableAssociationPriSubnetRouteTableAssocationResource creation Initiated13:05:23 UTC+0100CREATE_IN_PROGRESSAWS::EC2::VPCGatewayAttachmentInternetGatewayAttachmentResource creation Initiated13:05:23 UTC+0100CREATE_FAILEDAWS::EC2::RouteRouteInternetGatewayroute table rtb-4052c139 and network gateway igw-32e49f55 belong to different networks (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: b62c7006-a984-4a36-8334-0959de970002)13:05:23 UTC+0100CREATE_IN_PROGRESSAWS::EC2::VPCGatewayAttachmentInternetGatewayAttachment13:05:23 UTC+0100CREATE_IN_PROGRESSAWS::EC2::SubnetRouteTableAssociationPriSubnetRouteTableAssocation13:05:23 UTC+0100CREATE_IN_PROGRESSAWS::EC2::RouteRouteInternetGateway13:05:21 UTC+0100CREATE_COMPLETEAWS::EC2::SubnetPriSubnet13:05:21 UTC+0100CREATE_COMPLETEAWS::EC2::InternetGatewayInternetGateway13:05:05 UTC+0100CREATE_COMPLETEAWS::EC2::RouteTableRouteTable13:05:05 UTC+0100CREATE_IN_PROGRESSAWS::EC2::InternetGatewayInternetGatewayResource creation Initiated13:05:04 UTC+0100CREATE_IN_PROGRESSAWS::EC2::SubnetPriSubnetResource creation Initiated13:05:04 UTC+0100CREATE_IN_PROGRESSAWS::EC2::InternetGatewayInternetGateway13:05:04 UTC+0100CREATE_IN_PROGRESSAWS::EC2::SubnetPriSubnet13:05:04 UTC+0100CREATE_IN_PROGRESSAWS::EC2::RouteTableRouteTableResource creation Initiated13:05:03 UTC+0100CREATE_IN_PROGRESSAWS::EC2::RouteTableRouteTable13:05:00 UTC+0100CREATE_COMPLETEAWS::EC2::VPCNetwork13:04:43 UTC+0100CREATE_IN_PROGRESSAWS::EC2::VPCNetworkResource creation Initiated13:04:42 UTC+0100CREATE_IN_PROGRESSAWS::EC2::VPCNetwork13:04:40 UTC+0100CREATE_IN_PROGRESSAWS::CloudFormation::Stackrace-condition-check1User Initiated |   | 13:06:44 UTC+0100 | ROLLBACK_COMPLETE | AWS::CloudFormation::Stack | race-condition-check1 |   |   | 13:06:43 UTC+0100 | DELETE_COMPLETE | AWS::EC2::VPC | Network |   |   | 13:06:27 UTC+0100 | DELETE_IN_PROGRESS | AWS::EC2::VPC | Network |   |   | 13:06:26 UTC+0100 | DELETE_COMPLETE | AWS::EC2::Subnet | PriSubnet |   |   | 13:06:25 UTC+0100 | DELETE_COMPLETE | AWS::EC2::InternetGateway | InternetGateway |   |   | 13:06:11 UTC+0100 | DELETE_COMPLETE | AWS::EC2::RouteTable | RouteTable |   |   | 13:06:10 UTC+0100 | DELETE_IN_PROGRESS | AWS::EC2::Subnet | PriSubnet |   |   | 13:06:10 UTC+0100 | DELETE_IN_PROGRESS | AWS::EC2::RouteTable | RouteTable |   |   | 13:06:10 UTC+0100 | DELETE_IN_PROGRESS | AWS::EC2::InternetGateway | InternetGateway |   |   | 13:06:09 UTC+0100 | DELETE_COMPLETE | AWS::EC2::SubnetRouteTableAssociation | PriSubnetRouteTableAssocation |   |   | 13:06:09 UTC+0100 | DELETE_COMPLETE | AWS::EC2::VPCGatewayAttachment | InternetGatewayAttachment |   |   | 13:05:54 UTC+0100 | DELETE_IN_PROGRESS | AWS::EC2::SubnetRouteTableAssociation | PriSubnetRouteTableAssocation |   |   | 13:05:53 UTC+0100 | DELETE_COMPLETE | AWS::EC2::Route | RouteInternetGateway |   |   | 13:05:53 UTC+0100 | DELETE_IN_PROGRESS | AWS::EC2::VPCGatewayAttachment | InternetGatewayAttachment |   |   | 13:05:25 UTC+0100 | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | race-condition-check1 | The following resource(s) failed to create: [PriSubnetRouteTableAssocation, InternetGatewayAttachment, RouteInternetGateway]. . Rollback requested by user. |   | 13:05:24 UTC+0100 | CREATE_FAILED | AWS::EC2::SubnetRouteTableAssociation | PriSubnetRouteTableAssocation | Resource creation cancelled |   | 13:05:24 UTC+0100 | CREATE_FAILED | AWS::EC2::VPCGatewayAttachment | InternetGatewayAttachment | Resource creation cancelled |   | 13:05:24 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::SubnetRouteTableAssociation | PriSubnetRouteTableAssocation | Resource creation Initiated |   | 13:05:23 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::VPCGatewayAttachment | InternetGatewayAttachment | Resource creation Initiated |   | 13:05:23 UTC+0100 | CREATE_FAILED | AWS::EC2::Route | RouteInternetGateway | route table rtb-4052c139 and network gateway igw-32e49f55 belong to different networks (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: b62c7006-a984-4a36-8334-0959de970002) |   | 13:05:23 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::VPCGatewayAttachment | InternetGatewayAttachment |   |   | 13:05:23 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::SubnetRouteTableAssociation | PriSubnetRouteTableAssocation |   |   | 13:05:23 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::Route | RouteInternetGateway |   |   | 13:05:21 UTC+0100 | CREATE_COMPLETE | AWS::EC2::Subnet | PriSubnet |   |   | 13:05:21 UTC+0100 | CREATE_COMPLETE | AWS::EC2::InternetGateway | InternetGateway |   |   | 13:05:05 UTC+0100 | CREATE_COMPLETE | AWS::EC2::RouteTable | RouteTable |   |   | 13:05:05 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::InternetGateway | InternetGateway | Resource creation Initiated |   | 13:05:04 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::Subnet | PriSubnet | Resource creation Initiated |   | 13:05:04 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::InternetGateway | InternetGateway |   |   | 13:05:04 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::Subnet | PriSubnet |   |   | 13:05:04 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::RouteTable | RouteTable | Resource creation Initiated |   | 13:05:03 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::RouteTable | RouteTable |   |   | 13:05:00 UTC+0100 | CREATE_COMPLETE | AWS::EC2::VPC | Network |   |   | 13:04:43 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::VPC | Network | Resource creation Initiated |   | 13:04:42 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::VPC | Network |   |   | 13:04:40 UTC+0100 | CREATE_IN_PROGRESS | AWS::CloudFormation::Stack | race-condition-check1 | User Initiated
  | 13:06:44 UTC+0100 | ROLLBACK_COMPLETE | AWS::CloudFormation::Stack | race-condition-check1 |  
  | 13:06:43 UTC+0100 | DELETE_COMPLETE | AWS::EC2::VPC | Network |  
  | 13:06:27 UTC+0100 | DELETE_IN_PROGRESS | AWS::EC2::VPC | Network |  
  | 13:06:26 UTC+0100 | DELETE_COMPLETE | AWS::EC2::Subnet | PriSubnet |  
  | 13:06:25 UTC+0100 | DELETE_COMPLETE | AWS::EC2::InternetGateway | InternetGateway |  
  | 13:06:11 UTC+0100 | DELETE_COMPLETE | AWS::EC2::RouteTable | RouteTable |  
  | 13:06:10 UTC+0100 | DELETE_IN_PROGRESS | AWS::EC2::Subnet | PriSubnet |  
  | 13:06:10 UTC+0100 | DELETE_IN_PROGRESS | AWS::EC2::RouteTable | RouteTable |  
  | 13:06:10 UTC+0100 | DELETE_IN_PROGRESS | AWS::EC2::InternetGateway | InternetGateway |  
  | 13:06:09 UTC+0100 | DELETE_COMPLETE | AWS::EC2::SubnetRouteTableAssociation | PriSubnetRouteTableAssocation |  
  | 13:06:09 UTC+0100 | DELETE_COMPLETE | AWS::EC2::VPCGatewayAttachment | InternetGatewayAttachment |  
  | 13:05:54 UTC+0100 | DELETE_IN_PROGRESS | AWS::EC2::SubnetRouteTableAssociation | PriSubnetRouteTableAssocation |  
  | 13:05:53 UTC+0100 | DELETE_COMPLETE | AWS::EC2::Route | RouteInternetGateway |  
  | 13:05:53 UTC+0100 | DELETE_IN_PROGRESS | AWS::EC2::VPCGatewayAttachment | InternetGatewayAttachment |  
  | 13:05:25 UTC+0100 | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | race-condition-check1 | The following resource(s) failed to create: [PriSubnetRouteTableAssocation, InternetGatewayAttachment, RouteInternetGateway]. . Rollback requested by user.
  | 13:05:24 UTC+0100 | CREATE_FAILED | AWS::EC2::SubnetRouteTableAssociation | PriSubnetRouteTableAssocation | Resource creation cancelled
  | 13:05:24 UTC+0100 | CREATE_FAILED | AWS::EC2::VPCGatewayAttachment | InternetGatewayAttachment | Resource creation cancelled
  | 13:05:24 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::SubnetRouteTableAssociation | PriSubnetRouteTableAssocation | Resource creation Initiated
  | 13:05:23 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::VPCGatewayAttachment | InternetGatewayAttachment | Resource creation Initiated
  | 13:05:23 UTC+0100 | CREATE_FAILED | AWS::EC2::Route | RouteInternetGateway | route table rtb-4052c139 and network gateway igw-32e49f55 belong to different networks (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: b62c7006-a984-4a36-8334-0959de970002)
  | 13:05:23 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::VPCGatewayAttachment | InternetGatewayAttachment |  
  | 13:05:23 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::SubnetRouteTableAssociation | PriSubnetRouteTableAssocation |  
  | 13:05:23 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::Route | RouteInternetGateway |  
  | 13:05:21 UTC+0100 | CREATE_COMPLETE | AWS::EC2::Subnet | PriSubnet |  
  | 13:05:21 UTC+0100 | CREATE_COMPLETE | AWS::EC2::InternetGateway | InternetGateway |  
  | 13:05:05 UTC+0100 | CREATE_COMPLETE | AWS::EC2::RouteTable | RouteTable |  
  | 13:05:05 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::InternetGateway | InternetGateway | Resource creation Initiated
  | 13:05:04 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::Subnet | PriSubnet | Resource creation Initiated
  | 13:05:04 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::InternetGateway | InternetGateway |  
  | 13:05:04 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::Subnet | PriSubnet |  
  | 13:05:04 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::RouteTable | RouteTable | Resource creation Initiated
  | 13:05:03 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::RouteTable | RouteTable |  
  | 13:05:00 UTC+0100 | CREATE_COMPLETE | AWS::EC2::VPC | Network |  
  | 13:04:43 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::VPC | Network | Resource creation Initiated
  | 13:04:42 UTC+0100 | CREATE_IN_PROGRESS | AWS::EC2::VPC | Network |  
  | 13:04:40 UTC+0100 | CREATE_IN_PROGRESS | AWS::CloudFormation::Stack | race-condition-check1 | User Initiated

Rebrand main command line tool and refactor

Main

Rename the main cloudware command to be flightconnector and rebrand the help text to match.

Bonus

Whilst we're there, refactor command line usage to be user friendly, eg help when running the command with no options, displaying help. eg When doing things like this...

[root@controller[barkla] ~]# cloudware idunno --help
invalid command. Use --help for more information

Is not overly helpful.

I think @ColonelPanicks may have some more opinions to add to this as well.

Add waits to the power commands

All the other command wait except power on and power off.

Also it might be good to tell users they can hit ctrl-c to exit early at this point. However to do so at there own peril as it might still fail.

Store more information about a machine

Currently we're storing the following information about a machine in each provider:

State
External IP (if applicable)
Prv and mgt IP
Short hostname
Flavour

We could also do with optionally storing:

Instance ID if applicable based on the Cloud provider

Display public IP in machine list

If a machine has a public IP, it would be useful to be able to display that information in the machine list output -- at the moment I have to go to the cloud provider console to work this out.

`machine list` should list `state` of each machine

Refactor Prv to Pri

prv network should be called pri to keep in line with metalware terminology, bye bye prv

Can't Destroy Empty Domain

Lists of current domains and machines:

[vagrant@controller ~]$ cloudware domain list
[OK]  Fetching available domains
+----------------+--------------+-----------------+-----------------+----------+-----------+
| Domain name    | Network CIDR | Prv Subnet CIDR | Mgt Subnet CIDR | Provider | Region    |
+----------------+--------------+-----------------+-----------------+----------+-----------+
| laces          | 10.0.0.0/16  | 10.0.1.0/24     | 10.0.2.0/24     | aws      | eu-west-1 |
| prickly-pigeon | 10.0.0.0/16  | 10.0.1.0/24     | 10.0.2.0/24     | aws      | eu-west-1 |
+----------------+--------------+-----------------+-----------------+----------+-----------+
[vagrant@controller ~]$ cloudware machine list
[OK]  Fetching available machines
+------+--------+------+----------------+----------------+------+-------+
| Name | Domain | Role | Prv IP address | Mgt IP address | Type | State |
+------+--------+------+----------------+----------------+------+-------+
+------+--------+------+----------------+----------------+------+-------+

Attempting to delete prickly-pigeon

[vagrant@controller ~]$ cloudware domain destroy --name prickly-pigeon --trace
[OK]  Checking domain exists
/opt/cloudware/lib/cloudware/domain.rb:93:in `destroy': Unable to destroy domain with active machines (RuntimeError)
	from /opt/cloudware/lib/cloudware/cli.rb:134:in `block (2 levels) in <class:CLI>'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/command.rb:178:in `call'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/command.rb:153:in `run'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/runner.rb:446:in `run_active_command'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/runner.rb:68:in `run!'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/delegates.rb:15:in `run!'
	from /opt/cloudware/bin/cloudware:32:in `<main>'

Checked on AWS and there are no ec2 instances containing that name and no resources tagged with cloudware_name: prickly-pigeon besides the VPC (which it should then delete).

This issue may be specific to this domain but in case it isn't (despite having been able to delete other aws domains) then this issue can be used to track any other domains that have issues to prevent this going forwards.

Generate a `uuid` for domains and machines on create

Currently this is done in the Providers module. However this should be done in the Models before the object is created.

It should also be filled when the object is downloaded.

Naming a little inconsistent

The machine list shows the names of some stacks without the full domain prefix (perhaps something is being stripped?).

This means I'm unable to find the info for a specific machine if it shares the same "name" with another.

e.g. markt-domU2/login in the list below was created as markt-domU2-login and I can find info for it by doing machine info login; however I can't get info for markt-domU1/login.

+-------------------+---------------------------+---------+----------------+-----------+---------+
| Name              | Domain                    | Role    | Pri IP address | Type      | State   |
+-------------------+---------------------------+---------+----------------+-----------+---------+
| gateway           | markt-dom0                | gateway | 10.78.100.10   | c4.xlarge | running |
| login             | markt-domU2               | login   | 10.100.2.10    | c4.xlarge | running |
| node01            | awkward-anaconda-cluster1 | node    | 10.100.1.11    | c4.large  | running |
| login             | markt-domU1               | login   | 10.100.1.10    | c4.xlarge | running |
| cloudware-machine | tango-chicken             | master  | 10.78.100.42   | c4.xlarge | running |
| login             | awkward-anaconda-cluster1 | login   | 10.100.1.10    | c4.xlarge | running |
| gateway           | awkward-anaconda          | gateway | 10.78.100.10   | c4.xlarge | running |
+-------------------+---------------------------+---------+----------------+-----------+---------+

Logo Proposal for Cloudware

Hi, I'm a graphic designer and I like to collaborate with open source projects. Do you know that the graphic image of a project is very important? thinking about it I would like to design a logo for your Project Cloudware.

I will be pleased to collaborate with you.

Fix "argument_count" issue in lib/cloudware/azure.rb

Method create_machine has 9 arguments (exceeds 4 allowed). Consider refactoring.

https://codeclimate.com/github/alces-software/cloudware/lib/cloudware/azure.rb#issue_5a57e4b98d1a7c000100002e

Dry up the machine commands

There is a lot of duplication in the CLI and Command files

Collapse machine `flavour` and `type` into a single `CLI` input

They are always used together so they might as well be encoded together.

Suspend / resume / rebuild support

So that we can use cloudware as a orchestration facility in metalware we should implement a method to spoof metals powering off/on/status as well as an installs ability to reformat the disks and redeploy. I’d suggest

power off -> shutdown instance but do not terminate
power on -> resume instance
power status -> report running state of instance as ON or OFF

rebuild -> recreate instance as if it had just been created with the ‘create’ command

Connector image and vpn setup cloudinit script aws & azure

Implement a Flight connector image build script (modify cloudware/support/build/alces-cloudware-base-aws.ks) to build a clean image and then make the changes required for vpn setup via cloudinit on boot. As started in #25

Some of the things in the existing kickstart file can also be dropped (such as disabling firewalld, because we just enable it again in the build script.) Some of the install stages from the vpn init script can also be put in the kickstart, we only need to do the configuration / key generation parts via cloudinit.

Support listing of Failed domains and machines

These should be handled slightly differently as they are invalid. However it is still useful if the came up in the list

Fix "similar-code" issue in lib/cloudware/commands/machine/power/off.rb

Similar blocks of code found in 2 locations. Consider refactoring.

https://codeclimate.com/github/alces-software/cloudware/lib/cloudware/commands/machine/power/off.rb#issue_5af5e5659c1c250001000060

Azure Templates for FlightConnector

The current focus of FlightConnector is getting it to work with AWS but Azure support will be required/desired/great in the future, woo, hybrid cloud.

This means templates for:

domain types
machine types

And anything else we find we need on the AWS side of things.

Refactor Machine to FlightConnect Cluster node types

Currently the machine types (currently referred to as roles) are master and slave which are sourced from providers/aws/templates/machine-{master,slave}.yml.

Ideally the types that would exist are login, compute, storage and graphical.

Templates to be put in place by Stu as providers/aws/templates/machine-{login,compute,storage,graphical}/yml
Command to be refactored to support varying types (note: this may be supported already if role is just looked for as machine-ROLE.yml in the CLI object)

Worth considering whether the machine type (VM instance size) is hardcoded or supplied from CLI.

DRY up the common bits between `Base::Domains` and `Base::Machines`

They are both array like objects that use a builder. The same applies for the AWS versions of them

[BIG BAD ISSUE] Remove Dependency on Both Credentials Being Present/Valid

Remove the data from the Azure keys in cloudware.yml and the following will be seen:

[vagrant@controller cloudware]$ cloudware machine list --domain aws
error: {ing available machines
  "message": "Couldn't login to Azure, please verify your tenant id, client id and client secret",
  "request": null,
  "response": null
}. Use --trace to view backtrace
[vagrant@controller cloudware]$ cloudware machine list --domain aws --trace
/opt/cloudware/vendor/ruby/2.4.0/gems/ms_rest_azure-0.9.0/lib/ms_rest_azure/credentials/application_token_provider.rb:108:in `acquire_token': { (MsRestAzure::AzureOperationError)
  "message": "Couldn't login to Azure, please verify your tenant id, client id and client secret",
  "request": null,
  "response": null
}
	from /opt/cloudware/vendor/ruby/2.4.0/gems/ms_rest_azure-0.9.0/lib/ms_rest_azure/credentials/application_token_provider.rb:69:in `get_authentication_header'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/ms_rest-0.7.2/lib/ms_rest/credentials/token_credentials.rb:49:in `sign_request'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/ms_rest-0.7.2/lib/ms_rest/service_client.rb:51:in `block in make_request_async'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/ms_rest-0.7.2/lib/ms_rest/http_operation_request.rb:82:in `block (2 levels) in run_promise'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/faraday-0.13.1/lib/faraday/connection.rb:384:in `block in run_request'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/faraday-0.13.1/lib/faraday/connection.rb:398:in `block in build_request'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/faraday-0.13.1/lib/faraday/request.rb:26:in `block in create'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/faraday-0.13.1/lib/faraday/request.rb:25:in `tap'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/faraday-0.13.1/lib/faraday/request.rb:25:in `create'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/faraday-0.13.1/lib/faraday/connection.rb:394:in `build_request'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/faraday-0.13.1/lib/faraday/connection.rb:379:in `run_request'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/ms_rest-0.7.2/lib/ms_rest/http_operation_request.rb:80:in `block in run_promise'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/safe_task_executor.rb:24:in `block in execute'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `block in synchronize'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/safe_task_executor.rb:19:in `execute'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/promise.rb:531:in `block in realize'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `run_task'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in `block (3 levels) in create_worker'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `loop'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `block (2 levels) in create_worker'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `catch'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `block in create_worker'

If the Azure details are removed completely from the yaml file then the following occurs:

[vagrant@controller cloudware]$ cloudware machine list --domain aws
error: undefined method `[]' for nil:NilClass. Use --trace to view backtrace
[vagrant@controller cloudware]$ cloudware machine list --domain aws --trace
/opt/cloudware/lib/cloudware/config.rb:36:in `initialize': undefined method `[]' for nil:NilClass (NoMethodError)
	from /opt/cloudware/lib/cloudware.rb:34:in `new'
	from /opt/cloudware/lib/cloudware.rb:34:in `config'
	from /opt/cloudware/lib/cloudware/aws.rb:47:in `config'
	from /opt/cloudware/lib/cloudware/aws.rb:65:in `valid_credentials?'
	from /opt/cloudware/lib/cloudware/aws.rb:51:in `load_config'
	from /opt/cloudware/lib/cloudware/aws.rb:43:in `initialize'
	from /opt/cloudware/lib/cloudware/machine.rb:99:in `new'
	from /opt/cloudware/lib/cloudware/machine.rb:99:in `list'
	from /opt/cloudware/lib/cloudware/cli.rb:221:in `block (2 levels) in <class:CLI>'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/command.rb:178:in `call'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/command.rb:153:in `run'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/runner.rb:446:in `run_active_command'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/runner.rb:68:in `run!'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/delegates.rb:15:in `run!'
	from /opt/cloudware/bin/cloudware:32:in `<main>'

Similar is seen when the AWS details are empty

[vagrant@controller cloudware]$ cloudware machine list --domain azure
error: Authorization header or parameters are not formatted correctly.. Use --trace to view backtrace
[vagrant@controller cloudware]$ cloudware machine list --domain azure --trace
/opt/cloudware/lib/cloudware/cli.rb:237:in `rescue in block (2 levels) in <class:CLI>': Authorization header or parameters are not formatted correctly. (RuntimeError)
	from /opt/cloudware/lib/cloudware/cli.rb:217:in `block (2 levels) in <class:CLI>'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/command.rb:178:in `call'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/command.rb:153:in `run'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/runner.rb:446:in `run_active_command'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/runner.rb:68:in `run!'
	from /opt/cloudware/vendor/ruby/2.4.0/gems/commander-4.4.3/lib/commander/delegates.rb:15:in `run!'
	from /opt/cloudware/bin/cloudware:32:in `<main>'

And when the AWS is removed it's identical to the Azure details being removed.

Test ticket from Code Climate

Refactor Azure provider

The Azure provider could do with some refactoring, some things in specific:

Rework how resource clients are initialised
Clean up deployment process. It may be worth moving deployment to a separate class, as it takes up a large amount of the Azure class
Get rid of attr_*
Provide initial data when creating a node in a hash, rather than 30 different args

[Asset Tool] Hunt Mac Addresses for Assets

This issue is not a modification to hunter but instead to implement a hunter-style tool which can be used to update asset details with MAC addresses.

The Idea

Ideally a command that can run like hunter (given a starting point which auto-increments as more data comes in). A preliminary idea of the data we'd like to put into the command is below:

metal asset tool hunter --servers 'c6420*'  --network_adapter 'onboard' --interface 'em1'

Where:

--servers is a regex match of server assets to iterate through
--network_adapter is the network adapter within the servers to use
--interface is the the interface name within the network_adapter asset to modify the MAC address for

This would then function in a similar way to hunter with prompts

Waiting for new nodes to appear on the network, please network boot them now...
(Ctrl-C to terminate)
Detected a machine on the network (52:54:00:78:02:01). Please enter the destination attribute:  |[c6420a-server1.network_adapters.onboard.ports.em1.mac]|

Detected a machine on the network (52:54:00:78:02:02). Please enter the destination attribute:  |[c6420a-server2.network_adapters.onboard.ports.em1.mac]|

The outcome would be that the ports.em1.mac for the network adapter onboard for the server c6420a-server1

Changes to Metalware

Metal asset hunter tool
Ability to link network asset to a network in config/domain.yaml (e.g. network asset private linked to config.network.pri)

Changes to Repo

Change dhcp/default to identify the MAC address from the asset instead of the hunter cache
Change to config/domain.yaml to identify the network interface from the node asset

Remove the MGT Network

We only need a prv (which should be called pri) network for the domains that are going to be created so let's get rid of the extra guff to do with mgt.

Remove redundant domain parameters

This bug is not necessarily urgent and can be kicked down the road, it's just being logged here as a reminder to address at some point.

Currently the domain0 and domainU templates have no use for the networkCidr and priSubnetCidr variables in their CloudFormation templates. These are still in place because cloudware looks to merge parameters into the template that include these variables.

Going forward it would be nice to remove these redundant parameters.

Add validation to the machine `CLI` inputs

This is going to be progressive and include:

role
flavour
priip (If not removed)

Merge Provider's code with Model's

At first it wasn't obvious if inheritance of models was going to be required. However moving forward the different providers will need to download different things. As such the Providers::Base::Domain class will need to be merged with Models::Domain. This removes the need for the complex delegates.

The same should be done with Machine

Also remove aws specific attributes in Models::Domain:

network_id

Deployment states success, when a template actually fails

The CLI reports back that a machine has been deployed, even if the stack deployment fails:

[OK]  Creating new deployment

Logs show:

I, [2018-01-30T10:01:50.572400 #38116]  INFO -- : [Cloudware::Aws] Deploying new stack
Name: cloudy-cat-master1
Template: machine-master.yml
Params: [{:parameter_key=>"cloudwareDomain", :parameter_value=>"cloudy-cat"}, {:parameter_key=>"cloudwareId", :parameter_value=>"f6109526-f58f-42e6-850f-50cd97a4c2ac"}, {:parameter_key=>"prvIp", :parameter_value=>"10.0.1.5"}, {:parameter_key=>"mgtIp", :parameter_value=>"10.0.2.5"}, {:parameter_key=>"vmRole", :parameter_value=>"master"}, {:parameter_key=>"vmType", :parameter_value=>"c4.xlarge"}, {:parameter_key=>"vmName", :parameter_value=>"master1"}, {:parameter_key=>"networkId", :parameter_value=>"vpc-d7627dbe"}, {:parameter_key=>"prvSubnetId", :parameter_value=>"subnet-987875e3"}, {:parameter_key=>"mgtSubnetId", :parameter_value=>"subnet-da7f72a1"}, {:parameter_key=>"prvSubnetCidr", :parameter_value=>"10.0.1.0/24"}, {:parameter_key=>"mgtSubnetCidr", :parameter_value=>"10.0.2.0/24"}, {:parameter_key=>"vmFlavour", :parameter_value=>"compute"}]
E, [2018-01-30T10:01:50.718737 #38116] ERROR -- : Failed waiting for stack to create: Template error: Unable to get mapping for RegionMap::eu-west-2::AMI
Waiting for data... (interrupt to abort)

Convert `type` back to generic non-cloud specific

The machine type returns back the cloud-specific type, but we want to know the generic cloudware type:

[OK]  Fetching available machines
+---------+--------------+--------+----------------+----------------+--------------+
| Name    | Domain       | Role   | Prv IP address | Mgt IP address | Type         |
+---------+--------------+--------+----------------+----------------+--------------+
| master1 | burnt-boar   | master | 10.0.1.5       | 10.0.2.5       | c4.xlarge    |
| master1 | cheating-cat | master | 10.0.1.5       | 10.0.2.5       | Standard_F4s |
+---------+--------------+--------+----------------+----------------+--------------+

Refactor Domain to FlightConnect Domain Types

Currently the domain for AWS is created by providers/aws/templates/domain.yml.

With FlightConnect there will be 2 types of domain, domain0 and domainU. (Being the FlightConnect Gateway and FlightConnect Clients respectively).

The domain creation command should be able to use either template to launch a domain.

Templates to be created by Stu and stored as providers/aws/templates/{domain0.yml,domainU.yml}
Command to be refactored to support input of domain type

Extract a method for generating `AWS` deployment parameters

The parameters need to be an array of key value pairs. This creates a lot of boiler plate code that could be generated implicitly.

Ideally a method would take a Hash and generate the array.

Review the use of the `cloudware` tags

For both aws and azure the use of cloudware_ tags has been a source of confusion. As they are only tags, they are not necessarily meaningful.

A prime example is the cloudware_pri_subnet usage in the aws templates. The tag is included in with the templates but is then subsequently not used. This means when downloading the domain information, it isn't immediately obvious where the subnet information is coming form.

In a similar vein, the cloudware_domain tag ins't particularly useful either. Domains have a name parameter which is being duplicated by the cloudware_domain field. However it is the field that is required to be unique and thus is more important.

Ideally, the usage of the tags should be limited to the bear minimum and instead the properties of the resource should be used to populate the model information.

AWS image creator

Create AWS image from kickstart and import QCOW to AMI

Extract the filter for finding aws `cloudware` instances of things

They all use the same:
filters: [{ name: 'tag-key', values: ['cloudware_id'] }]

Domain Name Not Appended to Machine Name

If I create a domain:

[root@controller cloudware]# bin/cloud domain create -t domain0 --networkcidr 10.78.100.0/24 --prisubnetcidr 10.78.100.0/24 berserk-buzzard-dom0
[OK]  Creating new domain

And then create a machine within it:

[root@controller cloudware]# bin/cloud machine create -d berserk-buzzard-dom0 --role gateway --priip 10.78.100.10 gateway
[OK]  Creating new deployment

What I see (in CloudFormation) is:

Ideally the name would be berserk-buzzard-dom0-gateway.

Provider shouldn't be required for domain list

Running domain list without a provider exits with an error whereas it should be listing all of the domains from every cloud provider

[root@controller cloudware]# bin/cloud domain list
error: Missing the required --provider input.

Strangely, when a provider is specified it shows the domains for all providers

[root@controller cloudware]# bin/cloud domain list --provider aws
[OK]  Fetching available domains
+----------------------+---------------+-----------------+----------+-----------+
| Domain name          | Network CIDR  | Pri Subnet CIDR | Provider | Region    |
+----------------------+---------------+-----------------+----------+-----------+
| cloudware-azure-test | 10.0.0.0/16   |                 | azure    | uksouth   |
| prudent-prune        | 10.0.0.0/16   | 10.0.1.0/24     | aws      | eu-west-1 |
| cloudware-aws-test   | 10.0.0.0/16   | 10.0.1.0/24     | aws      | eu-west-1 |
| prickly-pigeon       | 10.0.0.0/16   | 10.0.1.0/24     | aws      | eu-west-1 |
| barkla2              | 10.100.0.0/16 | 10.0.1.0/24     | aws      | eu-west-1 |
| sly-slug             | 10.10.0.0/16  | 10.0.1.0/24     | aws      | eu-west-1 |
+----------------------+---------------+-----------------+----------+-----------+

Fix "method_lines" issue in lib/cloudware/aws.rb

Method domains has 36 lines of code (exceeds 25 allowed). Consider refactoring.

https://codeclimate.com/github/alces-software/cloudware/lib/cloudware/aws.rb#issue_5a57e4b98d1a7c0001000032

Prevent `Azure` from implicitly re-creating `Domains`

Azure uses an create_or_update command to create resources. This means if the resource already exists, the command will still work. I'm not 100% sure what will happen in this situation, but assumable it would recreate the same resource.

As this is an unexpected side affect of a create command and thus should be prevented. Instead, the domain create command should check if the domain already exists before creating it.

Integrate model attributes with templates

A mapping needs to be created between the models with templates attributes.

This way parameters like name and ids can be set.

There will have to be base attributes on the base and inherited classes.