The latest documentation is available on Read The Docs
rhpit / manageiq-cli Goto Github PK
View Code? Open in Web Editor NEWBecause we like managing the cloud via terminal!
License: GNU General Public License v3.0
Because we like managing the cloud via terminal!
License: GNU General Public License v3.0
The latest documentation is available on Read The Docs
Currently the project is using sphinx to document all of its documentation for the CLI. This issue is to determine and put in place how we plan to host the documentation. Some possible suggestions are:
The nice take away with read the docs is it provides the ability to have versioned documentation. This way if users want to stick with a certain version of the CLI, they can follow the documentation that aligns with it. As far as I know, GitHub Pages does not provide this capability out of the box. I did find a sphinx versioning tool [1] that may be possible to provide this capability for GitHub Pages hosting the documentation.
[1] https://robpol86.github.io/sphinxcontrib-versioning/index.html#
Currently the way we handle exiting the CLI is using the log.error(<message>, abort=True)
function call.
Error logging should have one purpose and that is to provide the user with a detailed message when an error happens. The abort parameter was added in case the program wanted to exit right then. @seandst mentioned a comment in a previous PR that we should look into having a log.abort()
function call to handle situations that need to exit right away when an error happens.
This issue is to add the new function, and revise the api module to use log.abort
over log.error(abort=True)
as needed. The function will be very simple to start but can be expanded over time.
From Tiago's comments on a previous issue, on how we should authenticate to the server:
We should have a token set in a temporary file to assist the command line to connect to the ManageIQ instance. But this is the responsibility of the CLI and not the user. The user has to provide only the username, password and the host wheren ManageIQ is installed.
The order for the token should be something like:
if a $HOME/.miqcli/auth file exist
validate token
if token doesn't work
create new token
update $HOME/.miqcli/auth
else
create $HOME/.miqcli/auth
create new token
update $HOME/.miqcli/auth
use token from $HOME/.miqcli/auth
Currently through rest API no support for Cloud SubNetworks. Hence not supported in cli.
To provision an instance on AWS a cloud sub network is a required field. The field is supplied as an id through the api. Create and automate method that will return information for cloud sub networks.
Input: Cloud Network by Name or ID
Cloud SubNetwork by Name or ID
Returns: List of Cloud SubNetworks, Name, id, cidr and network_provider
All input is optional. No input given list of all Cloud SubNetworks returned.
The first item the CLI tackles is handling reading configuration data that the user can set. User can set this data either in a file format, environment variable or as CLI global parameter options. I was testing the Config class which handles these actions and found when I have the directory /etc/miqcli created and an a empty file miqcli.yml created. It fails when trying to parse the file when it is empty.
Need to handle the use case where the file may be empty and we should ignore trying to parse it.
(miq-py36) [rywillia@laptop ~]$ tree /etc/miqcli/
/etc/miqcli/
└── miqcli.yml
0 directories, 1 file
(miq-py36) [rywillia@laptop ~]$ stat /etc/miqcli/miqcli.yml
File: /etc/miqcli/miqcli.yml
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: fd01h/64769d Inode: 1716768 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 1000/rywillia) Gid: ( 1000/rywillia)
Context: unconfined_u:object_r:etc_t:s0
Access: 2018-01-11 10:56:06.944662412 -0500
Modify: 2018-01-11 10:55:58.056608016 -0500
Change: 2018-01-11 10:55:58.056608016 -0500
Birth: -
(miq-py36) [rywillia@laptop ~]$ miqcli
Traceback (most recent call last):
File "/home/rywillia/.virtualenvs/miq-py36/bin/miqcli", line 6, in <module>
from miqcli.cli.main import cli
File "/home/rywillia/github/manageiq-cli/miqcli/cli/main.py", line 224, in <module>
silent=True)
File "/home/rywillia/github/manageiq-cli/miqcli/utils/__init__.py", line 74, in from_yaml
for key in config_data:
TypeError: 'NoneType' object is not iterable
In order for issue #23 to be started, we need to determine where we will establish the connection to the ManageIQ server defined by the user. Before the new year I started working on a prototype on how we could incorporate the ManageIQ API Client library but came across a problem. We can't establish the connection within the constructor for each collection class. If this is done, every time you run the CLI it would attempt to establish a connection to the server. This would potentially cause a DOS attack against your ManageIQ server. I.e. users install the client and run the client over and over to view the commands. It would be attempting to connect each time.
# connection would be attempted when running this command
$ miqcli <command>
I found that we can override the parent invoke method for the CLI sub-command class. This could allow us to perform the initial connection to ManageIQ (calling a method from a API class). Having this method available to be run at any point within the CLI is good because we may need to establish a connection again if a given token has expired.
I.e. When we are within the sub collections class we already know the command being run and have the command object. We can easily call a method to connect to the server.
class SubCollections(click.MultiCommand):
def invoke(self, ctx):
"""Invoke the sub-command selected.
Runs the sub-command given with all its arguments. Before it invokes
the sub-command action to run, it will attempt to establish a
connection to ManageIQ.
:param ctx: Click context.
:type ctx: Namespace
"""
self.collection.connect()
super(SubCollections, self).invoke(ctx)
Manageiq-client package has a new version on PyPI. This version contains the latest code changes we need for the client to connect to ManageIQ server by token authentication. Need to update the requirements.txt to remove installing from source.
Have CLI support for the OpenStack provider.
Support to create, teardown, add networking capabilities, query the resource (vm)
Now that the CLI is progressing, we need to make sure we handle errors properly. These errors need to be handled properly and we should not just throw a stack trace back to the user. This is not user friendly and the user should not need to sift through the stack trace in order to determine what went wrong.
We currently have a couple areas in the source code that raise exceptions if user error happened or run time failure. This issue is to create a solution that will provide the user with a helpful explanation as to what went wrong. If users wish to receive a stack trace from the error received, we should provide a verbose or debug mode. When this is enabled it will allow users to gain more information at run time.
I think it will be good to define this standard and implement earlier on when the CLI is taking shape. This way it can be used by future commits.
Validate all the input for the resource that is to be provisioned:
The current documentation structure is the default one sphinx creates when setting up a new project. Before we start diving into adding documentation about each command and its sub-commands, etc. We need to come up with the structure on where content will be placed.
This issue is to create the structure for the documentation.
Possible sections:
We need to add Config into the all variable if we later want to use import * from miqcli.utils
See line below the missing 'Config
from the all_ variable.
manageiq-cli/miqcli/utils/__init__.py
Line 24 in 691775d
Collections interfacing directly with a cloud provider will require specific ID's for key:values in a given payload.
I.e: Provision request needs the ID for a template "image" within the payload over the string value.
This means collections will need to do look ups to get the ID for the given element a provider needs. Currently between automation_requests and provision_requests collection classes, they both require these look ups. This opens the door for potential code duplication between collections to get a specific IDs.
We should have a common provider class that is generic enough to work with any cloud provider to provide the ID back for the given key string value.
support querying the status of a provisioning or teardown request:
Some of the automate methods created for project (get_floating_ip, release_floating_ip allocate_floating_ip and retire_floating_ip) allow input by name. Name's are not unique. These methods need to be updated to handle situations where the input is given by name and multiple entries exist with the name. Some may require additional input fields. All require proper error handling if method can not narrow the action to just one target.
Id's are unique and the methods work fine for ids.
Have CLI support for the OpenShift provider.
Support to create, teardown, add routes & query the resource (pod)
To provide support for provision requests in a beaker environment. The CLI should be able to perform actions that existing cloud providers can do within the beaker provider. These actions consist of the following:
Have CLI support for the AWS provider.
Support to create, teardown, add networking capabilities, query the resource (vm)
support for an option to teardown a resource
Providing an invalid command/sub-command causes the CLI to raise an exception. Exception is raised due to the program is unable to locate the Python module for the given command.
i.e.
(miq-py36) [rywillia@laptop ~]$ miqcli xyz
Traceback (most recent call last):
File "/home/rywillia/.virtualenvs/miq-py36/bin/miqcli", line 10, in <module>
sys.exit(cli())
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/rywillia/github/manageiq-cli/miqcli/cli/main.py", line 132, in invoke
super(ManageIQ, self).invoke(ctx)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/click/core.py", line 1061, in invoke
cmd_name, cmd, args = self.resolve_command(ctx, args)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/click/core.py", line 1100, in resolve_command
cmd = self.get_command(ctx, cmd_name)
File "/home/rywillia/github/manageiq-cli/miqcli/cli/main.py", line 86, in get_command
miq_module = import_module(COLLECTIONS_PACKAGE + '.' + name)
File "/home/rywillia/.virtualenvs/miq-py36/lib64/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'miqcli.collections.xyz'
(miq-py27) [rywillia@laptop ~]$ miqcli provision_requests xy
WARNING: Default configuration is used.
Traceback (most recent call last):
File "/home/rywillia/.virtualenvs/miq-py27/bin/miqcli", line 10, in <module>
sys.exit(cli())
File "/home/rywillia/.virtualenvs/miq-py27/lib/python2.7/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/home/rywillia/.virtualenvs/miq-py27/lib/python2.7/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/rywillia/github/manageiq-cli/miqcli/cli/main.py", line 135, in invoke
super(ManageIQ, self).invoke(ctx)
File "/home/rywillia/.virtualenvs/miq-py27/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/rywillia/github/manageiq-cli/miqcli/cli/main.py", line 274, in invoke
super(SubCollections, self).invoke(ctx)
File "/home/rywillia/.virtualenvs/miq-py27/lib/python2.7/site-packages/click/core.py", line 1061, in invoke
cmd_name, cmd, args = self.resolve_command(ctx, args)
File "/home/rywillia/.virtualenvs/miq-py27/lib/python2.7/site-packages/click/core.py", line 1100, in resolve_command
cmd = self.get_command(ctx, cmd_name)
File "/home/rywillia/github/manageiq-cli/miqcli/cli/main.py", line 203, in get_command
method = getattr(collection, name)
AttributeError: 'Collections' object has no attribute 'xy'
We should handle the exception and provide a message back stating the command is invalid.
Provide a way for the client to retrieve client configuration defined by the user.
(Need ManageIQ host and credentials -> token or username/password, enabling ssl, etc...)
Issue #34 brought the capability to have the CLI create an API connection to the declared ManageIQ server. This connection would only be created when the CLI goes to invoke "run" the commands sub-command.
$ miqcli providers create --option1 value1 --option2 value2 etc.
That is when we want the connection to be created when the sub-command is declared and its corresponding parameters needed to be invoked. That means all parameters including --help would allow the CLI to connect to the server. The help parameter is different than all other parameters. It is there to show all available parameters for the given sub-command and exit. It should not try to connect to the ManageIQ server.
Here is an example where the default MIQ configuration settings are defined but no ManageIQ server is running locally and the help parameter is given.
(miq-py36) [rywillia@laptop ~]$ miqcli providers create --help
Traceback (most recent call last):
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
raise err
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
conn.connect()
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/urllib3/connection.py", line 284, in connect
conn = self._new_conn()
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f797992da20>: Failed to establish a new connection: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='localhost', port=8443): Max retries exceeded with url: /api (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f797992da20>: Failed to establish a new connection: [Errno 111] Connection refused',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/rywillia/github/manageiq-cli/miqcli/api.py", line 75, in connect
verify_ssl=settings['enable_ssl_verify']
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/manageiq_client/api.py", line 44, in __init__
self._load_data()
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/manageiq_client/api.py", line 47, in _load_data
data = self.get(self._entry_point)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/manageiq_client/api.py", line 86, in get
partial(self._session.get, url, params=get_params))
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/manageiq_client/api.py", line 81, in _sending_request
raise last_connection_exception
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/manageiq_client/api.py", line 77, in _sending_request
return func()
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/requests/sessions.py", line 521, in get
return self.request('GET', url, **kwargs)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/requests/adapters.py", line 508, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='localhost', port=8443): Max retries exceeded with url: /api (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f797992da20>: Failed to establish a new connection: [Errno 111] Connection refused',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/rywillia/.virtualenvs/miq-py36/bin/miqcli", line 10, in <module>
sys.exit(cli())
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/rywillia/github/manageiq-cli/miqcli/cli/main.py", line 123, in invoke
super(ManageIQ, self).invoke(ctx)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/rywillia/github/manageiq-cli/miqcli/cli/main.py", line 224, in invoke
self.collection.connect()
File "/home/rywillia/github/manageiq-cli/miqcli/collections/__init__.py", line 46, in connect
super(Collection, self).connect(self.settings)
File "/home/rywillia/github/manageiq-cli/miqcli/api.py", line 78, in connect
raise RuntimeError(ex)
RuntimeError: HTTPSConnectionPool(host='localhost', port=8443): Max retries exceeded with url: /api (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f797992da20>: Failed to establish a new connection: [Errno 111] Connection refused',))
To fix this bug, the CLI code needs to be updated to not connect to the server when the help parameter is declared.
Investigation into how the client will support Beaker as a provider.
Issue #44 brought the ability to log messages to the console. Debug level messages are only logged when the root option --verbose is enabled. Part of the debug function, I use click's global built-ins to get the current context. From here I use this to get the verbose value to determine if debug logging should happen. Well this works great for any root command. I.e:
$ miqcli
When you want to log messages at the sub-command layer it fails with the following exception. I.e.
# I mocked the providers create method to log a debug message
$ miqcli --verbose providers create
(miq-py36) [rywillia@laptop miqcli]$ miqcli --verbose providers create
Traceback (most recent call last):
File "/home/rywillia/.virtualenvs/miq-py36/bin/miqcli", line 10, in <module>
sys.exit(cli())
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/rywillia/github/manageiq-cli/miqcli/cli/main.py", line 119, in invoke
super(ManageIQ, self).invoke(ctx)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/rywillia/github/manageiq-cli/miqcli/cli/main.py", line 217, in invoke
super(SubCollections, self).invoke(ctx)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/rywillia/.virtualenvs/miq-py36/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/rywillia/github/manageiq-cli/miqcli/cli/main.py", line 169, in func
return method(*args, **kwargs)
File "/home/rywillia/github/manageiq-cli/miqcli/collections/providers.py", line 30, in create
log.debug('Lets begin with creating a new provider.')
File "/home/rywillia/github/manageiq-cli/miqcli/utils/log.py", line 55, in debug
if click.get_current_context().parent.params['verbose']:
KeyError: 'verbose'
When the code gets run to log the debug message, it uses click.get_current_context to lookup the verbose value. The bug here is that the context it returns is for the sub-command context not the root command context. The root command is the one that allows you to set verbose parameter and saves its setting within the root context.
To fix this bug and allow the collection classes to log messages, we need to get the root context in order to determine if verbose mode was enabled or not. That will then allow the debug level messages to be logged.
Currently the MIQ REST API does not provide a collection with actions to create floating IP addresses and attach the IP to instances.
In order to provide networking capabilities #16 we need to leverage MIQ automate feature. We need to create an automate script (Ruby language) to create/attach IP to instances. Which can be called via the REST API automation_requests collection.
TBD
The AUTHDIR is used in combination with the environment variable HOME
, which is not set in all platforms. In order to be compatible with other platforms, we should use the function os.path.expanduser
. Read more about expanduser
here
I know we have discussed this before, but after exploring further some of the components of the client, I noticed that we are using settings
object by reference, and if at any time we change the settings
that will reflect in instability of the ClientAPI
class.
Any line that uses settings
within this class can be affected by a an external change to the given settings
(if we don't generate a new dict, it will be float around as a reference).
Another point is the amount of times that the same statement is repeated in a single function:
Line 87 in 691775d
Line 79 in 691775d
Line 95 in 691775d
Line 109 in 691775d
With a better constructor we can create smaller functions that do one thing only and do it well instead of later we suffer with maintenance of the big connect()
function.
With that in mind, I will propose a change to the ClientAPI class. Once the PR is sent, I would like to hear your opinion on that.
While working on issue #3 to enable a CI job for each pull request. I noticed the tox command would fail when it comes to building the sphinx documentation. The doc generator fails because the static folder does not exist. Console output below:
Running Sphinx v1.6.5
loading pickled environment... failed: unsupported pickle protocol: 4
building [mo]: all of 0 po files
building [html]: all source files
updating environment: 1 added, 0 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
generating indices... genindex
writing additional pages... search
copying static files...
Warning, treated as error:
html_static_path entry u'/home/rywillia/github/manageiq-cli/doc/source/static' does not exist
ERROR: InvocationError: '/home/rywillia/github/manageiq-cli/.tox/docs/bin/python setup.py build_sphinx'
________________________________________________ summary ________________________________________________
py36: commands succeeded
py27: commands succeeded
pep8: commands succeeded
ERROR: docs: commands failed
There are two ways we can resolve this:
Commit the static folder with a empty place holder .gitkeep.
We can modify the docs conf.py module to remove the static folder from the html_static_path variable.
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
-html_static_path = ['static']
+html_static_path = []
I would suggest suggestion 1: to add an empty place holder. Reason being is when the documentation grows, we probably will have static files that will need to be included during the doc build process. This way then we do not need to edit the conf.py adding back the static folder to the path. @tvieira what are your thoughts on this?
Create the CONTRIBUTING doc at the root of the project. This will define all the guidelines that existing and new contributors need to follow by when submitting pull requests.
https://help.github.com/articles/setting-guidelines-for-repository-contributors/
The CLI should provide the ability to query information related to provisioned resources. Resources will provide different information regarding the provider it was provisioned into.
I.e. query the host name for an instance or the public IP address to access the instance via SSH.
PR #53 brought automate scripts with methods available to perform networking capabilities for Openstack provider. There is now a dedicated folder under the root of the manageiq-cli project (Automate-Datastore) which contains all the files and a README.md. It also added a AUTOMATE-DATASTORE.rst at the root of the project explaining how to use the automate scripts.
I think it would be best if we could combine these two files or move AUTOMATE-DATASTORE.rst to the Automate-Datastore folder. This way everything related to automate scripts is in one location. What does everyone think?
We need to leverage the following project [1] to be used as the core underlying component to the CLI. This library provides access to MIQ REST API collections via Python. Our collections modules can use this library to perform the actions interfacing with MIQ.
This issue is to implement the solution on how we can incorporate this library into all collections. Maybe it is included as a mixin class to all collections classes, a utility module, etc.
The command line structure design mirrors the available ManageIQ REST API collections. The command line has commands collections which have sub-commands collection actions. @tvieira brought up a nice point of how can we validate these commands. How can we know if our commands are compatible with the REST API collections. The goal here is to validate all our commands are valid commands on ManageIQ.
I.e. A future release of ManageIQ could depreciate certain collections, add collections, change collection name or even add new actions to collections. How would we easily know about this: users report bugs, we see it in release notes, etc. It would be nice to have an automated way that we could tell if our command line structure is valid or not.
The idea here is to create tests to verify all the command line commands are valid commands with ManageIQ. This could be an integration test requiring ManageIQ server to be running and fetch the collections/actions to compare.
For release 1.0 the command line structure is inline with the current collections [1] provided by ManageIQ.
[1] http://manageiq.org/docs/reference/latest/api/reference/collections
Provision request returns very limited data. Need more Information to pass back to the user.
Need methods to return information for VM's and maybe floating IPs.
A list method where you can request all or narrow the request by input or even just ask for one.
Create List Methods for Floating ips and VM's.
It would be nice to enable a bot listening as a web hook for new pull requests. This bot could notify us via IRC (similar to how Fedora Infra) has for their PRs. This would remove the need to check email for
reviewing PRs. Part of this made me think that it would also be nice addition to have a bot be able to auto recommend reviewers to a PR. I found the following bot that has an algorithm that will review modules modified and suggest reviewers based on authors for that given module.
https://github.com/facebook/mention-bot
This is not a critical issue. I just wanted to create an issue capturing the thought.
support for an option to provision a resource
Currently you are able to use the client from command line.
$ miqcli <global-options> <command> <sub-command> <options>
This is great since it provides an easy way to interface with MIQ servers. To expand on this, we need to provide the ability to use the client as library import in Python modules. Right now the client is structured and built based on click library with each collection module accessing data from click's context.
Goal here is to have a unified class which you can import and instantiate to call MIQ collections/actions just like you would from the command line. If we did not provide a class like this, users would have to do something like the following:
from miqcli.api import ClientAPI
from miqcli.collections import provision_requests
# create client api
client_api = ClientAPI(<parameters>)
# build a click context for collections to use to access various attributes
ctx = Context(<paramerers>)
# create collection object
collection = provsion_requests.Collections()
and so on..
With the unified class, users would only have to do the following in their code:
from miqcli.api import ManageIQClient
miq = ManageIQClient(<manageiq_settings>)
collection = miq.get_collection('provision_requests')
collection.create(<parameters>)
collection.query(<paramerers>)
After pulling the latest changes, I got a problem when trying to run the CLI to view the available commands outside my local copy of the project. I now get an exception.
Steps to reproduce the problem:
# 1. create virtual environment
$ virtualenv miqcli && source miqcli/bin/activate
(miqcli) [rywillia@laptop manageiq-cli]$
# 2. Install the CLI via pip
(miqcli) [rywillia@laptop manageiq-cli]$ pip install -e .
# 3. Change directory to a different one other than the source code
(miqcli) [rywillia@laptop manageiq-cli]$ cd ~
# 4 Run the CLI
(miqcli) [rywillia@laptop ~]$ miqcli
Traceback (most recent call last):
File "/home/rywillia/miqcli/bin/miqcli", line 6, in <module>
from miqcli.cli.main import cli
File "/home/rywillia/github/manageiq-cli/miqcli/cli/main.py", line 26, in <module>
from miqcli.api import ClientAPI
File "/home/rywillia/github/manageiq-cli/miqcli/api.py", line 26, in <module>
from miqcli.constants import AUTHDIR, DEFAULT_CONFIG
File "/home/rywillia/github/manageiq-cli/miqcli/constants.py", line 29, in <module>
VERSION = get_version(PACKAGE)
File "/home/rywillia/miqcli/lib/python2.7/site-packages/pbr/packaging.py", line 755, in get_version
name=package_name))
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name miqcli was given, but was not able to be found.
If I switch back to the source code directory everything works fine.
(miqcli) [rywillia@laptop manageiq-cli]$ miqcli
Usage: miqcli [OPTIONS] COMMAND [ARGS]...
ManageIQ command line interface.
Options:
--version Show version and exit.
<Truncated>
Looks like pbr is unable to get the version based on the order they check for when outside a local copy of the git repository.
Currently the MIQ REST API does not provide a collection with actions to delete floating IP addresses and detach the IP from instances.
In order to provide networking capabilities #16 we need to leverage MIQ automate feature. We need to create an automate script (Ruby language) to delete/detach IP to instances. Which can be called via the REST API automation_requests collection.
TBD
When testing the Automate code for get_floating_ip, a floating ip was successfully generated.
However, I was unable to do anything with it as the return data did not include the id of that floating ip, it just looks like some guid:
"return":{"10.8.248.62":"8e8316c0-8d9e-456a-a579-a2d891c97e25"}
Currently our ClientAPI class raises RuntimeError's on error. The CLI now has a dedicated logging module which uses click's built-in logging capabilities. This issue is to switch from raising exceptions to use the logging functions to log the message to the console. This will eliminate the stack trace logged and still will exit the program correctly.
Right now when running the CLI, you do not know if you are using the default configuration or not. To make this clear to the user, it would be a nice addition to log a message stating if you are using the default configuration.
It is good to have Travis-CI badge in the README.rst for master branch so we can check the status of the build right after getting to the Github page.
We have to create Ruby scripts to automate the creation of AWS subnet just like we did for creating Floating IPs at Openstack (#27). We want to have this documented to assist us when we deploy a stage instance of ManageIQ for testing purpose.
support options to create and delete a provider
We have created a tox file to run tests in Python 3.6 + 2.7 + PEP8 standards check, but now we need an instance of ManageIQ running locally so we can run tests locally prior to create a new PR for the feature/fix.
There are few options that I have been testing:
The 3rd option I would discard, I don't want to install and maintain a local installation of ManageIQ and I don't think anyone would want that.
The 1st and 2nd options are a lot easier to manage. I would adopt docker because when using Vagrant we still have to have Virtualbox. Some of us will have only libvirt, or VMWare and there's no official image available for these providers.
I would vote for Docker as, at least the core developers for this project, we all have the technology locally to run containers.
Is there anyone that would prefer another option other than using the docker image created by ManageIQ? Otherwise I already have some work done with the docker image.
While working on #50, I realized the existing logging messages will never be executed. Each Config class method accepts a silent parameter. This parameter will return out silencing any failures that may occur at run time. This is may be troubling to the user if they are trying to load configuration from a file but the file does not have the correct file extension. (This is just an example).
Now that we have a verbose option the user can define via the CLI. We should use this value to determine if messages in the Config methods should be silenced or not. This will require re-ordering when the Configuration is loaded by the CLI.
I will propose a change to when we load configuration when the CLI is run. Once the PR is submitted, please provide your feedback.
support for this option
We should enable a Travis CI job to be run for any pull request to the project. This job would verify the code changes in the pull request meet our flake8 standards and verify all unit tests continue to pass. The job could easily run tox command to verify the latest changes for all supported Python versions.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.