Giter Site home page Giter Site logo

manageiq-automation_engine's Introduction

ManageIQ::AutomationEngine

CI Maintainability Test Coverage

Chat

Build history for master branch

Automation Engine plugin for ManageIQ.

Engine Description

There are two types of storage for the Automation Engine:

The Automation Datastore is the persistent storage and it can be viewed/edited by the EVM Automate Object Explorer. The data lives in SQL Tables beginning with miq_ae: miq_ae_namespaces, miq_ae_classes, miq_ae_fields, miq_ae_values, miq_ae_instances, miq_ae_methods. The Automation Workspace is the transient storage of the in-memory object hierarchy. There is a SQL Table called miq_ae_workspaces, but it is not currently leveraged and can be ignored for now.

Each Automation Engine invocation runs in an isolated Workspace.

Automate Engine is invoked with a URI and leverages URIs throughout. So, a solid understandings of URIs is a must. A URI stands for Uniform Resource Identifier and can be read about here http://en.wikipedia.org/wiki/Uniform_Resource_Identifier and http://tools.ietf.org/html/rfc3986.

Here is a grammar for a URI:

 URI         = scheme ":" hier-part [ "?" query ] [ "#" fragment ]

 hier-part   = "//" authority path-abempty
             / path-absolute
             / path-rootless
             / path-empty

Here is a typical example:

foo://example.com:8042/over/there?name=ferret#mouth
\_/   \______________/\_________/ \_________/ \___/
 |           |             |           |        |
scheme   authority        path       query   fragment

The Automation Engine supports two schemes (currently):

Scheme Description
miqaedb get an object from the Automation Datastore
miqaews get an object from the Automation Workspace

Database Layout:

MIQ_AE_NAMESPACES Table (each row defines a NAMESPACE in the system)
  id            ==> ID           of the Namespace
  parent_id     ==> ID           of the parent Namespace (NULL if root)
  display_name  ==> Display Name of the Namespace
  description   ==> Description  of the Namespace
  name          ==> Name         of the Namespace
  created_on    ==> ctime (Creation     Timestamp)
  updated_on    ==> mtime (Modification Timestamp)

MIQ_AE_CLASSES Table (each row defines a CLASS in the system)
  id            ==> ID           of the Class
  display_name  ==> Display Name of the Class
  description   ==> Description  of the Class
  namespace_id  ==> ID           of the Namespace
  name          ==> Name         of the Class
  type          ==> Type         of the Class (abstract ==> No Instances Allowed, ...)
  inherits      ==> Name         of the Inherited Class (NULL if no inheritance)
  visibility    ==> Private ==> only visible within Resolution; Public ==> accessible to caller of Resolution
  owner         ==> Owner        of the Class
  created_on    ==> ctime (Creation     Timestamp)
  updated_on    ==> mtime (Modification Timestamp)

MIQ_AE_FIELDS Table (each row defines a FIELD in the system)
  id            ==> ID           of the Field
  name          ==> Name         of the Field
  display_name  ==> Display Name of the Field
  description   ==> Description  of the Field
  aetype        ==> AE Type      of the Field      (assertion, attribute, relationship, method, state)
  datatype      ==> Type         of the Field Data (string, integer, boolean, ...)
  priority      ==> Order        of the Field      (within the Class)
  owner         ==> Owner        of the Field
  scope         ==> Scope        of the Field
  default_value ==> Default      of the Field      (the default value)
  substitute    ==> Substitution Enabled?
  message       ==> Message   that this Field responds to
  visibility    ==> Private ==> only visible within Resolution; Public ==> accessible to caller of Resolution
  collect       ==> Collection/Aggregation definition
  condition     ==> Condition (IF/UNLESS) that indicates whether to process
  class_id      ==> ID           of the Class  that this Field belongs to
  method_id     ==> ID           of the Method that this Field belongs to
  created_on    ==> ctime (Creation     Timestamp)
  updated_on    ==> mtime (Modification Timestamp)

MIQ_AE_INSTANCES Table (each row defines an INSTANCE of a specific class)
  id            ==> ID           of the Instance
  name          ==> Name         of the Instance
  display_name  ==> Display Name of the Instance
  description   ==> Description  of the Instance
  class_id      ==> ID           of the Class  that this Instance belongs to
  inherits      ==> Name         of the Inherited Class (NULL if no inheritance)
  created_on    ==> ctime (Creation     Timestamp)
  updated_on    ==> mtime (Modification Timestamp)

MIQ_AE_VALUES Table (each row defines a VALUE of a specific field in a specific instance)
  id            ==> ID           of the Value
  display_name  ==> Display Name of the Value
  value         ==> VALUE        of the Value
  instance_id   ==> ID           of the Instance
  field_id      ==> ID           of the Field
  collect       ==> Collection/Aggregation definition
  condition     ==> Condition (IF/UNLESS) that indicates whether to process
  created_on    ==> ctime (Creation     Timestamp)
  updated_on    ==> mtime (Modification Timestamp)

MIQ_AE_WORKSPACES Table (keeps instantiated trees)
  id            ==> ID           of the Workspace
  guid          ==> GUID         of the Workspace
  uri           ==> URI          of the Workspace
  workspace     ==> Binary representation of the Workspace
  setters       ==> Binary representation of attributes overridden externally
  created_on    ==> ctime (Creation     Timestamp)
  updated_on    ==> mtime (Modification Timestamp)

Dynamic Instantiation Process (AE_DIP)

  • Add display_name for namespace and value (for STEP field and completeness)
  • Add condition for field and value (IF/UNLESS)
  • Add collect for value (overrides definition in FIELD)
  • Define new type of field called STEP
  • AE_DIP should process steps as follows:
    1. Should convert steps to relationships appending #${#ae_message}
    2. Dynamically call methods before_step and after_step (or on_state_change)
    3. Should roughly internalize working of provision_state_machine to allow for others to be defined easier

Development

See the section on plugins in the ManageIQ Developer Setup

For quick local setup run bin/setup, which will clone the core ManageIQ repository under the spec directory and setup necessary config files. If you have already cloned it, you can run bin/update to bring the core ManageIQ code up to date.

License

The gem is available as open source under the terms of the Apache License 2.0.

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

manageiq-automation_engine's People

Contributors

agrare avatar astrozzc avatar bdunne avatar billfitzgerald0120 avatar blomquisg avatar bzwei avatar chessbyte avatar chrisarcand avatar d-m-u avatar dnnx avatar eilam20 avatar fabiendupont avatar fryguy avatar gberginc avatar gmcculloug avatar jameswnl avatar jprause avatar jrafanie avatar kbrock avatar ladas avatar lfu avatar miha-plesko avatar mkanoor avatar nicklamuro avatar pkomanek avatar shaneboulden avatar sseago avatar syncrou avatar tinaafitz avatar tzumainn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

manageiq-automation_engine's Issues

`remote_url` deprecation

  • Deprecate remote_url.

  • Make it call external_url or use the same column.

  • Add a migration that removes the URL column from the SystemConsole table (check if it's not used by the proxying piece).

  • Later on remove the remove_url mixin.

More background available in #328

introduce internal automate method

Currently every method in automate is configured to run in Drb and considered a user provided script. However manageiq is shipped with many automate methods which are not really user provided. We should introduce internal automate methods that run default implementations internally but allow users to override with an external script.

Add option for quota to be applied on group but enforced at user level

Related BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1653492

Business use-case requires that we efficiently apply a quota at the group level, which it is today, however we need this quota to map to each user.

For example, group1, has a quota of: 10 vCPU, 5GB RAM, and 1T Disk

Today: if group1 has 10 users, that quota is shared between all 10 users, so if user1 uses 8 vCPU and 4GB of RAM, there is little space left.

What we need: we need each user in group1 to have 10 vCPU, 5GB of RAM, and 1T quota associated with their user account. Additionally, it would be great to allocate instances/VMs to that user as well.

This would need to be an option to enable/disable, or even combine with the current quota implementation.

Allow to emit notifications for selected users from automate

If the audience of the notification is set to user, it is always being sent to the @workspace.ae_user and there's no possibility to notify e.g. the admin. A possible workaround would be to send the notification to the group of the admin user, however, this is not feasible as we're bound to notification types.

@pkomanek @d-m-u can you please take a look?

vm_reconfigure_request error checking quota's

A user with only a tenant quota for memory set to 256GB is trying to reconfigure a VM and add an additional HDD to his machine. We are seeing an error in the quota used method when calling check_quota. When the quota is turned off then he is able to provision the drive without error.

We are running MIQ Fine-4 and VMware vcenter 6.5.
Gist contains the logs for the vm_reconfigure_request.
https://gist.github.com/Stromweld/d59742714eda7e732378e262217dd412

Expose cloud_volume to automate

Looking to expose aws cloud_volume methods to automate
create/delete/attach volume and create/delete snapshot

miq_ae_service_manageiq-providers-cloud_manager-vm.rb
add expose :cloud_volumes, :association => true

miq_ae_service_cloud_volume.rb
add expose :create_volume_snapshot, :override_return => nil

Slow tests: miq_ae_dialog_spec: "properly instantiates dialogs" instantiates 1000s of rows

This single example takes ~21 seconds on travis and creates loads of DB records.

https://github.com/ManageIQ/manageiq/blob/8d6d23639d12105bb6a0a69e4e79b879e3c9830f/vmdb/spec/lib/miq_automation_engine/miq_ae_dialog_spec.rb#L259

[----] I, [2014-10-02T15:49:47.315131 #30878:3ff2d185e6e4]  INFO -- : ZZZ  Starting: ./spec/lib/miq_automation_engine/miq_ae_dialog_spec.rb:259
[----] I, [2014-10-02T15:49:47.315178 #30878:3ff2d185e6e4]  INFO -- : ZZZZ From Example group: ./spec/lib/miq_automation_engine/miq_ae_dialog_spec.rb:5
[----] D, [2014-10-02T15:50:06.992935 #30878:3ff2d185e6e4] DEBUG -- :   MiqAeClass Inst (1.7ms - 142rows)
[----] D, [2014-10-02T15:50:07.010563 #30878:3ff2d185e6e4] DEBUG -- :   MiqAeField Inst (4.8ms - 424rows)
[----] D, [2014-10-02T15:50:07.029055 #30878:3ff2d185e6e4] DEBUG -- :   MiqAeInstance Inst (7.3ms - 682rows)
[----] D, [2014-10-02T15:50:07.064661 #30878:3ff2d185e6e4] DEBUG -- :   MiqAeValue Inst (10.6ms - 946rows)
[----] D, [2014-10-02T15:50:07.087512 #30878:3ff2d185e6e4] DEBUG -- :   MiqAeField Inst (3.8ms - 342rows)
[----] D, [2014-10-02T15:50:09.762381 #30878:3ff2d185e6e4] DEBUG -- :   MiqAeClass Inst (1.5ms - 142rows)
[----] D, [2014-10-02T15:50:09.779401 #30878:3ff2d185e6e4] DEBUG -- :   MiqAeField Inst (4.7ms - 424rows)
[----] D, [2014-10-02T15:50:09.797099 #30878:3ff2d185e6e4] DEBUG -- :   MiqAeInstance Inst (7.2ms - 682rows)
[----] D, [2014-10-02T15:50:09.831310 #30878:3ff2d185e6e4] DEBUG -- :   MiqAeValue Inst (10.0ms - 946rows)
[----] D, [2014-10-02T15:50:10.403363 #30878:3ff2d185e6e4] DEBUG -- :   MiqAeField Inst (554.4ms - 342rows)
[----] I, [2014-10-02T15:50:10.920074 #30878:3ff2d185e6e4]  INFO -- : ZZZ  Completed: ./spec/lib/miq_automation_engine/miq_ae_dialog_spec.rb:259

This issue was moved to this repository from ManageIQ/manageiq#760, originally opened by @jrafanie

Refactor statemachine_task_status methods into mixin

The statemachine_task_status method exists across several service models with only minor differences and should be refactored into a common mixin.

Files to modify:

  • miq_ae_service_automation_task.rb
  • miq_ae_service_miq_host_provision.rb
  • miq_ae_service_miq_provision.rb
  • miq_ae_service_miq_provision_task.rb
  • miq_ae_service_service_reconfigure_task.rb
  • miq_ae_service_service_template_provision_task.rb
  • miq_ae_service_vm_migrate_task.rb

Incorrect error when deleting a VMware ESXi host.

When deleting a VMware ESXi host this unexpected error appears in the evm.log

INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_id=>964368, :event_stream_id=>964368, :event_type=>"HostDisconnectedEvent", "ExtManagementSystem::ems"=>2, :ems_id=>2, "Host::host"=>23, :host_id=>23} for object [EmsEvent.964368] with state [] to Automate
ERROR -- : MIQ(MiqAeEngine.deliver) Error delivering {:event_id=>964368, :event_stream_id=>964368, :event_type=>"HostDisconnectedEvent", "ExtManagementSystem::ems"=>2, :ems_id=>2, "Host::host"=>23, :host_id=>23, "User::user"=>1, "EventStream::event_stream"=>964368} for object [EmsEvent.964368] with state [] to Automate: Service Model not found

Automate export contains empty "options" hash data which breaks backwards compatibility

Following the step in Creating a Pull Request for enhancements to ManageIQ Automate Domain lead to a large number of files being updated to include an empty options hash property.

This was initially address in ManageIQ/manageiq#17080 but no longer effects the export files.

diff --git a/content/automate/ManageIQ/AutomationManagement/AnsibleTower/Operations/Methods.class/__methods__/available_credentials.yaml b/content/automate/ManageIQ/AutomationManagement/AnsibleTower/Operations/Methods.class/__methods__/available_credentials.yaml
index 28ac98c9..63ba2474 100644
--- a/content/automate/ManageIQ/AutomationManagement/AnsibleTower/Operations/Methods.class/__methods__/available_credentials.yaml
+++ b/content/automate/ManageIQ/AutomationManagement/AnsibleTower/Operations/Methods.class/__methods__/available_credentials.yaml
@@ -9,6 +9,7 @@ object:
     scope: instance
     language: ruby
     location: inline
+    options: {}
   inputs:
   - field:
       aetype: 

Also, the steps examples should be updated to match reflect the Automate model in the https://github.com/ManageIQ/manageiq-content repo.

Vmware "memory_reserve_expand" param cannot be set as true

Original BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1811925

Would like to provision a VM through the Vmware Provider with the " Reserve all guest memory (All locked) " box checked inside VMware. NOTE: "memory_reserve_expand" set to true is not working.

Steps to Reproduce:

  1. Create a custom domain having "memory_reserve_expand" set to true
  2. Provision a VM with Vmware provider
  3. Check if the " Reserve all guest memory (All locked) " box is checked inside Vmware.

See BZ for more details.

[RFE] Multi-level dependency resolving with embedded method

Currently, adding an embedded method doesn't resolve its dependencies. For example, one needs to rename a virtual machine in a method. The implementation is different for RHV and VMware, so it would be easier to have a facade method that calls the actual method for the provider.

Say that the method that needs to rename the VM is /Stuff/MyProcess/RenameVM. To be able to call a method from the utility class, it embeds it, say /Stuff/VM/Common/Rename. The actual implementation is provided by /Stuff/VM/VMware/Rename and /Stuff/VM/RHV. Naively, one would simply embed these two methods in /Stuff/VM/Common/Rename. But recursive dependency is not implemented and the methods will have to be embedded in /Stuff/MyProcess/RenameVM.

This RFE asks for recursive dependency implementation, so that embedding methods is kept simple from the user stand point.

Associated RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1609924

Custom message attribute is not considering with instance in logs in automation.log

Not getting custom message in logger file in automation.log
Below are the steps to reproduce issue.

  • setup:
  1. Create domain, namespace, class and instance pointing to method
  • testSteps:
  1. Navigate to automate > automation > simulation page
  2. Fill values for attribute/value pairs of namespace, class, instance and add message attribute with any value and click on submit.
  3. See automation.log
  • expectedResults:
  1. Custom message attribute should be considered with instance in logs

Resolve "warning: already initialized constant" message in tests

/home/travis/build/ManageIQ/manageiq-automation_engine/lib/miq_automation_engine/engine/miq_ae_method_service/miq_ae_service_model_base.rb:115: warning: already initialized constant MiqAeMethodService::MiqAeServiceManageIQ_Providers_InfraManager
/home/travis/build/ManageIQ/manageiq-automation_engine/lib/miq_automation_engine/engine/miq_ae_method_service/miq_ae_service_model_base.rb:115: warning: previous definition of MiqAeServiceManageIQ_Providers_InfraManager was here

Narrowed down the errors to test "loads all mapped models" here:

it "loads all mapped models" do

Related to the LEGACY_MODEL_NAMES mapping here:

Protected dialog fields id key incorrectly contains encrypted string

Found when fixing https://bugzilla.redhat.com/show_bug.cgi?id=1499278.

The URL shows the correct id at first but then by the time the field is being handled by Automate, the id contains the encrypted value and probably shouldn't.

Instantiating [/ImaanN/Methods/testI?MiqServer::miq_server=1&ServiceTemplate::service_template=2&User::user=1&dialog_display_result=Validate Root Password Correct!&object_name=testI&password::dialog_confirm_password=v2:%7BCIHAKUtq3XtK40mtPPHp1Q%3D%3D%7D&password::dialog_root_password=v2:%7Bb7S9YKlblb%2FPcriMeL7OJw%3D%3D%7D&vmdb_object_type=service_template]
[----] I, [2017-10-09T17:09:00.913213 #19729:3ff28ed8492c]  INFO -- : Updated namespace [/ImaanN/Methods/testI?MiqServer::miq_server=1&ServiceTemplate::service_template=2&User::user=1&dialog_display_result=Validate Root Password Correct!&object_name=testI&password::dialog_confirm_password=v2:%7BCIHAKUtq3XtK40mtPPHp1Q%3D%3D%7D&password::dialog_root_password=v2:%7Bb7S9YKlblb%2FPcriMeL7OJw%3D%3D%7D&vmdb_object_type=service_template  ImaanDomain/ImaanN]
...
[----] I, [2017-10-09T17:09:01.473837 #19729:3ff2847af22c]  INFO -- : <AEMethod testi>   Attribute - dialog_confirm_password_id: v2:{CIHAKUtq3XtK40mtPPHp1Q==}
...
[----] I, [2017-10-09T17:09:01.641641 #19729:3ff2847af22c]  INFO -- : <AEMethod testi>   Attribute - dialog_root_password_id: v2:{b7S9YKlblb/PcriMeL7OJw==} 

[RFE] Allow max_retries override at instance level

When one creates a state machine, the maximum number of retries for a state is set at the class level. However, a use case of state machines is to create generic state machines where the state name is 'StateX' and URI is specified only at the instance level, not the class. In this case, to stay generic, one is tempted to set the maximum number of retries to a very high number for each state, to cover whatever situation at the instance level.

Allowing to override the maximum number of retries at the instance level would help creating generic state machine classes (scaffolds) with no maximum retries set, and then let the instance specify the maximum retries based on the actual URI.

[Automate Engine] Looking up objects in the database is expensive

When executing an automation request there are a TON of database queries, and most of them are duplicated. Additionally, since the content of the automate domain doesn't change frequently during execution, we should be able to do some simple caching to eliminate a lot of the performance penalty.

Some ideas...

  • wrapping an automation request in ActiveRecord::QueryCache.cache
  • general caching of the entire domain structure, and busting the cache when an domain is updated and saved.
  • #409
  • optimizing the most common lookups

I'm sure there are other ideas, but it feels like there are likely some really quick wins in here. cc @tinaafitz

Automate expression method broken in jansa and kasparov

Hi,

I was checking jansa with expression methods as described in the Cloudforms / ManageIQ book [1] with user input and couldn't get it to work. It looks like variable resolution is no longer working for expression methods. Or maybe the syntax changed.

Affected versions (or what I tested):
jansa-2.20201027185742_b8d5deb
jansa-3.20210118183104_77bdc30
kasparov-1.20210203001902_15acbea

Steps to reproduce:
Follow the chapter "Expression methods" in the addendum [2]

Expected output:
Expression Method should deliver a result based on variables.

Actual result:
<None>

Additional info:
It is pretty interesting to compare the logging output in automation.log in ivanchuk and in jansa of an expression method:
Ivanchuck:
INFO -- : Invoking [expression] method [/Expression/General/my_expression_check] with inputs [{"arg1"=>"value1", "arg2"=>"foo", "arg3"=>"bar"}]

Jansa/Kasparov:
INFO -- : Invoking [expression] method [/Expression/General/my_expression_check] with inputs [{"arg1"=>"${/#dialog_my_field1}", "arg2"=>"${/#dialog_my_field2}", "arg3"=>"${/#dialog_my_field3}"}]
ERROR -- : Expression method ends

If that's important I tested expression method with tags.

Kind Regards,
phospi

Links

How to handle EmbeddedAnsible roles dependencies

With Embedded I want to use a role that depends on official MIQ automation role synchrou.manageiq_automate.

Q: Should I require synchrou.manageiq_automate at specific release/commit or should I just rely on latest master?

What we currently do is we list it in meta/main.yml as

dependencies:
  - syncrou.manageiq-automate

which means latest master gets installed when user adds Embedded repository. But what can happen - in fact, it happened today :) - is that latest master can get corrupt.

Which is a big problem for Embedded, because as far as I've tested, Embedded would install requirements only at first run, when you add the repository. It's not possible to reinstall them later via CloudForms GUI, but you have to SSH to the appliance and manually handle it. So basically if Embedded fetches a corrupt role, it cannot continue to work, but you need to remove entire repo and re-add it, which is a pain.

Looking forward to some opinion on this topic!

/cc @syncrou @mkanoor

Cleanup the README

The README is a holdover from a previous doc, so it needs some cleanup. I'll go through and do the basics as part of adding the badges, but this will need some general updates and markdown styling as well. We may also want to rip it out of the README and perhaps have a Wiki for this.

cc @mkanoor @gmcculloug @tinaafitz

Datastore import - name source

I've just noticed that we are using the file name as a domain name inside our automate datastore import. For the rest of the objects we are looking into the YAML file and taking the value from the name attribute. I am not sure which way is better, but I think we should be consistent.

Attribute value pairs message is not considered while simulation

Original BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1753523

Description of problem:
If you put 'message' in 'attribute/value pairs' field then it does not consider while simulation.

Version-Release number of selected component (if applicable):
5.11.0.25.20190913144023_5682444

How reproducible:
100%

Steps to Reproduce:

  1. Create domain, namespace, class and instance pointing to method
  2. Navigate to automate > automation > simulation page
  3. Fill values as shown in screenshot.
  4. See automation.logs

Actual results:
[----] I, [2019-09-19T03:29:39.214008 #10804:41c3a00] INFO -- : Followed Relationship [miqaedb:/test_name/test_class/test_instance#create]
[----] I, [2019-09-19T03:29:39.214742 #10804:41c3a00] INFO -- : Followed Relationship [miqaedb:/System/Request/Call_Instance#create]

It does not consider attribute 'message' in logs with test_instance.

Expected results:
[----] I, [2019-09-19T03:29:39.214008 #10804:41c3a00] INFO -- : Followed Relationship [miqaedb:/test_name/test_class/test_instance#hello]
[----] I, [2019-09-19T03:29:39.214742 #10804:41c3a00] INFO -- : Followed Relationship [miqaedb:/System/Request/Call_Instance#create]

It should consider attribute message in logs with test_instance as per BZ fixed for custom button simulation: https://bugzilla.redhat.com/show_bug.cgi?id=1651099

Additional info:

  • This behaviour is same for simulation on 5.10 and 5.11. Hence not adding it as a regression.

image

Change automate methods to communicate via REST API

Much of this was discussed at the ManageIQ design summit for Botvinnik. However, @mkanoor and I recently sat down and brainstormed our way through many of the larger problems, and we think we've come to a cleaner design.

There may be more things I'm not thinking of, but I just wanted to start getting this information out there. Let's keep this document updated as we find more issues and/or come up with more ways to solve them.


Automate currently runs methods by

  • starting a DRb server
  • spawning the ruby method wrapped in a preamble and postamble, with the preamble starting a DRb client
  • client uses service models and helper classes like $evm to communicate with the server

This causes a number of problems and headaches, which may be able to be addressed by moving to a REST API based implementation. Some notable issues:

  • DRb requires server side object management so that the server doesn't garbage collect objects in use by the client. This is a huge headache which will just go away with REST.

  • Every time we add a new object or method we must manually expose it via automate. This is prone to error. This would instead become an exercise in exposing via the REST API, which in nearly all cases should be done anyway.

  • DRb is used to handle interactions with the automate workspace, which is in the server's memory, from the client. This can be handled by dumping the workspace into some format like JSON or YAML, and passing it on STDIN into the launched process.

    The automate method preamble would take STDIN and hold it in memory, lazy loading it on first access. If the method needs to modify the workspace, then it can be modified. $evm would no longer be special and can just be a simple object defined in the preamble that has the knowledge of manipulating the workspace in this manner. $evm.log could even just write directly to the log in question or could write to STDOUT.

    In the postamble we will dump the workspace to STDOUT. In order to prevent collision with user usage of STDOUT, we can prefix the dump with some sort of delimiter (e.g. "=" * 80). STDERR can be used to transfer any exception, or perhaps that becomes part of the workspace that's dumped into STDOUT. Again, a delimiter might be necessary.

    The server, when it detects that the child process has completed, will read from the STDOUT of the child process, find the last delimiter, and process the workspace back into memory, if changed.

    There may be optimizations here to prevent unnecessarily dumping and loading the workspace.

  • DRb is also used for database interaction. The database interaction that is currently done via service models, can be done completely through the REST API. A Ruby gem needs to be created from the cfme_client code, which additionally has more of an OO design. (I'll call it manageiq_client for reference). Once that's available, then the $evm.vmdb can just be a front for accessing those objects from the manageiq_client gem. We could even deprecate $evm.vmdb in favor of having the automate methods use the normal manageiq_client gem directly.

    For example, currently $evm.vmdb["vm", 21] returns a MiqAeServiceVmVmware instance for VM 21. If the REST API gem had it's own OO based types, I could see $evm.vmdb["vm", 21] instead calling the manageiq_client gem, ultimately returning a ManageiqClient::Vm instance for VM 21

    With this in place, I think the entirety of the service models just goes away

  • DRb is also used for built-in methods (like send_email). This can be handled by exposing them from the REST API, and passing the dumped workspace from the method as a parameter to the REST API. These may need some sort of system-only access rules, but then again, maybe not.

Other thoughts:

  • With database interaction via the REST API, and the workspace passed via STDIN/STDOUT in a common format like JSON or YAML, then the support for other languages opens up. These formats are widely supported, so they could be passed to any kind of method whether it be Ruby, Python, Shell, etc. All that would be needed from the server is possibly some preamble/postamble code in the native language to simplify setting up helper objects and catching errors.

TODO

  • Other parts of $evm object that we haven't even looked at like get/set_state_var, instance_*, etc
  • What user would automate methods run as if they are accessing the REST API. For the system things like the built-in methods, it's some system account, but what about other methods? Can they just be system too?

cc @gmcculloug @mkanoor @tinaafitz @chessbyte
cc @abellotti (re: REST API stuff and cfme_client split out into manageiq_client gem)
cc @kbrock (re: system-only access rules)


This issue was moved to this repository from ManageIQ/manageiq#2215, originally opened by @Fryguy

[bug] Git import imports all available domains, but locks only one of them

When I import from git repo with two domains, both of them get imported (not nice, but acceptable when documented). But only one of them is refreshable (assigned git repository) and locked.

When refreshing a domain, it works as expected - only that one domain is refreshed. It would be great to work in the same way when importing...

Embedded Ansible - Add support for ansible.cfg files in user repositories

Original BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1737149

Description of problem:
If a user adds a git repository to be used with embedded ansible and it contains an ansible.cfg file we should use that file when running playbooks.

Version-Release number of selected component (if applicable): 5.11.0.17

Steps to Reproduce:

  1. Add a repo with an ansible.cfg file that overrides some settings (like roles_path)
  2. Run a playbook

Actual results:
Config settings from the repo file are not used.

Expected results:
Repo file config is used.

Additional info:
This is a core ansible feature. The priority for finding config files to use when running a playbook is described here https://docs.ansible.com/ansible/latest/reference_appendices/config.html#the-configuration-file

As a solution for this, we could ensure that our working directory is the one where the ansible.cfg exists when trying to run a playbook.

A word of warning for this though, we install the plugin-provided roles to a custom directory then set roles_path in /root/.ansible.cfg on the appliance.

If we put a user's ansible.cfg in front of ours in that priority list then those roles will be inaccessible for that playbook run. This behavior might be worth documenting.

This issue was originally raised in a comment here ManageIQ/manageiq#19079 (comment)

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

This repository currently has no open or pending branches.

Detected dependencies

github-actions
.github/workflows/ci.yaml
  • actions/checkout v4
  • ruby/setup-ruby v1
  • paambaati/codeclimate-action v5
  • manageiq/postgresql 13

  • Check this box to trigger a request for Renovate to run again on this repository

Error while loading a Service Dialog

Lately I'm getting an error in Rails console while loading a Service Dialog. Not sure if it's issue in API or automate or somewhere else.

I don't know how exactly to reproduce this issue, but it happens a lot on the Dialog attached in the BZ https://bugzilla.redhat.com/show_bug.cgi?id=1518600.

Here's the error message:

F, [2017-12-08T14:47:44.897841 #27922] FATAL -- :                                                                                    
F, [2017-12-08T14:47:44.897964 #27922] FATAL -- : SystemStackError (stack level too deep):                                           
F, [2017-12-08T14:47:44.898034 #27922] FATAL -- :                                                                                    
F, [2017-12-08T14:47:44.898185 #27922] FATAL -- : /home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_method_service.rb:2:in `const_missing'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_method_service.rb:3:in `const_missing'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_method_service.rb:3:in `const_missing'
...
...
...
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_method_service.rb:3:in `const_missing'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_method_service.rb:3:in `const_missing'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_method_service.rb:3:in `const_missing'
activesupport (5.0.6) lib/active_support/inflector/methods.rb:270:in `const_get'
activesupport (5.0.6) lib/active_support/inflector/methods.rb:270:in `block in constantize'
activesupport (5.0.6) lib/active_support/inflector/methods.rb:266:in `each'
activesupport (5.0.6) lib/active_support/inflector/methods.rb:266:in `inject'
activesupport (5.0.6) lib/active_support/inflector/methods.rb:266:in `constantize'
activesupport (5.0.6) lib/active_support/inflector/methods.rb:311:in `safe_constantize'
activesupport (5.0.6) lib/active_support/core_ext/string/inflections.rb:77:in `safe_constantize'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_object.rb:551:in `convert_value_based_on_datatype'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_object.rb:276:in `process_args_attribute'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_object.rb:259:in `block in process_args_as_attributes'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_object.rb:259:in `each'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_object.rb:259:in `process_args_as_attributes'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_workspace_runtime.rb:140:in `instantiate'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_workspace_runtime.rb:49:in `instantiate'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine.rb:301:in `resolve_automation_object'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine.rb:95:in `deliver'
app/models/resource_action.rb:80:in `deliver_to_automate_from_dialog_field'
app/models/dynamic_dialog_field_value_processor.rb:8:in `values_from_automate'
app/models/dynamic_dialog_field_value_processor.rb:3:in `values_from_automate'
app/models/dialog_field.rb:163:in `values_from_automate'
app/models/dialog_field_sorted_item.rb:104:in `raw_values'
app/models/dialog_field_sorted_item.rb:60:in `trigger_automate_value_updates'
app/models/dialog_field_serializer.rb:24:in `serialize'
app/models/dialog_group_serializer.rb:17:in `block in serialize_dialog_fields'
activerecord (5.0.6) lib/active_record/relation/delegation.rb:38:in `map'
activerecord (5.0.6) lib/active_record/relation/delegation.rb:38:in `map'
app/models/dialog_group_serializer.rb:16:in `serialize_dialog_fields'
app/models/dialog_group_serializer.rb:9:in `serialize'
app/models/dialog_tab_serializer.rb:16:in `block in serialize_dialog_groups'
activerecord (5.0.6) lib/active_record/relation/delegation.rb:38:in `map'
activerecord (5.0.6) lib/active_record/relation/delegation.rb:38:in `map'
app/models/dialog_tab_serializer.rb:15:in `serialize_dialog_groups'
app/models/dialog_tab_serializer.rb:9:in `serialize'
app/models/dialog_serializer.rb:16:in `block in serialize_dialog_tabs'
activerecord (5.0.6) lib/active_record/relation/delegation.rb:38:in `map'
activerecord (5.0.6) lib/active_record/relation/delegation.rb:38:in `map'
app/models/dialog_serializer.rb:15:in `serialize_dialog_tabs'
app/models/dialog_serializer.rb:22:in `block in serialize_dialogs'
app/models/dialog_serializer.rb:21:in `map'
app/models/dialog_serializer.rb:21:in `serialize_dialogs'
app/models/dialog_serializer.rb:9:in `serialize'
app/models/dialog.rb:111:in `content'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/service_dialogs_controller.rb:20:in `fetch_service_dialogs_content'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/base_controller/renderer.rb:268:in `fetch_direct_virtual_attribute'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/base_controller/renderer.rb:256:in `block in expand_virtual_attributes'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/base_controller/renderer.rb:252:in `each'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/base_controller/renderer.rb:252:in `expand_virtual_attributes'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/base_controller/renderer.rb:81:in `resource_to_jbuilder'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/base_controller/renderer.rb:17:in `render_resource'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/base_controller.rb:91:in `show'
actionpack (5.0.6) lib/action_controller/metal/basic_implicit_render.rb:4:in `send_action'
actionpack (5.0.6) lib/abstract_controller/base.rb:188:in `process_action'
actionpack (5.0.6) lib/action_controller/metal/rendering.rb:30:in `process_action'
actionpack (5.0.6) lib/abstract_controller/callbacks.rb:20:in `block in process_action'
activesupport (5.0.6) lib/active_support/callbacks.rb:126:in `call'
activesupport (5.0.6) lib/active_support/callbacks.rb:506:in `block (2 levels) in compile'
activesupport (5.0.6) lib/active_support/callbacks.rb:455:in `call'
activesupport (5.0.6) lib/active_support/callbacks.rb:101:in `__run_callbacks__'
activesupport (5.0.6) lib/active_support/callbacks.rb:750:in `_run_process_action_callbacks'
activesupport (5.0.6) lib/active_support/callbacks.rb:90:in `run_callbacks'
actionpack (5.0.6) lib/abstract_controller/callbacks.rb:19:in `process_action'
actionpack (5.0.6) lib/action_controller/metal/rescue.rb:20:in `process_action'
actionpack (5.0.6) lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
activesupport (5.0.6) lib/active_support/notifications.rb:164:in `block in instrument'
activesupport (5.0.6) lib/active_support/notifications/instrumenter.rb:21:in `instrument'
activesupport (5.0.6) lib/active_support/notifications.rb:164:in `instrument'
actionpack (5.0.6) lib/action_controller/metal/instrumentation.rb:30:in `process_action'
actionpack (5.0.6) lib/action_controller/metal/params_wrapper.rb:248:in `process_action'
activerecord (5.0.6) lib/active_record/railties/controller_runtime.rb:18:in `process_action'
actionpack (5.0.6) lib/abstract_controller/base.rb:126:in `process'
actionpack (5.0.6) lib/action_controller/metal.rb:190:in `dispatch'
actionpack (5.0.6) lib/action_controller/metal.rb:262:in `dispatch'
actionpack (5.0.6) lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
actionpack (5.0.6) lib/action_dispatch/routing/route_set.rb:32:in `serve'
actionpack (5.0.6) lib/action_dispatch/journey/router.rb:39:in `block in serve'
actionpack (5.0.6) lib/action_dispatch/journey/router.rb:26:in `each'
actionpack (5.0.6) lib/action_dispatch/journey/router.rb:26:in `serve'
actionpack (5.0.6) lib/action_dispatch/routing/route_set.rb:727:in `call'
secure_headers (3.0.3) lib/secure_headers/middleware.rb:10:in `call'
rack (2.0.3) lib/rack/etag.rb:25:in `call'
rack (2.0.3) lib/rack/conditional_get.rb:25:in `call'
rack (2.0.3) lib/rack/head.rb:12:in `call'
rack (2.0.3) lib/rack/session/abstract/id.rb:232:in `context'
rack (2.0.3) lib/rack/session/abstract/id.rb:226:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/cookies.rb:613:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/callbacks.rb:38:in `block in call'
activesupport (5.0.6) lib/active_support/callbacks.rb:97:in `__run_callbacks__'
activesupport (5.0.6) lib/active_support/callbacks.rb:750:in `_run_call_callbacks'
activesupport (5.0.6) lib/active_support/callbacks.rb:90:in `run_callbacks'
actionpack (5.0.6) lib/action_dispatch/middleware/callbacks.rb:36:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/executor.rb:12:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/remote_ip.rb:79:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/debug_exceptions.rb:49:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
railties (5.0.6) lib/rails/rack/logger.rb:36:in `call_app'
railties (5.0.6) lib/rails/rack/logger.rb:26:in `call'
sprockets-rails (3.2.1) lib/sprockets/rails/quiet_assets.rb:13:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/request_id.rb:24:in `call'
rack (2.0.3) lib/rack/method_override.rb:22:in `call'
rack (2.0.3) lib/rack/runtime.rb:22:in `call'
activesupport (5.0.6) lib/active_support/cache/strategy/local_cache_middleware.rb:28:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/executor.rb:12:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/static.rb:136:in `call'
rack (2.0.3) lib/rack/sendfile.rb:111:in `call'
rack-mini-profiler (0.10.7) lib/mini_profiler/profiler.rb:282:in `call'
railties (5.0.6) lib/rails/engine.rb:522:in `call'
puma (3.7.1) lib/puma/configuration.rb:232:in `call'
puma (3.7.1) lib/puma/server.rb:578:in `handle_request'
puma (3.7.1) lib/puma/server.rb:415:in `process_client'
puma (3.7.1) lib/puma/server.rb:275:in `block in run'
puma (3.7.1) lib/puma/thread_pool.rb:120:in `block in spawn_thread'

Embedded Ansible erroring on 5.10.0.20

Hey guys, Embedded Ansible is periodically failing for me on latest CFME appliance, putting following into evm.log:

[----] E, [2018-10-25T10:25:02.644473 #30085:338f88] ERROR -- : MIQ(ManageIQ::Providers::EmbeddedAnsible::AutomationManager::Refresher#refresh) EMS: [Embedded Ansible Automation Manager], id: [1000000000003] Unable to perform refre
sh for the following targets:
[----] E, [2018-10-25T10:25:02.644651 #30085:338f88] ERROR -- : MIQ(ManageIQ::Providers::EmbeddedAnsible::AutomationManager::Refresher#refresh)  --- ManageIQ::Providers::EmbeddedAnsible::AutomationManager [Embedded Ansible Automati
on Manager] id [1000000000003]
[----] I, [2018-10-25T10:25:02.657125 #30085:338f88]  INFO -- : MIQ(ManageIQ::Providers::EmbeddedAnsible::AutomationManager::Refresher#refresh) Refreshing all targets...Complete
[----] E, [2018-10-25T10:25:02.657456 #30085:338f88] ERROR -- : MIQ(MiqQueue#deliver) Message id: [1000000020918], Error: [undefined method `result_stdout' for #<AnsibleTowerClient::ProjectUpdate:0x0000000016318970>]
[----] E, [2018-10-25T10:25:02.657649 #30085:338f88] ERROR -- : [ManageIQ::Providers::BaseManager::Refresher::PartialRefreshError]: undefined method `result_stdout' for #<AnsibleTowerClient::ProjectUpdate:0x0000000016318970>  Metho
d:[block (2 levels) in <class:LogProxy>]
[----] E, [2018-10-25T10:25:02.657829 #30085:338f88] ERROR -- : /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:67:in `refresh'
/var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:11:in `refresh'
/var/www/miq/vmdb/app/models/ems_refresh.rb:103:in `block in refresh'
/var/www/miq/vmdb/app/models/ems_refresh.rb:102:in `each'
/var/www/miq/vmdb/app/models/ems_refresh.rb:102:in `refresh'
/var/www/miq/vmdb/app/models/miq_queue.rb:455:in `block in dispatch_method'
/usr/share/ruby/timeout.rb:93:in `block in timeout'
/usr/share/ruby/timeout.rb:33:in `block in catch'
/usr/share/ruby/timeout.rb:33:in `catch'
/usr/share/ruby/timeout.rb:33:in `catch'
/usr/share/ruby/timeout.rb:108:in `timeout'
/var/www/miq/vmdb/app/models/miq_queue.rb:453:in `dispatch_method'
/var/www/miq/vmdb/app/models/miq_queue.rb:430:in `block in deliver'
/var/www/miq/vmdb/app/models/user.rb:267:in `with_user_group'
/var/www/miq/vmdb/app/models/miq_queue.rb:430:in `deliver'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:104:in `deliver_queue_message'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:137:in `deliver_message'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:155:in `block in do_work'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:149:in `loop'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:149:in `do_work'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:329:in `block in do_work_loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:326:in `loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:326:in `do_work_loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:153:in `run'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:127:in `start'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:22:in `start_worker'
/var/www/miq/vmdb/app/models/miq_worker.rb:402:in `block in start_runner_via_fork'
/opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.4/lib/nakayoshi_fork.rb:23:in `fork'
/opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.4/lib/nakayoshi_fork.rb:23:in `fork'
/var/www/miq/vmdb/app/models/miq_worker.rb:400:in `start_runner_via_fork'
/var/www/miq/vmdb/app/models/miq_worker.rb:390:in `start_runner'
/var/www/miq/vmdb/app/models/miq_worker.rb:441:in `start'
/var/www/miq/vmdb/app/models/miq_worker.rb:271:in `start_worker'
/var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:74:in `start_worker_for_ems'
/var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:52:in `block in sync_workers'
/var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:51:in `each'
/var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:51:in `sync_workers'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:53:in `block in sync_workers'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `each'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `sync_workers'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:22:in `monitor_workers'
/var/www/miq/vmdb/app/models/miq_server.rb:339:in `block in monitor'
/opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-1ee45ac9278b/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-1ee45ac9278b/lib/gems/pending/util/extensions/miq-benchmark.rb:28:in `realtime_block'
/var/www/miq/vmdb/app/models/miq_server.rb:339:in `monitor'
/var/www/miq/vmdb/app/models/miq_server.rb:380:in `block (2 levels) in monitor_loop'
/opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-1ee45ac9278b/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-1ee45ac9278b/lib/gems/pending/util/extensions/miq-benchmark.rb:35:in `realtime_block'
/var/www/miq/vmdb/app/models/miq_server.rb:380:in `block in monitor_loop'
/var/www/miq/vmdb/app/models/miq_server.rb:379:in `loop'
/var/www/miq/vmdb/app/models/miq_server.rb:379:in `monitor_loop'
/var/www/miq/vmdb/app/models/miq_server.rb:241:in `start'
/var/www/miq/vmdb/lib/workers/evm_server.rb:27:in `start'
/var/www/miq/vmdb/lib/workers/evm_server.rb:48:in `start'
/var/www/miq/vmdb/lib/workers/bin/evm_server.rb:4:in `<main>'

Bug with handling of Arrays in Automate

There's an issue when passing arrays of strings with commas between automate methods (indcluding output from multiselect dialog).

Current Behavior:

Saving ["test1,test2", "test3,test4"] (2 elements) under $evm (or selecting multiple strings with commas in a dialog) and then reading the value in another/different automate method returns ["\"test1", "test2\"", "\"test3", "test4\""] (4 elements)

Expected Behavior:

Saving an array of strings in one method should produce the exact same array when reading the value in another method.

Steps To Reproduce:

  1. Create two automate methods that are run one after another.
  2. Save the above array under $evm in the first method.
    $evm.root['test'] = ["test1,test2", "test3,test4"]
  3. Read the value in the second method.
    $evm.log('info', "Array: #{$evm.root['test']}")
  4. See a different array returned than was saved.

Environment:

ManageIQ Lasker-1

Anything else:

I suspect the split method here:

return value.gsub(/[\[\]]/, '').strip.split(/\s*,\s*/) if datatype == 'array' && value.class == String

splits the string (array) on every comma even if they are a part of a string and not a delimiter between elements.

Using eval() would solve this problem, such as many others (nested arrays or other comma issues). Though I must admit I don't know how the values are handled on the other end to judge if this would be a safe use of the eval method.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.