manageiq / manageiq-automation_engine Goto Github PK
View Code? Open in Web Editor NEWAutomation engine for ManageIQ
License: Apache License 2.0
Automation engine for ManageIQ
License: Apache License 2.0
The statemachine_task_status
method exists across several service models with only minor differences and should be refactored into a common mixin.
Files to modify:
Deprecate remote_url
.
Make it call external_url
or use the same column.
Add a migration that removes the URL column from the SystemConsole
table (check if it's not used by the proxying piece).
Later on remove the remove_url
mixin.
More background available in #328
MiqAeNamespace is requested to walk its tree fairly frequently, but since it uses acts_as_tree, it effectively emits 1 query per level of the tree. One common path for this is asking an object's fqname. If MiqAeNamespace used ancestry, then it would be one query to get the fqname.
cc @tinaafitz
A user with only a tenant quota for memory set to 256GB is trying to reconfigure a VM and add an additional HDD to his machine. We are seeing an error in the quota used method when calling check_quota. When the quota is turned off then he is able to provision the drive without error.
We are running MIQ Fine-4 and VMware vcenter 6.5.
Gist contains the logs for the vm_reconfigure_request.
https://gist.github.com/Stromweld/d59742714eda7e732378e262217dd412
From automate editor code, be able to create feature/bugfix GIT branch in order to follow the best practices regarding code management
Also see: http://talk.manageiq.org/t/solved-git-flow-for-automate-code/3362
Hey guys, Embedded Ansible is periodically failing for me on latest CFME appliance, putting following into evm.log:
[----] E, [2018-10-25T10:25:02.644473 #30085:338f88] ERROR -- : MIQ(ManageIQ::Providers::EmbeddedAnsible::AutomationManager::Refresher#refresh) EMS: [Embedded Ansible Automation Manager], id: [1000000000003] Unable to perform refre
sh for the following targets:
[----] E, [2018-10-25T10:25:02.644651 #30085:338f88] ERROR -- : MIQ(ManageIQ::Providers::EmbeddedAnsible::AutomationManager::Refresher#refresh) --- ManageIQ::Providers::EmbeddedAnsible::AutomationManager [Embedded Ansible Automati
on Manager] id [1000000000003]
[----] I, [2018-10-25T10:25:02.657125 #30085:338f88] INFO -- : MIQ(ManageIQ::Providers::EmbeddedAnsible::AutomationManager::Refresher#refresh) Refreshing all targets...Complete
[----] E, [2018-10-25T10:25:02.657456 #30085:338f88] ERROR -- : MIQ(MiqQueue#deliver) Message id: [1000000020918], Error: [undefined method `result_stdout' for #<AnsibleTowerClient::ProjectUpdate:0x0000000016318970>]
[----] E, [2018-10-25T10:25:02.657649 #30085:338f88] ERROR -- : [ManageIQ::Providers::BaseManager::Refresher::PartialRefreshError]: undefined method `result_stdout' for #<AnsibleTowerClient::ProjectUpdate:0x0000000016318970> Metho
d:[block (2 levels) in <class:LogProxy>]
[----] E, [2018-10-25T10:25:02.657829 #30085:338f88] ERROR -- : /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:67:in `refresh'
/var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:11:in `refresh'
/var/www/miq/vmdb/app/models/ems_refresh.rb:103:in `block in refresh'
/var/www/miq/vmdb/app/models/ems_refresh.rb:102:in `each'
/var/www/miq/vmdb/app/models/ems_refresh.rb:102:in `refresh'
/var/www/miq/vmdb/app/models/miq_queue.rb:455:in `block in dispatch_method'
/usr/share/ruby/timeout.rb:93:in `block in timeout'
/usr/share/ruby/timeout.rb:33:in `block in catch'
/usr/share/ruby/timeout.rb:33:in `catch'
/usr/share/ruby/timeout.rb:33:in `catch'
/usr/share/ruby/timeout.rb:108:in `timeout'
/var/www/miq/vmdb/app/models/miq_queue.rb:453:in `dispatch_method'
/var/www/miq/vmdb/app/models/miq_queue.rb:430:in `block in deliver'
/var/www/miq/vmdb/app/models/user.rb:267:in `with_user_group'
/var/www/miq/vmdb/app/models/miq_queue.rb:430:in `deliver'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:104:in `deliver_queue_message'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:137:in `deliver_message'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:155:in `block in do_work'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:149:in `loop'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:149:in `do_work'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:329:in `block in do_work_loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:326:in `loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:326:in `do_work_loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:153:in `run'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:127:in `start'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:22:in `start_worker'
/var/www/miq/vmdb/app/models/miq_worker.rb:402:in `block in start_runner_via_fork'
/opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.4/lib/nakayoshi_fork.rb:23:in `fork'
/opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.4/lib/nakayoshi_fork.rb:23:in `fork'
/var/www/miq/vmdb/app/models/miq_worker.rb:400:in `start_runner_via_fork'
/var/www/miq/vmdb/app/models/miq_worker.rb:390:in `start_runner'
/var/www/miq/vmdb/app/models/miq_worker.rb:441:in `start'
/var/www/miq/vmdb/app/models/miq_worker.rb:271:in `start_worker'
/var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:74:in `start_worker_for_ems'
/var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:52:in `block in sync_workers'
/var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:51:in `each'
/var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:51:in `sync_workers'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:53:in `block in sync_workers'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `each'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `sync_workers'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:22:in `monitor_workers'
/var/www/miq/vmdb/app/models/miq_server.rb:339:in `block in monitor'
/opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-1ee45ac9278b/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-1ee45ac9278b/lib/gems/pending/util/extensions/miq-benchmark.rb:28:in `realtime_block'
/var/www/miq/vmdb/app/models/miq_server.rb:339:in `monitor'
/var/www/miq/vmdb/app/models/miq_server.rb:380:in `block (2 levels) in monitor_loop'
/opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-1ee45ac9278b/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-1ee45ac9278b/lib/gems/pending/util/extensions/miq-benchmark.rb:35:in `realtime_block'
/var/www/miq/vmdb/app/models/miq_server.rb:380:in `block in monitor_loop'
/var/www/miq/vmdb/app/models/miq_server.rb:379:in `loop'
/var/www/miq/vmdb/app/models/miq_server.rb:379:in `monitor_loop'
/var/www/miq/vmdb/app/models/miq_server.rb:241:in `start'
/var/www/miq/vmdb/lib/workers/evm_server.rb:27:in `start'
/var/www/miq/vmdb/lib/workers/evm_server.rb:48:in `start'
/var/www/miq/vmdb/lib/workers/bin/evm_server.rb:4:in `<main>'
When I import from git repo with two domains, both of them get imported (not nice, but acceptable when documented). But only one of them is refreshable (assigned git repository) and locked.
When refreshing a domain, it works as expected - only that one domain is refreshed. It would be great to work in the same way when importing...
For details see: https://bugzilla.redhat.com/show_bug.cgi?id=1824764
Description of problem:
While Updating the method we are not getting a proper error message. It does not tell what is a problem actually.
Lately I'm getting an error in Rails console while loading a Service Dialog. Not sure if it's issue in API or automate or somewhere else.
I don't know how exactly to reproduce this issue, but it happens a lot on the Dialog attached in the BZ https://bugzilla.redhat.com/show_bug.cgi?id=1518600.
Here's the error message:
F, [2017-12-08T14:47:44.897841 #27922] FATAL -- :
F, [2017-12-08T14:47:44.897964 #27922] FATAL -- : SystemStackError (stack level too deep):
F, [2017-12-08T14:47:44.898034 #27922] FATAL -- :
F, [2017-12-08T14:47:44.898185 #27922] FATAL -- : /home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_method_service.rb:2:in `const_missing'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_method_service.rb:3:in `const_missing'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_method_service.rb:3:in `const_missing'
...
...
...
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_method_service.rb:3:in `const_missing'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_method_service.rb:3:in `const_missing'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_method_service.rb:3:in `const_missing'
activesupport (5.0.6) lib/active_support/inflector/methods.rb:270:in `const_get'
activesupport (5.0.6) lib/active_support/inflector/methods.rb:270:in `block in constantize'
activesupport (5.0.6) lib/active_support/inflector/methods.rb:266:in `each'
activesupport (5.0.6) lib/active_support/inflector/methods.rb:266:in `inject'
activesupport (5.0.6) lib/active_support/inflector/methods.rb:266:in `constantize'
activesupport (5.0.6) lib/active_support/inflector/methods.rb:311:in `safe_constantize'
activesupport (5.0.6) lib/active_support/core_ext/string/inflections.rb:77:in `safe_constantize'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_object.rb:551:in `convert_value_based_on_datatype'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_object.rb:276:in `process_args_attribute'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_object.rb:259:in `block in process_args_as_attributes'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_object.rb:259:in `each'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_object.rb:259:in `process_args_as_attributes'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_workspace_runtime.rb:140:in `instantiate'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_workspace_runtime.rb:49:in `instantiate'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine.rb:301:in `resolve_automation_object'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-automation_engine-a71f1de4e04f/lib/miq_automation_engine/engine/miq_ae_engine.rb:95:in `deliver'
app/models/resource_action.rb:80:in `deliver_to_automate_from_dialog_field'
app/models/dynamic_dialog_field_value_processor.rb:8:in `values_from_automate'
app/models/dynamic_dialog_field_value_processor.rb:3:in `values_from_automate'
app/models/dialog_field.rb:163:in `values_from_automate'
app/models/dialog_field_sorted_item.rb:104:in `raw_values'
app/models/dialog_field_sorted_item.rb:60:in `trigger_automate_value_updates'
app/models/dialog_field_serializer.rb:24:in `serialize'
app/models/dialog_group_serializer.rb:17:in `block in serialize_dialog_fields'
activerecord (5.0.6) lib/active_record/relation/delegation.rb:38:in `map'
activerecord (5.0.6) lib/active_record/relation/delegation.rb:38:in `map'
app/models/dialog_group_serializer.rb:16:in `serialize_dialog_fields'
app/models/dialog_group_serializer.rb:9:in `serialize'
app/models/dialog_tab_serializer.rb:16:in `block in serialize_dialog_groups'
activerecord (5.0.6) lib/active_record/relation/delegation.rb:38:in `map'
activerecord (5.0.6) lib/active_record/relation/delegation.rb:38:in `map'
app/models/dialog_tab_serializer.rb:15:in `serialize_dialog_groups'
app/models/dialog_tab_serializer.rb:9:in `serialize'
app/models/dialog_serializer.rb:16:in `block in serialize_dialog_tabs'
activerecord (5.0.6) lib/active_record/relation/delegation.rb:38:in `map'
activerecord (5.0.6) lib/active_record/relation/delegation.rb:38:in `map'
app/models/dialog_serializer.rb:15:in `serialize_dialog_tabs'
app/models/dialog_serializer.rb:22:in `block in serialize_dialogs'
app/models/dialog_serializer.rb:21:in `map'
app/models/dialog_serializer.rb:21:in `serialize_dialogs'
app/models/dialog_serializer.rb:9:in `serialize'
app/models/dialog.rb:111:in `content'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/service_dialogs_controller.rb:20:in `fetch_service_dialogs_content'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/base_controller/renderer.rb:268:in `fetch_direct_virtual_attribute'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/base_controller/renderer.rb:256:in `block in expand_virtual_attributes'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/base_controller/renderer.rb:252:in `each'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/base_controller/renderer.rb:252:in `expand_virtual_attributes'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/base_controller/renderer.rb:81:in `resource_to_jbuilder'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/base_controller/renderer.rb:17:in `render_resource'
/home/rblanco/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/manageiq-api-462b79b228b4/app/controllers/api/base_controller.rb:91:in `show'
actionpack (5.0.6) lib/action_controller/metal/basic_implicit_render.rb:4:in `send_action'
actionpack (5.0.6) lib/abstract_controller/base.rb:188:in `process_action'
actionpack (5.0.6) lib/action_controller/metal/rendering.rb:30:in `process_action'
actionpack (5.0.6) lib/abstract_controller/callbacks.rb:20:in `block in process_action'
activesupport (5.0.6) lib/active_support/callbacks.rb:126:in `call'
activesupport (5.0.6) lib/active_support/callbacks.rb:506:in `block (2 levels) in compile'
activesupport (5.0.6) lib/active_support/callbacks.rb:455:in `call'
activesupport (5.0.6) lib/active_support/callbacks.rb:101:in `__run_callbacks__'
activesupport (5.0.6) lib/active_support/callbacks.rb:750:in `_run_process_action_callbacks'
activesupport (5.0.6) lib/active_support/callbacks.rb:90:in `run_callbacks'
actionpack (5.0.6) lib/abstract_controller/callbacks.rb:19:in `process_action'
actionpack (5.0.6) lib/action_controller/metal/rescue.rb:20:in `process_action'
actionpack (5.0.6) lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
activesupport (5.0.6) lib/active_support/notifications.rb:164:in `block in instrument'
activesupport (5.0.6) lib/active_support/notifications/instrumenter.rb:21:in `instrument'
activesupport (5.0.6) lib/active_support/notifications.rb:164:in `instrument'
actionpack (5.0.6) lib/action_controller/metal/instrumentation.rb:30:in `process_action'
actionpack (5.0.6) lib/action_controller/metal/params_wrapper.rb:248:in `process_action'
activerecord (5.0.6) lib/active_record/railties/controller_runtime.rb:18:in `process_action'
actionpack (5.0.6) lib/abstract_controller/base.rb:126:in `process'
actionpack (5.0.6) lib/action_controller/metal.rb:190:in `dispatch'
actionpack (5.0.6) lib/action_controller/metal.rb:262:in `dispatch'
actionpack (5.0.6) lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
actionpack (5.0.6) lib/action_dispatch/routing/route_set.rb:32:in `serve'
actionpack (5.0.6) lib/action_dispatch/journey/router.rb:39:in `block in serve'
actionpack (5.0.6) lib/action_dispatch/journey/router.rb:26:in `each'
actionpack (5.0.6) lib/action_dispatch/journey/router.rb:26:in `serve'
actionpack (5.0.6) lib/action_dispatch/routing/route_set.rb:727:in `call'
secure_headers (3.0.3) lib/secure_headers/middleware.rb:10:in `call'
rack (2.0.3) lib/rack/etag.rb:25:in `call'
rack (2.0.3) lib/rack/conditional_get.rb:25:in `call'
rack (2.0.3) lib/rack/head.rb:12:in `call'
rack (2.0.3) lib/rack/session/abstract/id.rb:232:in `context'
rack (2.0.3) lib/rack/session/abstract/id.rb:226:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/cookies.rb:613:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/callbacks.rb:38:in `block in call'
activesupport (5.0.6) lib/active_support/callbacks.rb:97:in `__run_callbacks__'
activesupport (5.0.6) lib/active_support/callbacks.rb:750:in `_run_call_callbacks'
activesupport (5.0.6) lib/active_support/callbacks.rb:90:in `run_callbacks'
actionpack (5.0.6) lib/action_dispatch/middleware/callbacks.rb:36:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/executor.rb:12:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/remote_ip.rb:79:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/debug_exceptions.rb:49:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
railties (5.0.6) lib/rails/rack/logger.rb:36:in `call_app'
railties (5.0.6) lib/rails/rack/logger.rb:26:in `call'
sprockets-rails (3.2.1) lib/sprockets/rails/quiet_assets.rb:13:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/request_id.rb:24:in `call'
rack (2.0.3) lib/rack/method_override.rb:22:in `call'
rack (2.0.3) lib/rack/runtime.rb:22:in `call'
activesupport (5.0.6) lib/active_support/cache/strategy/local_cache_middleware.rb:28:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/executor.rb:12:in `call'
actionpack (5.0.6) lib/action_dispatch/middleware/static.rb:136:in `call'
rack (2.0.3) lib/rack/sendfile.rb:111:in `call'
rack-mini-profiler (0.10.7) lib/mini_profiler/profiler.rb:282:in `call'
railties (5.0.6) lib/rails/engine.rb:522:in `call'
puma (3.7.1) lib/puma/configuration.rb:232:in `call'
puma (3.7.1) lib/puma/server.rb:578:in `handle_request'
puma (3.7.1) lib/puma/server.rb:415:in `process_client'
puma (3.7.1) lib/puma/server.rb:275:in `block in run'
puma (3.7.1) lib/puma/thread_pool.rb:120:in `block in spawn_thread'
Found when fixing https://bugzilla.redhat.com/show_bug.cgi?id=1499278.
The URL shows the correct id at first but then by the time the field is being handled by Automate, the id contains the encrypted value and probably shouldn't.
Instantiating [/ImaanN/Methods/testI?MiqServer::miq_server=1&ServiceTemplate::service_template=2&User::user=1&dialog_display_result=Validate Root Password Correct!&object_name=testI&password::dialog_confirm_password=v2:%7BCIHAKUtq3XtK40mtPPHp1Q%3D%3D%7D&password::dialog_root_password=v2:%7Bb7S9YKlblb%2FPcriMeL7OJw%3D%3D%7D&vmdb_object_type=service_template]
[----] I, [2017-10-09T17:09:00.913213 #19729:3ff28ed8492c] INFO -- : Updated namespace [/ImaanN/Methods/testI?MiqServer::miq_server=1&ServiceTemplate::service_template=2&User::user=1&dialog_display_result=Validate Root Password Correct!&object_name=testI&password::dialog_confirm_password=v2:%7BCIHAKUtq3XtK40mtPPHp1Q%3D%3D%7D&password::dialog_root_password=v2:%7Bb7S9YKlblb%2FPcriMeL7OJw%3D%3D%7D&vmdb_object_type=service_template ImaanDomain/ImaanN]
...
[----] I, [2017-10-09T17:09:01.473837 #19729:3ff2847af22c] INFO -- : <AEMethod testi> Attribute - dialog_confirm_password_id: v2:{CIHAKUtq3XtK40mtPPHp1Q==}
...
[----] I, [2017-10-09T17:09:01.641641 #19729:3ff2847af22c] INFO -- : <AEMethod testi> Attribute - dialog_root_password_id: v2:{b7S9YKlblb/PcriMeL7OJw==}
With Embedded I want to use a role that depends on official MIQ automation role synchrou.manageiq_automate.
Q: Should I require synchrou.manageiq_automate
at specific release/commit or should I just rely on latest master?
What we currently do is we list it in meta/main.yml as
dependencies:
- syncrou.manageiq-automate
which means latest master gets installed when user adds Embedded repository. But what can happen - in fact, it happened today :) - is that latest master can get corrupt.
Which is a big problem for Embedded, because as far as I've tested, Embedded would install requirements only at first run, when you add the repository. It's not possible to reinstall them later via CloudForms GUI, but you have to SSH to the appliance and manually handle it. So basically if Embedded fetches a corrupt role, it cannot continue to work, but you need to remove entire repo and re-add it, which is a pain.
Looking forward to some opinion on this topic!
/home/travis/build/ManageIQ/manageiq-automation_engine/lib/miq_automation_engine/engine/miq_ae_method_service/miq_ae_service_model_base.rb:115: warning: already initialized constant MiqAeMethodService::MiqAeServiceManageIQ_Providers_InfraManager
/home/travis/build/ManageIQ/manageiq-automation_engine/lib/miq_automation_engine/engine/miq_ae_method_service/miq_ae_service_model_base.rb:115: warning: previous definition of MiqAeServiceManageIQ_Providers_InfraManager was here
Narrowed down the errors to test "loads all mapped models" here:
Related to the LEGACY_MODEL_NAMES
mapping here:
Looking to expose aws cloud_volume methods to automate
create/delete/attach volume and create/delete snapshot
miq_ae_service_manageiq-providers-cloud_manager-vm.rb
add expose :cloud_volumes, :association => true
miq_ae_service_cloud_volume.rb
add expose :create_volume_snapshot, :override_return => nil
Original BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1753523
Description of problem:
If you put 'message' in 'attribute/value pairs' field then it does not consider while simulation.
Version-Release number of selected component (if applicable):
5.11.0.25.20190913144023_5682444
How reproducible:
100%
Steps to Reproduce:
Actual results:
[----] I, [2019-09-19T03:29:39.214008 #10804:41c3a00] INFO -- : Followed Relationship [miqaedb:/test_name/test_class/test_instance#create]
[----] I, [2019-09-19T03:29:39.214742 #10804:41c3a00] INFO -- : Followed Relationship [miqaedb:/System/Request/Call_Instance#create]
It does not consider attribute 'message' in logs with test_instance.
Expected results:
[----] I, [2019-09-19T03:29:39.214008 #10804:41c3a00] INFO -- : Followed Relationship [miqaedb:/test_name/test_class/test_instance#hello]
[----] I, [2019-09-19T03:29:39.214742 #10804:41c3a00] INFO -- : Followed Relationship [miqaedb:/System/Request/Call_Instance#create]
It should consider attribute message in logs with test_instance as per BZ fixed for custom button simulation: https://bugzilla.redhat.com/show_bug.cgi?id=1651099
Additional info:
Much of this was discussed at the ManageIQ design summit for Botvinnik. However, @mkanoor and I recently sat down and brainstormed our way through many of the larger problems, and we think we've come to a cleaner design.
There may be more things I'm not thinking of, but I just wanted to start getting this information out there. Let's keep this document updated as we find more issues and/or come up with more ways to solve them.
Automate currently runs methods by
This causes a number of problems and headaches, which may be able to be addressed by moving to a REST API based implementation. Some notable issues:
DRb requires server side object management so that the server doesn't garbage collect objects in use by the client. This is a huge headache which will just go away with REST.
Every time we add a new object or method we must manually expose it via automate. This is prone to error. This would instead become an exercise in exposing via the REST API, which in nearly all cases should be done anyway.
DRb is used to handle interactions with the automate workspace, which is in the server's memory, from the client. This can be handled by dumping the workspace into some format like JSON or YAML, and passing it on STDIN into the launched process.
The automate method preamble would take STDIN and hold it in memory, lazy loading it on first access. If the method needs to modify the workspace, then it can be modified. $evm
would no longer be special and can just be a simple object defined in the preamble that has the knowledge of manipulating the workspace in this manner. $evm.log
could even just write directly to the log in question or could write to STDOUT.
In the postamble we will dump the workspace to STDOUT. In order to prevent collision with user usage of STDOUT, we can prefix the dump with some sort of delimiter (e.g. "=" * 80
). STDERR can be used to transfer any exception, or perhaps that becomes part of the workspace that's dumped into STDOUT. Again, a delimiter might be necessary.
The server, when it detects that the child process has completed, will read from the STDOUT of the child process, find the last delimiter, and process the workspace back into memory, if changed.
There may be optimizations here to prevent unnecessarily dumping and loading the workspace.
DRb is also used for database interaction. The database interaction that is currently done via service models, can be done completely through the REST API. A Ruby gem needs to be created from the cfme_client code, which additionally has more of an OO design. (I'll call it manageiq_client for reference). Once that's available, then the $evm.vmdb
can just be a front for accessing those objects from the manageiq_client gem. We could even deprecate $evm.vmdb
in favor of having the automate methods use the normal manageiq_client gem directly.
For example, currently $evm.vmdb["vm", 21]
returns a MiqAeServiceVmVmware
instance for VM 21. If the REST API gem had it's own OO based types, I could see $evm.vmdb["vm", 21]
instead calling the manageiq_client gem, ultimately returning a ManageiqClient::Vm
instance for VM 21
With this in place, I think the entirety of the service models just goes away
DRb is also used for built-in methods (like send_email). This can be handled by exposing them from the REST API, and passing the dumped workspace from the method as a parameter to the REST API. These may need some sort of system-only access rules, but then again, maybe not.
Other thoughts:
TODO
$evm
object that we haven't even looked at like get/set_state_var
, instance_*
, etccc @gmcculloug @mkanoor @tinaafitz @chessbyte
cc @abellotti (re: REST API stuff and cfme_client split out into manageiq_client gem)
cc @kbrock (re: system-only access rules)
This issue was moved to this repository from ManageIQ/manageiq#2215, originally opened by @Fryguy
Modify the data that YAML encodes for a service model object to just the class name and object ID, which is the ID of the backing ActiveRecord object.
Also see: https://stackoverflow.com/questions/10629209/how-can-i-control-which-fields-to-serialize-with-yaml
When deleting a VMware ESXi host this unexpected error appears in the evm.log
INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_id=>964368, :event_stream_id=>964368, :event_type=>"HostDisconnectedEvent", "ExtManagementSystem::ems"=>2, :ems_id=>2, "Host::host"=>23, :host_id=>23} for object [EmsEvent.964368] with state [] to Automate
ERROR -- : MIQ(MiqAeEngine.deliver) Error delivering {:event_id=>964368, :event_stream_id=>964368, :event_type=>"HostDisconnectedEvent", "ExtManagementSystem::ems"=>2, :ems_id=>2, "Host::host"=>23, :host_id=>23, "User::user"=>1, "EventStream::event_stream"=>964368} for object [EmsEvent.964368] with state [] to Automate: Service Model not found
Following the step in Creating a Pull Request for enhancements to ManageIQ Automate Domain lead to a large number of files being updated to include an empty options
hash property.
This was initially address in ManageIQ/manageiq#17080 but no longer effects the export files.
diff --git a/content/automate/ManageIQ/AutomationManagement/AnsibleTower/Operations/Methods.class/__methods__/available_credentials.yaml b/content/automate/ManageIQ/AutomationManagement/AnsibleTower/Operations/Methods.class/__methods__/available_credentials.yaml
index 28ac98c9..63ba2474 100644
--- a/content/automate/ManageIQ/AutomationManagement/AnsibleTower/Operations/Methods.class/__methods__/available_credentials.yaml
+++ b/content/automate/ManageIQ/AutomationManagement/AnsibleTower/Operations/Methods.class/__methods__/available_credentials.yaml
@@ -9,6 +9,7 @@ object:
scope: instance
language: ruby
location: inline
+ options: {}
inputs:
- field:
aetype:
Also, the steps examples should be updated to match reflect the Automate model in the https://github.com/ManageIQ/manageiq-content repo.
If the audience of the notification is set to user
, it is always being sent to the @workspace.ae_user
and there's no possibility to notify e.g. the admin. A possible workaround would be to send the notification to the group of the admin user, however, this is not feasible as we're bound to notification types.
When one creates a state machine, the maximum number of retries for a state is set at the class level. However, a use case of state machines is to create generic state machines where the state name is 'StateX' and URI is specified only at the instance level, not the class. In this case, to stay generic, one is tempted to set the maximum number of retries to a very high number for each state, to cover whatever situation at the instance level.
Allowing to override the maximum number of retries at the instance level would help creating generic state machine classes (scaffolds) with no maximum retries set, and then let the instance specify the maximum retries based on the actual URI.
I've just noticed that we are using the file name as a domain name inside our automate datastore import. For the rest of the objects we are looking into the YAML file and taking the value from the name attribute. I am not sure which way is better, but I think we should be consistent.
See #419 (comment)
The following classes are affected by the queue_name change but do not have proper test coverage:
When executing an automation request there are a TON of database queries, and most of them are duplicated. Additionally, since the content of the automate domain doesn't change frequently during execution, we should be able to do some simple caching to eliminate a lot of the performance penalty.
Some ideas...
I'm sure there are other ideas, but it feels like there are likely some really quick wins in here. cc @tinaafitz
Currently, adding an embedded method doesn't resolve its dependencies. For example, one needs to rename a virtual machine in a method. The implementation is different for RHV and VMware, so it would be easier to have a facade method that calls the actual method for the provider.
Say that the method that needs to rename the VM is /Stuff/MyProcess/RenameVM
. To be able to call a method from the utility class, it embeds it, say /Stuff/VM/Common/Rename
. The actual implementation is provided by /Stuff/VM/VMware/Rename
and /Stuff/VM/RHV
. Naively, one would simply embed these two methods in /Stuff/VM/Common/Rename
. But recursive dependency is not implemented and the methods will have to be embedded in /Stuff/MyProcess/RenameVM
.
This RFE asks for recursive dependency implementation, so that embedding methods is kept simple from the user stand point.
Associated RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1609924
The README is a holdover from a previous doc, so it needs some cleanup. I'll go through and do the basics as part of adding the badges, but this will need some general updates and markdown styling as well. We may also want to rip it out of the README and perhaps have a Wiki for this.
Related BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1653492
Business use-case requires that we efficiently apply a quota at the group level, which it is today, however we need this quota to map to each user.
For example, group1, has a quota of: 10 vCPU, 5GB RAM, and 1T Disk
Today: if group1 has 10 users, that quota is shared between all 10 users, so if user1 uses 8 vCPU and 4GB of RAM, there is little space left.
What we need: we need each user in group1 to have 10 vCPU, 5GB of RAM, and 1T quota associated with their user account. Additionally, it would be great to allocate instances/VMs to that user as well.
This would need to be an option to enable/disable, or even combine with the current quota implementation.
Related BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1659092
When passing a list as an extra variable to an ansible playbook, the list is always treated as a string. Should be able to insert a list into an ansible playbook as an extra variable.
Original BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1662972
Steps to Reproduce:
Actual results:
When I select an image and then click on continue I get Error requesting data from server
There's an issue when passing arrays of strings with commas between automate methods (indcluding output from multiselect dialog).
Saving ["test1,test2", "test3,test4"]
(2 elements) under $evm (or selecting multiple strings with commas in a dialog) and then reading the value in another/different automate method returns ["\"test1", "test2\"", "\"test3", "test4\""]
(4 elements)
Saving an array of strings in one method should produce the exact same array when reading the value in another method.
$evm.root['test'] = ["test1,test2", "test3,test4"]
$evm.log('info', "Array: #{$evm.root['test']}")
ManageIQ Lasker-1
I suspect the split method here:
Using eval() would solve this problem, such as many others (nested arrays or other comma issues). Though I must admit I don't know how the values are handled on the other end to judge if this would be a safe use of the eval method.
This single example takes ~21 seconds on travis and creates loads of DB records.
[----] I, [2014-10-02T15:49:47.315131 #30878:3ff2d185e6e4] INFO -- : ZZZ Starting: ./spec/lib/miq_automation_engine/miq_ae_dialog_spec.rb:259
[----] I, [2014-10-02T15:49:47.315178 #30878:3ff2d185e6e4] INFO -- : ZZZZ From Example group: ./spec/lib/miq_automation_engine/miq_ae_dialog_spec.rb:5
[----] D, [2014-10-02T15:50:06.992935 #30878:3ff2d185e6e4] DEBUG -- : MiqAeClass Inst (1.7ms - 142rows)
[----] D, [2014-10-02T15:50:07.010563 #30878:3ff2d185e6e4] DEBUG -- : MiqAeField Inst (4.8ms - 424rows)
[----] D, [2014-10-02T15:50:07.029055 #30878:3ff2d185e6e4] DEBUG -- : MiqAeInstance Inst (7.3ms - 682rows)
[----] D, [2014-10-02T15:50:07.064661 #30878:3ff2d185e6e4] DEBUG -- : MiqAeValue Inst (10.6ms - 946rows)
[----] D, [2014-10-02T15:50:07.087512 #30878:3ff2d185e6e4] DEBUG -- : MiqAeField Inst (3.8ms - 342rows)
[----] D, [2014-10-02T15:50:09.762381 #30878:3ff2d185e6e4] DEBUG -- : MiqAeClass Inst (1.5ms - 142rows)
[----] D, [2014-10-02T15:50:09.779401 #30878:3ff2d185e6e4] DEBUG -- : MiqAeField Inst (4.7ms - 424rows)
[----] D, [2014-10-02T15:50:09.797099 #30878:3ff2d185e6e4] DEBUG -- : MiqAeInstance Inst (7.2ms - 682rows)
[----] D, [2014-10-02T15:50:09.831310 #30878:3ff2d185e6e4] DEBUG -- : MiqAeValue Inst (10.0ms - 946rows)
[----] D, [2014-10-02T15:50:10.403363 #30878:3ff2d185e6e4] DEBUG -- : MiqAeField Inst (554.4ms - 342rows)
[----] I, [2014-10-02T15:50:10.920074 #30878:3ff2d185e6e4] INFO -- : ZZZ Completed: ./spec/lib/miq_automation_engine/miq_ae_dialog_spec.rb:259
This issue was moved to this repository from ManageIQ/manageiq#760, originally opened by @jrafanie
Followup work from https://bugzilla.redhat.com/show_bug.cgi?id=1461427
cc @d-m-u
Refactor based on comments in: ManageIQ/manageiq#12539 (comment)
This issue was moved to this repository from ManageIQ/manageiq#12696, originally opened by @gmcculloug
Currently every method in automate is configured to run in Drb and considered a user provided script. However manageiq is shipped with many automate methods which are not really user provided. We should introduce internal automate methods that run default implementations internally but allow users to override with an external script.
Not getting custom message in logger file in automation.log
Below are the steps to reproduce issue.
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
This repository currently has no open or pending branches.
.github/workflows/ci.yaml
actions/checkout v4
ruby/setup-ruby v1
paambaati/codeclimate-action v6
manageiq/postgresql 13
Original BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1737149
Description of problem:
If a user adds a git repository to be used with embedded ansible and it contains an ansible.cfg file we should use that file when running playbooks.
Version-Release number of selected component (if applicable): 5.11.0.17
Steps to Reproduce:
Actual results:
Config settings from the repo file are not used.
Expected results:
Repo file config is used.
Additional info:
This is a core ansible feature. The priority for finding config files to use when running a playbook is described here https://docs.ansible.com/ansible/latest/reference_appendices/config.html#the-configuration-file
As a solution for this, we could ensure that our working directory is the one where the ansible.cfg exists when trying to run a playbook.
A word of warning for this though, we install the plugin-provided roles to a custom directory then set roles_path in /root/.ansible.cfg on the appliance.
If we put a user's ansible.cfg in front of ours in that priority list then those roles will be inaccessible for that playbook run. This behavior might be worth documenting.
This issue was originally raised in a comment here ManageIQ/manageiq#19079 (comment)
The new create_service_provision_request
method is called directly from $evm
, however the 2 'sibling' methods create_automation_request
and create_provision_request
are called via $evm.execute
. This is inconsistent and potentially confusing.
Suggest we move create_service_provision_request
into MiqAeServiceMethods
for consistency.
Original BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1811925
Would like to provision a VM through the Vmware Provider with the " Reserve all guest memory (All locked) " box checked inside VMware. NOTE: "memory_reserve_expand" set to true is not working.
Steps to Reproduce:
See BZ for more details.
Based on work from #415 Guard against hitting const_missing from within const_missing
See #415 (comment)
Hi,
I was checking jansa with expression methods as described in the Cloudforms / ManageIQ book [1] with user input and couldn't get it to work. It looks like variable resolution is no longer working for expression methods. Or maybe the syntax changed.
Affected versions (or what I tested):
jansa-2.20201027185742_b8d5deb
jansa-3.20210118183104_77bdc30
kasparov-1.20210203001902_15acbea
Steps to reproduce:
Follow the chapter "Expression methods" in the addendum [2]
Expected output:
Expression Method should deliver a result based on variables.
Actual result:
<None>
Additional info:
It is pretty interesting to compare the logging output in automation.log in ivanchuk and in jansa of an expression method:
Ivanchuck:
INFO -- : Invoking [expression] method [/Expression/General/my_expression_check] with inputs [{"arg1"=>"value1", "arg2"=>"foo", "arg3"=>"bar"}]
Jansa/Kasparov:
INFO -- : Invoking [expression] method [/Expression/General/my_expression_check] with inputs [{"arg1"=>"${/#dialog_my_field1}", "arg2"=>"${/#dialog_my_field2}", "arg3"=>"${/#dialog_my_field3}"}]
ERROR -- : Expression method ends
If that's important I tested expression method with tags.
Kind Regards,
phospi
gitter
: https://gitter.im/ManageIQ/manageiq?at=606eb27497cf5067464c362dA declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.