satelliteqe/airgun

AirGun is a Python library that is built over Widgetastic and navmazing to make Satellite 6 UI testing easier.

airgun's Issues

Add test coverage for user group entity

Add necessary entity and views for User Group functionality(Administer->User Group):

• All entities page
• Create/Edit entity page
• User Group Details - User Group tab
• User Group Details - Roles tab
  

Check Tab patternfly widget and check if it is applicable for us

Add/Expand Organization entity with CRUD logic

We already have some code for Organization entity (like working with current session context), but now we need to test specifically that entity, so we need to add test coverage for:

That entity has other structure in comparison with ones that we already implemented, so it will require time for investigation to find best approach

At least, edit procedure is a part of create procedure and amount of entities involved is tramendous

Resolve all #fixme and # TODO tags

Create coverage for Repository entity

Views: Repository entity
Location:
Content -> Products -> (product) -> Repositories

Views:

• Product Details - Repositories tab
• Create Repository
• Repository Details
• Repository Details - Packages
• Repository Details - Package Groups

Add smart variable entity

Configure -> Smart Variables

• All entities page
  
• Create/Update view from default page
  
• Create/Update view from Puppet Classes page
  

Important notes:

• At least one valid Puppet Class should be present in the system prior any possible operation with Smart Variables permitted
• Generic detail page should be divided into sub-views as there are many test cases that like to operate only with matchers for example

Add activation key entity with all helpers needed for tier3

Docker does not support '[]' in container names

This is mainly a problem when parametrized ui_airgun-tests are run on selenium-docker containers.
Seen for tests/foreman/ui_airgun/test_operatingsystem.py::test_positive_create[0].

Error was:

self = <docker.client.Client object at 0x7f7eabf59160>, response = <Response [500]>, explanation = None

    def _raise_for_status(self, response, explanation=None):
        """Raises stored :class:`APIError`, if one occurred."""
        try:
            response.raise_for_status()
        except requests.exceptions.HTTPError as e:
            if e.response.status_code == 404:
                raise errors.NotFound(e, response, explanation=explanation)
>           raise errors.APIError(e, response, explanation=explanation)
E           docker.errors.APIError: 500 Server Error: Internal Server Error ("b'{"message":"Invalid container name (test_positive_create[0]_5oo), only [a-zA-Z0-9][a-zA-Z0-9_.-] are allowed"}'")

../../.local/lib/python3.6/site-packages/docker/client.py:174: APIError

Force close the session if any exception raised during session opening

Issue by abalakh
Monday Feb 19, 2018 at 12:35 GMT
Originally opened as abalakh/airgun#24

---

Due to context manager nature, __exit__() catches exceptions in CM body, but not in CM's __enter__() section. This means in case any exception happens in __enter__() (e.g. host is unreachable) - session won't be closed and browser will remain opened after test run.
The easiest way to solve this is probably to use try: except: in __enter__() method which will call __exit__() in case of exception.

temporary user created for each test should be cleaned on tierdown

Issue by jhutar
Friday Feb 16, 2018 at 11:45 GMT
Originally opened as abalakh/airgun#20

---

Every test creates its own user - should that one be deleted on tier down?
Correct. Totally forgot about that.

In RTD docs, API -> Entities and API -> Views containes only class names, not a docstrings

Check:

https://airgun.readthedocs.io/en/latest/api/airgun.entities.html
https://airgun.readthedocs.io/en/latest/api/airgun.views.html

and compare to e.g.:

https://airgun.readthedocs.io/en/latest/api/airgun.browser.html

Create test coverage for partition table entity

Add necessary code for views and entities to cover:


Cover all widgets in Template, Locations and Organizations tabs. Verify that Help tab is not empty

ACE editor

is already has own custom widget implemented, so please reuse it

Look for tier2 tests in Robottelo and move all of them into new airgun folder using new framework

LCE seclector issue

Here is issue that I faced and cannot resolve in any way that I tried. Already spent too much time, so fresh look is needed.
Seems as architectural blocker.
Quick reproducer:

ak_name = gen_string('alpha')
    non_searchable_ak_name = gen_string('alpha')
    org = entities.Organization().create()
    envs_list = ['STAGING', 'DEV', 'IT', 'UAT', 'PROD']
    for name in envs_list:
        entities.LifecycleEnvironment(name=name, organization=org).create()
    env_name = random.choice(envs_list)
    cv = entities.ContentView(organization=org).create()
    cv.publish()
    promote(
        cv.read().version[0],
        entities.LifecycleEnvironment(name=env_name).search()[0].id
    )
    with session:
        session.organization.select(org_name=org.name)
        for name in [ak_name, non_searchable_ak_name]:
            session.activationkey.create({
                'name': name,
                'lce': {env_name: True},
                'content_view': cv.name
            })
            assert session.activationkey.search(name) == name

so you ends up in:

Here is way of my thoughts, but it is obviously wrong:

class LCESelectorGroup(Widget):
    ROOT = ".//div[@path-selector='environments']"
    ITEMS = "./div"

    def get_selector(self, criteria):
        """Return all lce selectors from the group"""
        # return [
        #     LCESelector(selector.locator)
        #     for selector
        #     in self.browser.elements(self.ITEMS)
        # ]
        # or
        # for selector in self.browser.elements(self.ITEMS):
        #   if criteria in selector.get_attribute('innerHTML'):
        #       return LCESelector(selector.locator)

    def fill(self, value):
        checkbox_name = list(value.keys())[0]
        selector = self.get_selector(checkbox_name)
        return selector.fill(value)

or through

    ROOT = ".//div[@path-selector='environments']"
    item = LCESelector("./div[contains(., '{}')]")

1. You cannot work with LCESelector or any widget outside of initialization as you finish up with WidgetDescriptor. Anyway it will not help a lot as implementation will look really bad
2. You cannot initialize widget on view level as you don't know name of LCE in advance
3. Working with names anyway seems as not the best option, but till that moment seems as only one possible. If we go with names it will be impossible to select 'Library' checkbox in third row for example, but if not names, there are nothing unique at all. Indexes are obviously not acceptable idea.
4. It is not nested view or view at all. Basically, I don't like idea of lce selectors group in widget format too, but we cannot modify LCESelector in any way as it should not have any connection to group or parent frame as it is wrong approach from architecture point of view

Create custom widgets specific to Satellite

List of widgets that I am already classified:

• Classical select list in case we don't have it:
  
• Select field with search:
  
• Select field with inherit
  
• Radio group
  
• Name-Value parameter
  
• Matcher
  
• ace editor
  
• calendar
  
• Maybe paginator

Submit airgun to PyPI

Implement list/remove/add view

Part of #27

Provide implementation that will cover all possible instances of the view in one class. For example:

or

Preliminary idea:

class Filter(Widget):
    filter = TextInput(locator="//input[contains(@placeholder, 'Filter')]")
    search_button = Text("@ng-click='table.search(table.searchTerm)']")

    def fill(self, value):
        self.filter.fill(value)
        if search_button.exists?
            search_button.click

    def read(self):
        return self.filter.read()

class AddRemoveResources(View):
    @View.nested
    class assigned(Tab):
        TAB_NAME = 'List/Remove'
        list_item = Checkbox()
        remove_item = Text('Remove Selected')
        content = Table()
        filter = Filter()

        def fill(self, value):
            self.filter.fill(value)
            self.list_item.fill(True)
            self.remove_item.click()
  
        def read(self):
            return self.content.read()

    @View.nested
    class vacant(Tab):
        TAB_NAME = 'Add'
        list_item = Checkbox('Add Selected')
        add_item = Text()
        content = Table()
        filter = Filter()

        def fill(self, value):
            self.filter.fill(value)
            self.list_item.fill(True)
            self.add_item.click()
  
        def read(self):
            return self.content.read()

Join two lists returned
It is better to have loop not on widgets level in case we need to add/remove multiple items, but can be time consuming operation due 'fill/read` architecture approach

Implement delete entity mixin

Similar to robottelo.ui.Base.delete we should create a helper method which will handle entity deletion and ensure deleted entity can't be found using search.
Couple of improvements which would be nice to have (thanks @oshtaier for pointing them):

• cover the case when deleted entity was the last one available, so searchbar is not present anymore on entity list page
• delete should return True/False whether deletion succeeded so we can easily assert deletion in tests like:

assert session.architecture.delete('foo')

so we could mark the test as failed (not errored) in case element was found.

Update airgun according to widgetastic guidelines

Recently widgetastic guidelines have been introduced:
https://github.com/RedHatQE/widgetastic.core#widgetastic-usage-guidelines

We should take a closer look to them and update all inconsistent places in airgun accordingly.

Automate one real case of navigating to sibling using shortest path

Current NavigateSteps are pretty simplistic - they all are top-level with little to no preconditions required, and we still haven't tried navmazing in use case which will actually save us time in comparison with robottelo's navigator: navigating to sibling or finding the shortest path.

Quick example:
Assume you have product1 and repo1 inside of it, repo1 page is open, and you want to create repo2 inside the same product.
Full path would be: Navigate to products -> open product1 -> navigate to repos tab -> hit 'Create' button.
Optimal path is (since we're inside product1 repos already): click 'product1 repos' link -> hit 'Create' button.

Refactor Search widget

Issue by abalakh
Monday Feb 19, 2018 at 12:34 GMT
Originally opened as abalakh/airgun#23

---

Current implementation doesn't respect layers isolation - read() returns not value of searchbar, but value of results table, which is outside of widget and should be handled on view level. Such implementation will also cause issues when automating some uncommon entities where multiple fields from results table should be taken into consideration.
Ideally widget's fill and read should only work with searchbar, and view should contain separate search() method which will return results from table.
We can also think of some SearchableMixin for views not to have to implement search for every view, since most of views will have very similar implementation and only locator will differ in most cases.

Add test coverage for Domain entity

Add views and entity for Domain:

So correspondingly cover Domain, Parameters, Organization and Location tabs
Add create, edit, read and delete actions to entity
Refer to already implemented code for proper approach and issues #58 or #57 to get more idea on what things should be covered for each specific tab

Keeping the read/fill interface

Issue by mfalesni
Thursday Feb 08, 2018 at 12:34 GMT
Originally opened as abalakh/airgun#11

---

I noticed that one of your widgets https://github.com/abalakh/airgun/blob/master/airgun/widgets.py#L51 does not conform to one of the concepts WT is built on.

Whatever is returned by read() must be acceptable by fill(). So essentially, x.fill(x.read()) should work at any time.

Add proper teardown methodology

Add activation key entity

Lazy entities init

Issue by abalakh
Monday Feb 19, 2018 at 12:35 GMT
Originally opened as abalakh/airgun#25

---

This feature was requested by @omaciel during PoC demo.
Entities initialization do not have any logic and do not perform any calculations, that's why we decided to init all of them inside session - to have nice shortcut and ability to call any entity's helper without the need to explicitly import them in tests.
However, since majority of tests will only use just few entities and definitely not all of them - we should probably think of lazy init.

Few ideas off the top of my head:

1. (preferable) Use cached_property for entities initialization.
2. Write our own implementation of simple cache, probably on top of regular properties
3. (last resort) Write custom __getattr__() which will init entity. Definitely a bad practice and rather a hack.

Add test coverage for Product entity

Location:
Content -> Products

Views:

• Products list
• Create Product
• Repo Discovery
• Product Details - Details tab
• Product Details - Tasks - (task)

Details:

• All products page:
  

Add searchable mixin, 'Create Product' button widget, SelectActionList widget and table widget

• Create Product page:
  

Add:
Name input widget, Label input widget, GPG key Select widget, Sync Plan and Description

• Edit Product page:
  Add SelectActionList widget
• Details Tab
  

Add EditableEntry and ReadOnlyEntry widgets for same elements covered for create entity page

• Repositories Tab
  

Probably, at that moment we can leave a stub here as it is completely other entity to be connected here, but if you feel able to start proper implementation - it will be appreciated.

---

Product Entity

Create basic actions like 'create', 'update', 'delete', 'read' and etc
Create all necessary navigation between entities

---

We can start with very simple tests that check that all framework elements work properly.
Then we can look on the existing test coverage in Robottelo and especially on tests from tier2 layer:
https://github.com/SatelliteQE/robottelo/blob/master/tests/foreman/ui/test_product.py

E.g.:
test_positive_create_in_different_orgs
https://github.com/SatelliteQE/robottelo/blob/master/tests/foreman/ui/test_product.py#L64
test_positive_update_to_original_name
https://github.com/SatelliteQE/robottelo/blob/master/tests/foreman/ui/test_product.py#L167

Add test coverage for User entity

First, it is necessary to describe all views that exists for User entity:

• All Users View
  

Basically, it contains 'Users' title, search widget, 'Create User' button and table with content

• Create New User View
  • User Tab

Add all input and select list widgets here

• Email Preferences Tab

Add one checkbox widget

• Locations Tab

Add one MultiSelect and one select list widget

• Organizations Tab

Add one MultiSelect and one select list widget

• Roles Tab
  

Add one checkbox and one MultiSelect widget

• Edit User View

Re-use Create New User View as it is pretty the same

---

User Entity

Create basic actions like 'create', 'update', 'delete', 'read' and etc
Create all necessary navigation between entities

---

Test coverage

We can start with very simple tests that check that all framework elements work properly.
Then we can look on the existing test coverage in Robottelo and especially on tests from tier2 layer:
https://github.com/SatelliteQE/robottelo/blob/master/tests/foreman/ui/test_user.py

E.g.:
test_positive_create_with_multiple_roles
https://github.com/SatelliteQE/robottelo/blob/master/tests/foreman/ui/test_user.py#L263
test_positive_create_with_all_roles
https://github.com/SatelliteQE/robottelo/blob/master/tests/foreman/ui/test_user.py#L288
test_positive_create_with_multiple_orgs
https://github.com/SatelliteQE/robottelo/blob/master/tests/foreman/ui/test_user.py#L329
...

Update widgets and views according to satellite 6.4 changes

As per @omaciel request, we should develop against sat6.4. At least navigation menu differs there, we should update it (and all the rest discovered places).

Add coverage for puppet environments

• All entities page
  
• Create/Edit navigation
• Details - Environment
• Details - Locations
• Details - Organizations
• Import(Changed environment) page
  

references to widgetastic and navmazing in documentation does not work

There is this in docs/config.py and I was unable to make the documentation build work without it:

nitpick_ignore = [
    [...]
    ('py:class', 'widgetastic.browser.Browser'),
    [...]
intersphinx_mapping = {
    'python': ('http://docs.python.org/3.6', None),
    # 'widgetastic':
    #   ('http://widgetasticcore.readthedocs.io/en/latest/', None),
    # 'navmazing': ('http://navmazing.readthedocs.io/en/latest/', None),

IMO these two mappings do not work because RedHatQE/widgetastic.core#86 and RedHatQE/navmazing#18. I hope that once these are resolved, we can uncomment above and remove lots of items from nitpick_ignore list.

This is a follow up on this discussion: #22 (comment) and on #22 (comment)

Make parametrized tests xdist-compatible

Issue by abalakh
Monday Feb 19, 2018 at 12:34 GMT
Originally opened as abalakh/airgun#22

---

(robottelo issue)
Currently test runs with parametrized tests using xdist are failing with msgs like:

collecting 0 items / 3 errors
==================================== ERRORS ====================================
_____________________________ ERROR collecting gw1 _____________________________
Different tests were collected between gw0 and gw1. The difference is:

This is due to random data generated inside parametrize. We should probably use the same hash seed across different workers. More info here: pytest-dev/pytest#1075

Implement proper am_i_here() for Edit*EntityName* NavigateStep

Currently NavigateSteps for navigating to entity edit page are only checking whether entity details view is open.
E.g. if you have activation keys foo1, foo2, have foo1 details page opened and want to edit foo2 - navigator will think that you're on foo2 details page already, as it doesn't check specific entity name but just ensures any AK details page is opened.
The task will probably require adding Breadcrumbs patternfly widget.

Add test coverage for LCE entity

class internal references in documentation does not work

There is this in docs/config.py and I was unable to make the documentation build work without it:

nitpick_ignore = [
    [...]
    ('py:meth', 'current_org')
    [...]

IMO it is just some omission somewhere. Found https://stackoverflow.com/questions/21289806/link-to-class-method-in-python-docstring, but that does not seem to help.

This is a follow up on this discussion: #22 (comment)

Add test coverage for content credentials entity

Add test coverage for Role managment

Expand logic for search widget for situtations when last entity is removed from the page

Check that button doesn't exist and class of the page is 'blank...' or whatever then [] value otherwise raise the exception

Add compute profile entity

Implement test coverage for templates

Identify all Views that must be added

We need to identify all Views that must be added to support AirGun usage with latest Foreman/Katello upstream.

This will also allow us to clearly identify what tasks are available for future collaborators.

Refactor settings; introduce settings validations

Current settings implementation is basically a stub for PoC, they should definitely evolve into something better.
Two main issues - no validations at all and no support of nested sections due to configparser limitations (webdrivercapabilities section is supposed to be a part of selenium section).
I was thinking about using configobj instead of configparser, but absence of new commits for more than year encourages to look for a better alternative.

Create "landing page" for new contributors

As a potential new contributor (upstream or downstream), I want to know how to get started contributing with this project and what available tasks are ready for me

Add basic compute resource entity

Update README.rst

README.rst should include proper description, entities/views/widgets/session explanation, install guide, configuring section (both by configure() call and settings.ini) and some example of usage.

Create generic widget for actions list (button with dropdown)

There's widgetastic_patternfly's Dropdown, our SelectActionList, TableActionList introduced in #67 for similar purpose which still do not cover 100% of Satellite actions widget (button with dropdown). We should implement 1 generic instead.

what would be best way how to catch errors like https://projects.theforeman.org/issues/23035 ?

Can we assert that page does not contain string {{ ... }} automatically?

See https://projects.theforeman.org/issues/23035 for example please.

test passing with Chrome, but failing with Firefox

Issue by jhutar
Tuesday Feb 06, 2018 at 12:53 GMT
Originally opened as abalakh/airgun#7

---

Following README.rst, when you run pytest tests/foreman/ui_airgun/test_architecture.py with this setup it works:

$ grep ^webdriver robottelo.properties
webdriver=chrome
webdriver_binary=venv/bin/chromedriver
$ pytest tests/foreman/ui_airgun/test_architecture.py 
=== test session starts ===
platform linux -- Python 3.6.4, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
shared_function enabled - OFF - scope:  - storage: file
rootdir: /home/pok/Checkouts/robottelo, inifile:
plugins: wait-for-1.0.9, services-1.2.1, mock-1.6.3
collected 2 items                                                                                                                                                                                                                            
2018-02-06 13:23:21 - conftest - DEBUG - BZ deselect is disabled in settings


tests/foreman/ui_airgun/test_architecture.py ..                                                                                                                                                                                        [100%]

=== 2 passed in 84.81 seconds ===

But when I run the same on Firefox (firefox-58.0-4.fc27.x86_64, geckodriver-v0.19.1-linux64.tar.gz), it fails:

$ grep ^webdriver robottelo.properties
webdriver=firefox
webdriver_binary=venv/bin/geckodriver

Error is:

self = <airgun.browser.AirgunBrowser object at 0x7f90a97e2d68>, locator = Locator(by='xpath', locator="//a[contains(@href, '/architectures/new')]"), args = (), kwargs = {}, vcheck = None, elements = []

    def element(self, locator, *args, **kwargs):
        """Returns one :py:class:`selenium.webdriver.remote.webelement.WebElement`
    
            See: :py:meth:`elements`
    
            Returns:
                :py:class:`selenium.webdriver.remote.webelement.WebElement`
    
            Raises:
                :py:class:`selenium.common.exceptions.NoSuchElementException`
            """
        try:
            vcheck = self._locator_force_visibility_check(locator)
            if vcheck is not None:
                kwargs['check_visibility'] = vcheck
            elements = self.elements(locator, *args, **kwargs)
            if len(elements) > 1:
                visible_elements = [e for e in elements if self.is_displayed(e)]
                if visible_elements:
                    return visible_elements[0]
                else:
                    return elements[0]
            else:
>               return elements[0]
E               IndexError: list index out of range

venv/lib/python3.6/site-packages/widgetastic/browser.py:337: IndexError
[...]

Login: Entity or NavigateStep?

Currently we do have Login as entity with methods like login and logout. Session calls those methods in __enter__() and __exit__() accordingly. That's basically the approach robottelo is using.

CFME and navmazing use different one though - Login and Logout are NavigateSteps, so navigator handles logging in and logging out if necessary.

Question is - do cfme's way introduce some valuable benefits or that's basically just another equal approach? Should we use it or stay with what we have?

I was thinking about that and couldn't highlight any serious benefits of either approach. The only thing is cfme's way will allow us to ensure we're logged in before proceeding with navigation to desired page. On the other hand it introduces extra prerequisite for top-level navigate steps. And since Satellite6 doesn't have any actions which could lead user to appear accidentally logged out of application there's not much value in it, more than that - if some action will close user's session i'd like to catch such bug and not to hide it with navmazing.

@oshtaier @mfalesni @jhutar let's discuss.

Add documentation

This task consists of following subtasks:

• Create docs/ dir, install sphinx there
• Configure sphinx' conf.py
• Add rst files for all the modules
• Introduce travis check of documentation errors
• Publish docs to readthedocs.org

Add coverage for host entity