Selenium2Library architecture changes

[marketsquare]

Selenium2Library architecture changes

[emanlove]

Looking into this now. One of the advantages of the old implementation was I was pretty easily able to overwrite the find functionality. This made it easy to add a implicit Wait for Angular call on every keyword that used a locator. I see that code has changed and so I am looking at how the AngularJS Library needs to change.

[aaltat]

All feedback is welcomed, but I have to admit that I did not think this option when we where doing the new architecture. It depends greatly what methods are overwritten and how Selenium2Library inheritance happens , but example keywords methods are saved in a self.keywyords instance attribute. And updating that attribute is done in the robotlibcore.py modules.

[emanlove]

What I was able to do was to override the _find_element method in the Selenium2Library. The new architecture this method is a little more hidden. Looking at the library instance

(Pdb) self._s2l
<Selenium2Library.Selenium2Library object at 0x7fb799bed610>
(Pdb) dir(self._s2l)
['ROBOT_LIBRARY_LISTENER', 'ROBOT_LIBRARY_SCOPE', 'ROBOT_LIBRARY_VERSION', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__format__', '__getattr__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_browser', '_browsers', '_cache', '_current_browser', '_find_keywords', '_get_arg_spec', '_get_keyword_tags_supported', '_get_members', '_get_members_from_instannce', '_implicit_wait_in_secs', '_run_on_failure_keyword', '_running_on_failure_routine', '_speed_in_secs', '_timeout_in_secs', 'add_cookie', 'add_location_strategy', 'alert_should_be_present', 'assign_id_to_element', 'capture_page_screenshot', 'checkbox_should_be_selected', 'checkbox_should_not_be_selected', 'choose_cancel_on_next_confirmation', 'choose_file', 'choose_ok_on_next_confirmation', 'clear_element_text', 'click_button', 'click_element', 'click_element_at_coordinates', 'click_image', 'click_link', 'close_all_browsers', 'close_browser', 'close_window', 'confirm_action', 'create_webdriver', 'current_frame_contains', 'current_frame_should_not_contain', 'delete_all_cookies', 'delete_cookie', 'dismiss_alert', 'double_click_element', 'drag_and_drop', 'drag_and_drop_by_offset', 'element_should_be_disabled', 'element_should_be_enabled', 'element_should_be_visible', 'element_should_contain', 'element_should_not_be_visible', 'element_should_not_contain', 'element_text_should_be', 'execute_async_javascript', 'execute_javascript', 'focus', 'frame_should_contain', 'get_alert_message', 'get_all_links', 'get_cookie_value', 'get_cookies', 'get_element_attribute', 'get_element_size', 'get_horizontal_position', 'get_keyword_arguments', 'get_keyword_documentation', 'get_keyword_names', 'get_keyword_tags', 'get_list_items', 'get_location', 'get_locations', 'get_matching_xpath_count', 'get_selected_list_label', 'get_selected_list_labels', 'get_selected_list_value', 'get_selected_list_values', 'get_selenium_implicit_wait', 'get_selenium_speed', 'get_selenium_timeout', 'get_source', 'get_table_cell', 'get_text', 'get_title', 'get_value', 'get_vertical_position', 'get_webelement', 'get_webelements', 'get_window_identifiers', 'get_window_names', 'get_window_position', 'get_window_size', 'get_window_titles', 'go_back', 'go_to', 'input_password', 'input_text', 'input_text_into_prompt', 'keywords', 'list_selection_should_be', 'list_should_have_no_selections', 'list_windows', 'location_should_be', 'location_should_contain', 'locator_should_match_x_times', 'log_location', 'log_source', 'log_title', 'maximize_browser_window', 'mouse_down', 'mouse_down_on_image', 'mouse_down_on_link', 'mouse_out', 'mouse_over', 'mouse_up', 'open_browser', 'open_context_menu', 'page_should_contain', 'page_should_contain_button', 'page_should_contain_checkbox', 'page_should_contain_element', 'page_should_contain_image', 'page_should_contain_link', 'page_should_contain_list', 'page_should_contain_radio_button', 'page_should_contain_textfield', 'page_should_not_contain', 'page_should_not_contain_button', 'page_should_not_contain_checkbox', 'page_should_not_contain_element', 'page_should_not_contain_image', 'page_should_not_contain_link', 'page_should_not_contain_list', 'page_should_not_contain_radio_button', 'page_should_not_contain_textfield', 'press_key', 'radio_button_should_be_set_to', 'radio_button_should_not_be_selected', 'register_browser', 'register_keyword_to_run_on_failure', 'reload_page', 'remove_location_strategy', 'run_keyword', 'screenshot_root_directory', 'select_all_from_list', 'select_checkbox', 'select_frame', 'select_from_list', 'select_from_list_by_index', 'select_from_list_by_label', 'select_from_list_by_value', 'select_radio_button', 'select_window', 'set_browser_implicit_wait', 'set_screenshot_directory', 'set_selenium_implicit_wait', 'set_selenium_speed', 'set_selenium_timeout', 'set_window_position', 'set_window_size', 'simulate', 'submit_form', 'switch_browser', 'table_cell_should_contain', 'table_column_should_contain', 'table_footer_should_contain', 'table_header_should_contain', 'table_row_should_contain', 'table_should_contain', 'textarea_should_contain', 'textarea_value_should_be', 'textfield_should_contain', 'textfield_value_should_be', 'title_should_be', 'unselect_checkbox', 'unselect_frame', 'unselect_from_list', 'unselect_from_list_by_index', 'unselect_from_list_by_label', 'unselect_from_list_by_value', 'wait_for_condition', 'wait_until_element_contains', 'wait_until_element_does_not_contain', 'wait_until_element_is_enabled', 'wait_until_element_is_not_visible', 'wait_until_element_is_visible', 'wait_until_page_contains', 'wait_until_page_contains_element', 'wait_until_page_does_not_contain', 'wait_until_page_does_not_contain_element', 'xpath_should_match_x_times']
(Pdb)

Should have some time to look at this this week and I will definitely keep in mind moving the library forward and not backwards nor sidewards. Sorry I have been absent on the development side.

[aaltat]

The new architecture is almost ready, the two things left are:

1. Modify ElementFinder to have access to context.
2. Move the methods with TODO from LibraryComponent[1] to ElementFinder class. Against the TODO, most likely, the wrapper is not going to be preserved.

But other than that code changes should be minimal.

[1] https://github.com/robotframework/Selenium2Library/blob/master/src/Selenium2Library/base.py
[2] https://github.com/robotframework/Selenium2Library/blob/master/src/Selenium2Library/locators/elementfinder.py

[emanlove]

Just trying to understand what I am seeing there... is the TODO saying "Move logic into elementfinder.ElementFinder" or move the code from elementfinder into LibraryComponent class?

Also I noticed that for someone trying to create a "parallel library" like this one there is no function for them to call to add locator strategies with the strategy as a provided python method. There is the add_locator_stategy is exposed but that is for keyword based strategies.

[aaltat]

1. TODO is trying to saying: move method and it's logic to elementfinder.ElementFinder. But after the TODO comments, there has been many other changes and it might be that some of the methods are not cross referenced anymore and they could return to keyword classes. At least element_find is heavily used from keyword classes and is moving to elementfinder.ElementFinder class.

2. I did not know this, could you share a link?

[emanlove]

I am mistaken with my second point about adding a locator strategy. I looked back at Selenium2Library v1.7.4 and the AngularJSLibrary and see the solutions are the same as well as the code for the new architecture. I can (and do) add locator strategies using that method. My problem lies with not yet overriding the _element_finder method where previously I added {{ prefix to default to (Angular) binding strategy.

So I am back to trying to figure out how to work with the DynamicLibrary class. Where I once before overrode the _element_finder method, this method is no longer exposed. ... I'm reading the Robot Framework documentation on Dynamic Libraries now ...

[emanlove]

[Thinking out loud and sharing where I am exploring]

I was looking into the _get_members method of robotlibcore.py to see what I can expose through that. First tried

(Pdb) [m for m in self._s2l._get_members('ElementKeywords')]
[('__add__', <slot wrapper '__add__' of 'str' objects>), ('__class__', <type 'type'>), ...]
(Pdb)

which I realized was just giving me members of the type str. Then tried one of the keywords[m for m in self._s2l._get_members(self._s2l.get_webelement())]. Note that here I am invoking the method when instead I want the method (which is 95% the opposite of what one usually wants). So now I am looking at

(Pdb) [m for m in self._s2l._get_members(self._s2l.get_webelement)]
[('__call__', <slot wrapper '__call__' of 'instancemethod' objects>), ('__class__', <type 'type'>), ('__cmp__', <slot wrapper '__cmp__' of 'instancemethod' objects>), ('__delattr__', <slot wrapper '__delattr__' of 'instancemethod' objects>), ('__doc__', 'instancemethod(function, instance, class)\n\nCreate an instance method object.'), ('__format__', <method '__format__' of 'object' objects>), ('__func__', <member '__func__' of 'instancemethod' objects>), ('__get__', <slot wrapper '__get__' of 'instancemethod' objects>), ('__getattribute__', <slot wrapper '__getattribute__' of 'instancemethod' objects>), ('__hash__', <slot wrapper '__hash__' of 'instancemethod' objects>), ('__init__', <slot wrapper '__init__' of 'object' objects>), ('__new__', <built-in method __new__ of type object at 0x903ca0>), ('__reduce__', <method '__reduce__' of 'object' objects>), ('__reduce_ex__', <method '__reduce_ex__' of 'object' objects>), ('__repr__', <slot wrapper '__repr__' of 'instancemethod' objects>), ('__self__', <member '__self__' of 'instancemethod' objects>), ('__setattr__', <slot wrapper '__setattr__' of 'instancemethod' objects>), ('__sizeof__', <method '__sizeof__' of 'object' objects>), ('__str__', <slot wrapper '__str__' of 'object' objects>), ('__subclasshook__', <built-in method __subclasshook__ of type object at 0x903ca0>), ('im_class', <member 'im_class' of 'instancemethod' objects>), ('im_func', <member 'im_func' of 'instancemethod' objects>), ('im_self', <member 'im_self' of 'instancemethod' objects>), ('robot_name', None), ('robot_tags', ())]
(Pdb)

Not entirely useful be this path may be productive ...

Another note for those interested in what I am doing previously the main Selenium2Library class was

class Selenium2Library(
    _LoggingKeywords,
    _RunOnFailureKeywords,
    _BrowserManagementKeywords,
    _ElementKeywords,
    _TableElementKeywords,
    _FormElementKeywords,
    _SelectElementKeywords,
    _JavaScriptKeywords,
    _CookieKeywords,
    _ScreenshotKeywords,
    _WaitingKeywords
):

where _element_finder() was a method of the _ElementsKeyword class thus exposing that function. Now the Selenium2Library class is

class Selenium2Library(DynamicCore):

and the underlying classes aren't exposed the same way. The library keywords are exposed but I prefer not to override each and every keyword that use a locator (along the lines of what Richard does with the Extended Selenium2 Library). Instead I prefer to simply override in one spot the find function. So I am looking to see how I can do the same in this new architecture ...

[aaltat]

How do you use Selenium2Library? Do you just inherit the class or do you dig up the library instance from Robot Framework?

[aaltat]

Ah, it but it's not important. I have an idea how to do overwrite the required methods. It's more complicated than before and I need to test it out with real code and by using real keyboard...

[emanlove]

I get the instance from Robot Framework. Here is the Library class, its __init__ and the _s2l property which contains the instance of the Selenium2Library.

class AngularJSLibrary:
    ROBOT_LIBRARY_SCOPE = 'GLOBAL'
    ROBOT_LIBRARY_VERSION = '0.0.5.dev1'
    def __init__(self, root_selector=None, implicit_angular_wait=30.0, ignore_implicit_angular_wait=False):
        self.ignore_implicit_angular_wait = ignore_implicit_angular_wait
    
        if not root_selector:
            self.root_selector = '[ng-app]'
        else:
            self.root_selector = root_selector
    
        # Override default locators to include binding {{ }}
        self._s2l._element_finder = ngElementFinder(self.root_selector, ignore_implicit_angular_wait)
    
        # Add Angular specific locator strategies
        self._s2l.add_location_strategy('ng-binding', self._find_by_binding, persist=True)
        self._s2l.add_location_strategy('binding', self._find_by_binding, persist=True)
        self._s2l.add_location_strategy('ng-model', self._find_by_model, persist=True)
        self._s2l.add_location_strategy('model', self._find_by_model, persist=True)
        self._s2l.add_location_strategy('ng-repeater', self._find_by_ng_repeater, persist=True)
        self._s2l.add_location_strategy('repeater', self._find_by_ng_repeater, persist=True)
        self.trackOutstandingTimeouts = True

    @property
    def _s2l(self):
        return BuiltIn().get_library_instance('Selenium2Library')

[aaltat]

Now the _element_finder is hidden in each library class, but there was a idea to move the _element_finder to ElementFinder class. In the end, it should look like the TableElementFinder in the TableElementKeyword class.

But to solve the problem in hand, then in theory you could get the keyword classes instances from the S2L, they should be in libraries attribute. Then from each library class intanse, one would need to check does it have intanse from the ElementFinder class. If yes, then one should overwrite the _element_finder method from the ElementFinder instance. That was my idea, but I haven't found time to test it (sun has been shining and I have been eating ice cream).

What do you think, is the access to the method in too hard place from your point of view or should we move the method somewhere where it can be accessed in more common way?

https://github.com/robotframework/Selenium2Library/blob/master/src/Selenium2Library/keywords/tableelement.py

[emanlove]

[Smiling] You like ice cream too?! Will have to remember that next time we meet up. I think I owe you and @pekkaklarck a double/triple scoop.

Will think about this later today. This does sound like a good solution.

[pekkaklarck]

How to extend S2L after architecture changes is a very important topic. If needed, I'm more than happy to enhance the PythonLibCore to make it easier.

[pekkaklarck]

@aaltat and I did two changes today to ease extending:

• There's now element_finder attribute in the library itself that can be overridden by extending classes. Notice that it's API isn't exactly the same as in earlier versions, though.
• PythonLibCore that the library uses internally got new add_library_components method that can be used to register new library components after making calling __init__ of the parent class.

We hope that these changes make it easier to extend the library in general. Hopefully also fixing the code that gets broken due to the architecture changes isn't too complicated. Old extending approaches have used private interfaces and being fully compatible with them is not possible.

Architecture changes ought to be now done except that we still need to fix registering custom locator strategies. After that we are very close to alpha 1 release.

[aaltat]

The custom locator change is now fixed and merged to the master. The API should be now backwards compatible. There should not be any architecture code changes and items pending before new (alpha/beta) release are related to renewing the release tools and other administrative tasks.

Please take a look and let's us know does the changes meet your needs.

[aaltat]

The alpha release of Selenium2Library/SeleniumLibrary is out. @emanlove have you encountered any problems in your side?

[emanlove]

AngularJSLibrary version 0.0.7 added support for SeleniumLibrary and dropped support for Selenium2Library.