open-tool/ultron

Easiest framework to develop Android UI tests

Ultron

Ultron is the simplest framework to develop Android UI tests. It provides simplicity, stability, and maintainability to your tests. It's constructed upon the Espresso, UI Automator and Compose UI testing frameworks. Ultron introduces a range of remarkable new features. Furthermore, Ultron puts you in complete control of your tests!

Additionally, you don't need to learn any new classes or special syntax. All magic actions and assertions are provided from crunch. Ultron can be easially customised and extended. Wish you exclusively stable tests!

What are the benefits of using the framework?

• Exceptional support for Compose
• Out-of-the-box generation of Allure report
• A straightforward and expressive syntax
• Ensured Stability for all actions and assertions
• Complete control over every action and assertion
• Incredible interaction with RecyclerView and Compose lists.
• An Architectural approach to developing UI tests
• An incredible mechanism for setups and teardowns (You can even set up preconditions for a single test within a test class, without affecting the others)
• The ability to effortlessly extend the framework with your own operations
• Accelerated UI Automator operations
• Ability to monitor each stage of operation execution with Listeners
• Custom operation assertions

---

Wiki

The framework offers an excellent wiki that addresses the majority of significant usage scenarios.

A few words about syntax

The standard syntax provided by Google is intricate and not intuitive. This is especially evident when dealing with RecyclerView interactions.

Let's explore some examples:

1. Simple compose operation (refer to the wiki here)

Compose framework

composeTestRule.onNode(hasTestTag("Continue")).performClick()
composeTestRule.onNodeWithText("Welcome").assertIsDisplayed()

Ultron

hasTestTag("Continue").click()
hasText("Welcome").assertIsDisplayed()

2. Compose list operation (refer to the wiki here)

Compose framework

val itemMatcher = hasText(contact.name)
composeRule
    .onNodeWithTag(contactsListTestTag)
    .performScrollToNode(itemMatcher)
    .onChildren()
    .filterToOne(itemMatcher)
    .assertTextContains(contact.name)

Ultron

composeList(hasTestTag(contactsListTestTag))
    .item(hasText(contact.name))
    .assertTextContains(contact.name)

3. Simple Espresso assertion and action.

Espresso

onView(withId(R.id.send_button)).check(isDisplayed()).perform(click())

Ultron

withId(R.id.send_button).isDisplayed().click()

This presents a cleaner approach. Ultron's operation names mirror Espresso's, while also providing additional operations.

Refer to the wiki for further details.

4. Action on RecyclerView list item

Espresso

onView(withId(R.id.recycler_friends))
    .perform(
        RecyclerViewActions
            .actionOnItem<RecyclerView.ViewHolder>(
                hasDescendant(withText("Janice")),
                click()
            )
        )

Ultron

withRecyclerView(R.id.recycler_friends)
    .item(hasDescendant(withText("Janice")))
    .click()

Explore the wiki to unveil Ultron's magic with RecyclerView interactions.

5. Espresso WebView operations

Espresso

onWebView()
    .withElement(findElement(Locator.ID, "text_input"))
    .perform(webKeys(newTitle))
    .withElement(findElement(Locator.ID, "button1"))
    .perform(webClick())
    .withElement(findElement(Locator.ID, "title"))
    .check(webMatches(getText(), containsString(newTitle)))

Ultron

id("text_input").webKeys(newTitle)
id("button1").webClick()
id("title").hasText(newTitle)

Refer to the wiki for more details.

6. UI Automator operations

UI Automator

val device = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation())
device
    .findObject(By.res("com.atiurin.sampleapp:id", "button1"))
    .click()

Ultron

byResId(R.id.button1).click() 

Refer to the wiki

---

Acquiring the result of any operation as Boolean value

val isButtonDisplayed = withId(R.id.button).isSuccess { isDisplayed() }
if (isButtonDisplayed) {
    //do some reasonable actions
}

---

Why are all Ultron actions and assertions more stable?

The framework captures a list of specified exceptions and attempts to repeat the operation during a timeout period (default is 5 seconds). Of course, you have the ability to customize the list of handled exceptions. You can also set a custom timeout for any operation.

withId(R.id.result).withTimeout(10_000).hasText("Passed")

---

3 steps to develop a test using Ultron

We advocate for a proper test framework architecture, division of responsibilities between layers, and other best practices. Therefore, when using Ultron, we recommend the following approach:

1. Create a Page Object and specify screen UI elements as Matcher<View> objects.

object ChatPage : Page<ChatPage>() {
    private val messagesList = withId(R.id.messages_list)
    private val clearHistoryBtn = withText("Clear history")
    private val inputMessageText = withId(R.id.message_input_text)
    private val sendMessageBtn = withId(R.id.send_button)
}

It's recommended to make all Page Objects as object and descendants of Page class. This allows for the utilization of convenient Kotlin features. It also helps you to keep Page Objects stateless.

2. Describe user step methods in Page Object.

object ChatPage : Page<ChatPage>() {
    fun sendMessage(text: String) = apply {
        inputMessageText.typeText(text)
        sendMessageBtn.click()
        getMessageListItem(text).text
             .isDisplayed()
             .hasText(text)
    }

    fun clearHistory() = apply {
        openContextualActionModeOverflowMenu()
        clearHistoryBtn.click()
    }
}

Refer to the full code sample ChatPage.class

3. Call user steps in test

    @Test
    fun friendsItemCheck(){
        FriendsListPage {
            assertName("Janice")
            assertStatus("Janice","Oh. My. God")
        }
    }
    @Test
    fun sendMessage(){
        FriendsListPage.openChat("Janice")
        ChatPage {
            clearHistory()
            sendMessage("test message")
        }
    }

Refer to the full code sample DemoEspressoTest.class

In essence, your project's architecture will look like this:

Allure report

Ultron has built in support to generate artifacts for Allure reports. Just apply the recommended configuration and set testIntrumentationRunner.

For the complete guide, refer to the wiki

@BeforeClass @JvmStatic
fun setConfig() {
    UltronConfig.applyRecommended()
    UltronAllureConfig.applyRecommended()
}

for Compose add 4 lines more

@BeforeClass @JvmStatic
fun setConfig() {
    ...
    UltronComposeConfig.applyRecommended() 
    UltronComposeConfig.addListener(ScreenshotAttachListener())
    UltronComposeConfig.addListener(WindowHierarchyAttachListener())
    UltronComposeConfig.addListener(DetailedOperationAllureListener())
}

Add Ultron to your project

Gradle

repositories {
    mavenCentral()
}

dependencies {
    androidTestImplementation 'com.atiurin:ultron:<latest_version>'
    androidTestImplementation 'com.atiurin:ultron-allure:<latest_version>'
    androidTestImplementation 'com.atiurin:ultron-compose:<latest_version>'
}

Please, read gradle dependencies management wiki page.

AndroidX

It is required to use AndroidX libraries. You can get some problems with Android Support ones.

Roadmap

• #50 Meta information for UI elements
• #33 Screenshot testign ?

ultron's Issues

Allow to limit number of items in UltronRecyclerView.getItemsAdapterPositionList

Screen recorder

Swipe action improvement

Ability to setup messages for assertions

Add new function that will allow to setup message for assertion, so when the assertion fails the message will be displayed in logs instead of default one, e.g.
matcher.withUseUnmergedTree(true).withMessage("Element should be enabled").assertIsEnabled()

It would be helpful to have such method not only for compose elements, but also for xml, uiobjects etc

Add 'isListened' boolean parameter for 'isSuccess' function

As for now, when 'isSuccess' returns false, ScreenshotListener will make a screenshot. It's not required in some cases so, if possible, it would be useful to have such switcher and control either should the invoked function be listened or not

Fix: compose list - assert item not displayed

Screenshot testing ?

Add hasTextColor matcher

Missing matcher to check text color

Add interceptors

Maybe cool to add interceptors for check request/response after actions?

Ability to interact with DatePicker component of Jetpack Compose

Custom name for matcher object for Espresso, UiAutomator, Compose

Use in exception message

Allure support

Easy plugged in allure support

move to mavenCentral

Jcenter will be closed soon

DumpHierarchy to allure report

Soft assertions

Extend click action api for espresso

1. Add methods:

- clickLeft()
- clickRight()
- clickTop()
- clickBottom()

2. Add click with duration: Long param

Нow can I specify the order in which tests are run?

How can I specify the order in which tests are run?

Example LoginTest.kt, Example1Test.kt, Example2Test.kt, ExitTest.kt

Thank you in advance)

Espresso Test

How to write a looping statement in Record Espresso Test using kotlin?

Update libs

Meta information for UI elements

Idling resources to ignore the endless async task for espresso

Hi there,

It's not an issue, it's a question if you already have the way in the framework how to ignore the endless async tasks.
I've not found a community where to ask it.

Is there a way how to say espresso to not wait for the specific endless async task from the AsyncTask.THREAD_POOL_EXECUTOR ?

Thanks for the advice.

Custom assertion on action result

Sometimes operation executed successfully. But the action wasn't performed. It's usually a minor bug in application.
It's planned to add custom lambda assertion to check the operation really was executed. In case of failure, operation will be repeated.

Logcat to allure report

Compose support

Research new api - https://developer.android.com/jetpack/compose/testing

Add support for junitparams framework

Parameterized tests not working with custom name

Code sample:

open class BaseTest {

    val activityTestRule = createUltronComposeRule<MainActivity>()

    val beforeEachTestRule = SetUpRule("Precondition before each test").add {
              ...
    }
    
         @get:Rule
    open val ruleSequence = RuleSequence(activityTestRule, beforeEachTestRule)

    companion object {

        @BeforeClass
        @JvmStatic
        fun configuration() {
            UltronConfig.applyRecommended()
            UltronAllureConfig.applyRecommended()
            UltronComposeConfig.applyRecommended()
            UltronComposeConfig.addListener(ScreenshotAttachListener())
            UltronComposeConfig.addListener(DetailedOperationAllureListener())
        }
    }
}

@RunWith(Parameterized::class)
SubclassTest(private val testValue: String) : BaseTest() {
   
companion object {

 @JvmStatic
 @Parameters(name = "{0}")
 fun testData() = listOf(
  "testValue1",
  "testValue2",
 )
 }

@Test fun myAwesomeTest() {
   ...
   }
}

Fix incorrect timeout in operation description

If use default timeout than in description it is shown as null

Update espresso to 3.4.0