HtmlUnitDriver unable to findElement when URL has queryStrings

[seleniumhq]

HtmlUnitDriver unable to findElement when URL has queryStrings

[asashour]

I tried with master branch, and it works fine.

[asashour]

Can you try with htmlunit-driver 2.21, without referencing httpclientor selenium-java?

[VishJP]

Unable to access url using HtmlUnitDriver

@Test
    public void testing() {

        DesiredCapabilities capabilities = DesiredCapabilities.htmlUnit();
        capabilities.setJavascriptEnabled(true);
        HtmlUnitDriver driver = new HtmlUnitDriver(true);
        driver.get("https://www.google.com");
        WebElement element = driver.findElement(By.id("lst-ib"));
        element.sendKeys("Selenium");
        System.out.println("Page title is: " + driver.getTitle());
        driver.quit();

    }

Maven Dependencies:

<dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>htmlunit-driver</artifactId>
            <version>2.21</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-api</artifactId>
            <version>2.53.0</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-support</artifactId>
            <version>2.53.0</version>
        </dependency>

Error

java.lang.NoClassDefFoundError: org/apache/http/conn/ssl/NoopHostnameVerifier
    at com.gargoylesoftware.htmlunit.httpclient.HtmlUnitSSLConnectionSocketFactory.buildSSLSocketFactory(HtmlUnitSSLConnectionSocketFactory.java:111)
    at com.gargoylesoftware.htmlunit.HttpWebConnection.configureHttpsScheme(HttpWebConnection.java:607)
    at com.gargoylesoftware.htmlunit.HttpWebConnection.createHttpClient(HttpWebConnection.java:542)
    at com.gargoylesoftware.htmlunit.HttpWebConnection.getHttpClientBuilder(HttpWebConnection.java:504)
    at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:163)
    at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1357)
    at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1274)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:382)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:304)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:451)
    at org.openqa.selenium.htmlunit.HtmlUnitDriver.get(HtmlUnitDriver.java:528)
    at org.openqa.selenium.htmlunit.HtmlUnitDriver.get(HtmlUnitDriver.java:517)
    at testing.htmlunitdriver.TestingUnitDriver.testing(TestingUnitDriver.java:17)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:86)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:646)
    at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:823)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1131)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
    at org.testng.TestRunner.privateRun(TestRunner.java:778)
    at org.testng.TestRunner.run(TestRunner.java:632)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:366)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:361)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:319)
    at org.testng.SuiteRunner.run(SuiteRunner.java:268)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1225)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1150)
    at org.testng.TestNG.runSuites(TestNG.java:1075)
    at org.testng.TestNG.run(TestNG.java:1047)
    at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:126)
    at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:137)
    at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:58)
Caused by: java.lang.ClassNotFoundException: org.apache.http.conn.ssl.NoopHostnameVerifier
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 38 more

[asashour]

I retested for the second time.

You are definitely mixing incorrect versions (remove everything else from your `pom.xml').

The below:

    @Test
    public void testing() {
        log(org.openqa.selenium.htmlunit.HtmlUnitDriver.class);
        log(com.gargoylesoftware.htmlunit.WebClient.class);
        log(org.apache.http.auth.AuthState.class);
        log(org.apache.http.HttpHost.class);
    }

    private void log(Class<?> klass) {
        System.out.println(klass.getProtectionDomain().getCodeSource().getLocation());
    }

Should print:

htmlunit-driver-2.21.jar
htmlunit-2.21.jar
httpclient-4.5.2.jar
httpcore-4.4.4.jar

[jayantsarda]

Hi thanks for looking into it. It turned out that its an ecliple maven plugin issue. When I tried running from command line the same test works fine.

[VishJP]

Hey Guys, Thanks for your prompt responses and I can execute with Maven (There is some conflict between some other dependency since i had too many in pom file. Finally i removed some of them and kept only required one and it works for me).

One more drawback in both IDE and HtmlUnitDriver, it is not handling javascript search and filter pages.

Example: I want to search specific data in my project and print the results (Which is return by javascript), it didnt actually filter the records and when get count, it shows total list.

[asashour]

If you have a new question, please open a new issue with minimal test case to check against.