Failing simulations are blocked instead of terminated with gatling 3.10.1

[nvervelle]

Failing simulations are blocked instead of terminated with gatling 3.10.1

[slandelle]

Please provide a reproducer.

The following simulation works as expected for me: req2 fails so req3 doesn't get executed).

import io.gatling.javaapi.core.*;
import io.gatling.javaapi.http.*;

import static io.gatling.javaapi.core.CoreDsl.*;
import static io.gatling.javaapi.http.HttpDsl.*;

public class ExitSimulation extends Simulation {

    HttpProtocolBuilder httpProtocol =
        http.baseUrl("https://computer-database.gatling.io")
            .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
            .acceptLanguageHeader("en-US,en;q=0.5")
            .acceptEncodingHeader("gzip, deflate")
            .userAgentHeader(
                "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0"
            );

    ScenarioBuilder users = scenario("Users")
            .exitBlockOnFail().on(
                    http("req1").get("/"),
                    http("req2").get("/").check(status().is(999)),
                    http("req3").get("/")
            );

    {
        setUp(
            users.injectOpen(atOnceUsers(1))
        ).protocols(httpProtocol);
    }
}

2023-12-19 11:57:23                                           0s elapsed
---- Requests ------------------------------------------------------------------
> Global                                                   (OK=3      KO=1     )
> req1                                                     (OK=1      KO=0     )
> req1 Redirect 1                                          (OK=1      KO=0     )
> req2                                                     (OK=1      KO=0     )
> req2 Redirect 1                                          (OK=0      KO=1     )
---- Errors --------------------------------------------------------------------
> status.find.is(999), but actually found 200                         1 (100,0%)

[nvervelle]

I will try to find a way to reproduce it with a simple example.

What you describe is what I experience with 3.9.5.6, but with 3.9.10.1, I have the following difference :

• The error is only visible in the logs as DEBUG, but not in the textual report (there's no request in KO status, no - Errors ----...)
• The logs only contain one line for the error, not the detailed version (with the Request, Session, HTTP request and HTTP response parts)
• The simulation simply doesn't do anything more and never stops.

[nvervelle]

Hi @slandelle , I did some tests with 3.9.10.1 and 3.9.5.6 with a simplified example based on yours. I notice a really strange difference in the console output between the two versions :

With 3.9.5.6, the logs shows the body of the HTTP response :

12:38:44.766 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'req2 Redirect 1' failed for user 1: status.find.is(999), but actually found 200
12:38:44.776 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - 
>>>>>>>>>>>>>>>>>>>>>>>>>>
Request:
req2 Redirect 1: KO status.find.is(999), but actually found 200
=========================
Session:
Session(Users,1,HashMap(gatling.http.cache.baseUrl -> https://computer-database.gatling.io, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@cbcf4ab, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@3f15b0db, gatling.http.referer -> https://computer-database.gatling.io/computers, 3d865a5c-99ef-4f85-955d-0cd4d73e727f -> 0),OK,List(TryMaxBlock(3d865a5c-99ef-4f85-955d-0cd4d73e727f,tryMax-1-inner,KO)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$563/0x00007f16d84fbe68@1ef1d272,io.netty.channel.epoll.EpollEventLoop@7bfc3126)
=========================
HTTP request:
GET https://computer-database.gatling.io/computers
headers:
        accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
        accept-language: en-US,en;q=0.5
        accept-encoding: gzip, deflate
        user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0
        host: computer-database.gatling.io
=========================
HTTP response:
status:
        200 OK
headers:
        Date: Tue, 19 Dec 2023 11:38:44 GMT
        Content-Type: text/html; charset=UTF-8
        Transfer-Encoding: chunked
        Connection: keep-alive
        Server: Finch
        content-encoding: gzip

body:
<html><head>...</head><body>...</body></html>
<<<<<<<<<<<<<<<<<<<<<<<<<

With 3.10.0.1, the logs show a different error with a StringIndexOutOfBoundsException, as if gatling failed after the request failed :

12:45:06.262 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'req2 Redirect 1' failed for user 1: status.find.is(999), but actually found 200
12:45:06.273 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - 
>>>>>>>>>>>>>>>>>>>>>>>>>>
Request:
req2 Redirect 1: KO status.find.is(999), but actually found 200
=========================
Session:
Session(Users,1,HashMap(gatling.http.cache.baseUrl -> https://computer-database.gatling.io, e2b0dd1b-c91e-43c2-8aa7-f595c435b8b9 -> 0, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@690e039a, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@27f22535, gatling.http.referer -> https://computer-database.gatling.io/computers),OK,List(TryMaxBlock(e2b0dd1b-c91e-43c2-8aa7-f595c435b8b9,tryMax-1-inner,KO)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$642/0x00007f9c04525598@5e21417c,io.netty.channel.epoll.EpollEventLoop@36a7abe1)
=========================
HTTP request:
GET https://computer-database.gatling.io/computers
headers:
        accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
        accept-language: en-US,en;q=0.5
        accept-encoding: gzip, deflate
        user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0
        host: computer-database.gatling.io
=========================
HTTP response:
version:
        HTTP/1.1
status:
        200 OK
headers:
        Date: Tue, 19 Dec 2023 11:45:06 GMT
        Content-Type: text/html; charset=UTF-8
        Transfer-Encoding: chunked
        Connection: keep-alive
        Server: Finch
        content-encoding: gzip

body:
Could not decode response body: j.l.StringIndexOutOfBoundsException: Range [0, 500000) out of bounds for length 2455
<<<<<<<<<<<<<<<<<<<<<<<<<

Simplified simulation used :

import io.gatling.javaapi.core.CoreDsl;
import io.gatling.javaapi.core.ScenarioBuilder;
import io.gatling.javaapi.core.Simulation;
import io.gatling.javaapi.http.HttpDsl;
import io.gatling.javaapi.http.HttpProtocolBuilder;

@SuppressWarnings("PMD.NonStaticInitializer")
public class ExitSimulation extends Simulation {

  HttpProtocolBuilder httpProtocol =
      HttpDsl.http
          .baseUrl("https://computer-database.gatling.io")
          .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
          .acceptLanguageHeader("en-US,en;q=0.5")
          .acceptEncodingHeader("gzip, deflate")
          .userAgentHeader(
              "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0");

  ScenarioBuilder users =
      CoreDsl.scenario("Users")
          .exitBlockOnFail()
          .on(CoreDsl.exec(HttpDsl.http("req2").get("/").check(HttpDsl.status().is(999))));

  {
    setUp(users.injectOpen(CoreDsl.atOnceUsers(1))).protocols(httpProtocol);
  }
}

[slandelle]

@nvervelle Your help is much appreciated, but really, this is a but tracker.
Please stick to 1 ticket per issue. Different issues have different causes and will get fixed by different commits, possibly not shipped in the same release.

I've opened a ticket for the other StringIndexOutOfBoundsException issue: #4496 and already fixed it.

Waiting for your reproducer about exitBlockOnFail.

[slandelle]

Can you reproduce without enable DEBUG/TRACE logging?

[nvervelle]

I'm still trying to reproduce the problem with a simple example, but still no success... I'm doing other attempts and will keep you posted.

I posted the problem with the exception here, because I thought it may have a link to the problem I see. Apparently no, so I will try further to produce a simple reproducer.

[nvervelle]

My problem is related, at least partially, to #4497 : the 415 errors I was receiving were due to the HTTP Content-Type passed by the new version of gatling. With 3.10.2, the requests don't fail, so my simulations are working, but I have to check what happens if a request fail. I will keep you posted.

[nvervelle]

I tried again running the simulations in a real load testing session, with failures in the requests : I don't see the problem anymore. Maybe the various fixes you did were sufficient to fix my problem ?

I suggest to close this issue, and I will post again if I have again this problem.

[slandelle]

OK. Thanks for your feedback!