Comments (16)
This can fix the issue if control gets to this line:
Because with this, if the tests complete but the process hangs, the run can succeed. However, the telltale message is not in the output, so this didn't run, and the previous run you linked wasn't hanging after the test run has completed (i.e. all tests produced their outputs) but somewhere before that.
BTW there are a huge number of exceptions in the workflow output, I suggest checking these out, e.g.:
2024-07-10T00:14:22.5934506Z 2024-07-10 00:13:55.5118|Default|00-0b8735f889ef785eacdbe7443da03d71-387064e610ec3783-00||Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware|WARN|The response has already started, the error page middleware will not be executed.
2024-07-10T00:14:22.5939726Z 2024-07-10 00:13:55.5118|Default|00-8b46da17ed6f9ef1aabca0f86ea1be6c-c7c3e103a1df7f90-00||Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware|ERROR|An unhandled exception has occurred while executing the request. System.InvalidOperationException: Two concurrent threads have been detected accessing the same ISession instance from:
...
And a lot more, even Shouldly ones, so this should've really failed (though completed).
from ui-testing-toolbox.
Related/same? #228 (and Lombiq/Open-Source-Orchard-Core-Extensions#736).
Did you try configuring dotnet-test-process-timeout
so the timeout is handled for the test process instead of the whole workflow? (Or even the per-test timeouts.)
from ui-testing-toolbox.
Related/same? #228 (and Lombiq/Open-Source-Orchard-Core-Extensions#736).
The examples I mentioned happened on Ubuntu and restarting did not fix them, so I don't think #228 is related.
Did you try configuring dotnet-test-process-timeout so the timeout is handled for the test process instead of the whole workflow? (Or even the per-test timeouts.)
Thanks, I will try that.
from ui-testing-toolbox.
Despite the title of that issue, this happened many times under Ubuntu too (but first it seemed it's Windows-only). But yeah, that's about random hangs, not consistent ones. That look more like an app-specific issue.
from ui-testing-toolbox.
Adding dotnet-test-process-timeout
magically fixed the problem in OCC (OrchardCMS/OrchardCore.Commerce#454). What does that mean?
from ui-testing-toolbox.
Don't question it.
I would never. All praises to the Omnissiah!!
BTW there are a huge number of exceptions in the workflow output, I suggest checking these out, e.g.:
Yes, that's why I said that the test hangs instead of fails. (btw on TDEAL it only hangs with the dev
build that uses the standard runner, in PRs if a test fails the run correctly stops) This dotnet-test-process-timeout
is really useful, because now I can see the errors (unlike previously) and I can address them.
from ui-testing-toolbox.
I see. Perhaps this is not actually a hang, then? But rather, it runs retries of a lot of tests, which is slow, and it just times out? With 6 hours in TDEAL that would be extreme, but not impossible (on the slow 2-core HDD default runner; OCC, as public repos, use 4-core SSD runners by default).
from ui-testing-toolbox.
I'm 100% certain it's actually a hang. With TDEAL I know that only the visual verification test failed. Just one test. The same run on buildjet only took 12.5 minutes. So if it took up to an hour with the standard runner I could stomach that, but 6 hours is not possible.
from ui-testing-toolbox.
Then I guess it can hang due to some threads deadlocking with just the two cores. We've seen issues like before, and an ASP.NET Core sync-over-async issue, still unfixed (somewhere linked in the other issue I linked), can cause this.
from ui-testing-toolbox.
Nothing else to do here then, though?
from ui-testing-toolbox.
As part of NEST-501 I also experience this: the DotNest UI tests didn't produce any output here and the workflow timed out after an hour. After adding dotnet-test-process-timeout: 600000
I could see the actual failing test.
from ui-testing-toolbox.
So you also had problems with security scanning. In OCC as well, after filtering out the expected error testing, all other error logs were from the full security scan saying "InvalidOperationException System.InvalidOperationException: Two concurrent threads have been detected accessing the same ISession instance". I've removed the dotnet-test-process-timeout
and temporarily disabled the test and now the run passed (with no |ERROR|
in log) in 8 minutes. I think the security test is accidentally stress testing the runner or YesSql's thread safety by it's starting many requests (nearly) concurrently.
from ui-testing-toolbox.
It's expected for the security scan to start concurrent requests, though the goal is not stress testing and I believe the rate can be adjusted if necessary. However, concurrent requests mustn't result in such a YesSql exception: that only happens if two threads use the same ISession
, not simply access the DB independently. This is something to avoid. Each request uses its own ISession
, so concurrent requests in itself shouldn't cause this (unless some singleton service keeps using the same ISession
under multiple requests, what should be avoided too).
from ui-testing-toolbox.
Under NEST-501 I discovered that security scanning maxing out the CPU (and maybe also RAM) of the runner can cause dotnet test
to hang. You can try Lombiq/GitHub-Actions#370 to see CPU and RAM metrics of runs and try to correlate that with issues you encounter (as a run with an oversaturated CPU can do all kinds of funny things).
I still don't think there's a general issue here, maybe some documentation.
from ui-testing-toolbox.
And if there is a general issue, we should get back to #228.
from ui-testing-toolbox.
So, closing, since there's no new general issue.
from ui-testing-toolbox.
Related Issues (20)
- SignInDirectly randomly fails with 404 (OSOE-727) HOT 11
- Generalize Failure Dump to Test Dump (OSOE-732)
- Ability to configure default browser (OSOE-733) HOT 1
- All UI tests fail (OSOE-748) HOT 13
- Fix the BasicVisualVerificationTests.VerifyNavbar sample (OSOE-755) HOT 5
- Security scan fails with an empty result list when used from a NuGet package. (OSOE-767) HOT 3
- Add support for structured html-validate output (OSOE-770) HOT 4
- Remove MultipleActiveResultSets=True (OSOE-807)
- "Creating the web driver failed..." exception when trying to run tests locally (OSOE-817)
- Update testing dependencies (OSOE-838) HOT 2
- Chrome processes remaining after concurrent headless tests finish (OSOE-839) HOT 4
- Build and Test Windows workflows are timing out (OSOE-840) HOT 9
- Don't include a file called .htmlvalidate.json by default (OSOE-844) HOT 7
- Basic Orchard features testing should include Workflows and Audit Trail (OSOE-847) HOT 4
- Disable "allow pasting" warning in Chrome's dev tools (OSOE-873) HOT 5
- Add support for automatic GraphQL schema discovery (OSOE-882)
- Update `Atata.WebDriverSetup` to latest (OSOE-889)
- Update `Atata.WebDriverSetup` to 2.13.0 to receive Edge driver download fix (OSOE-890)
- No-browser WebDriver for HTML-only testing (OSOE-895)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ui-testing-toolbox.