fleid/asa.unittest

Azure Stream Analytics unit testing solution (fixture and script)

asa.unittest's Issues

Failure with latest Microsoft.Azure.StreamAnalytics.CICD 3.0.0

The latest release seems to be breaking the test scripts
https://www.nuget.org/packages/Microsoft.Azure.StreamAnalytics.CICD/

Can we lock the package to an older version?

07-Sep-2020 08:37:54 | VERBOSE: 301 - Update each conf file
-- | --
07-Sep-2020 08:37:55 | VERBOSE: 401 - Run SA in parallel jobs
07-Sep-2020 08:37:56 | VERBOSE: 402 - Waiting for all jobs to end...
07-Sep-2020 08:38:07 | VERBOSE: 403 - Jobs done
07-Sep-2020 08:38:07 | VERBOSE: 501 - Calculating diffs
07-Sep-2020 08:38:08 | [-] ASAHelloWorld.unit-test 30.75s
07-Sep-2020 08:38:08 | ArgumentException: Invalid array passed in, ',' expected. (41): [{"EventId":"1","EventMessage":"Hello"} {"EventId":"2","EventMessage":"World"} {"EventId":"3","EventMessage":"Hello"} {"EventId":"4","EventMessage":"Friends"}]
07-Sep-2020 08:38:08 | at <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\asa.unittest\1.0.9\private\Get-AutRunResult.ps1: line 88
07-Sep-2020 08:38:08 | at Get-AutRunResult<Process>, C:\Program Files\WindowsPowerShell\Modules\asa.unittest\1.0.9\private\Get-AutRunResult.ps1: line 63
07-Sep-2020 08:38:08 | at Start-AutRun<Process>, C:\Program Files\WindowsPowerShell\Modules\asa.unittest\1.0.9\public\Start-AutRun.ps1: line 128
07-Sep-2020 08:38:08 | at <ScriptBlock>, C:\Users\syscon\bamboo-agent-home\xml-data\build-dir\HUBCD-HUBWINPR54-SAUT\stream_analytic\ASAHelloWorld.Tests\ASAHelloWorld.Tests.ps1: line 14
07-Sep-2020 08:38:08 | Tests completed in 30.75s
07-Sep-2020 08:38:08 | Passed: 0 Failed: 1 Skipped: 0 Pending: 0 Inconclusive: 0

Doc : Explain the parameter strategy of prun

Why the repetition of solution path in -solutionPath and -assertPath

Retrieve Test Results

How can we retrieve the unit test test results (*.xml)?
There are no test reports in 3_assert folder after test execution.

Doc : Update internal chapter

asa.unittest did not copy custom functions

Functions are not cloned to test execution and the generated .asaproj file

Start-AutRun : [CICD nuget 2.4.1] non valid errors found

Since the recent release of version 2.4.1 of the Microsoft.Azure.StreamAnalytics.CICD package, Start-AutRun will return phantom errors due to ordering differences in the result sets.

This is a new behavior not encountered previously.

Start-AutRun : [error]Can't find sa.exe at d:\a\1\s\...\Microsoft.Azure.StreamAnalytics.CICD.2.4.0\tools\sa.exe

Since the recent release of version 2.4.1 of the Microsoft.Azure.StreamAnalytics.CICD package, Start-AutRun will throw an error indicating it can't find version 2.4.0 of sa.exe.

Release 1.0.3

Script : Prevent the installation script to run if the path provided is not valid

Set-Location will fail, but the rest of the script still runs blissfully.

Start-AutRun fails all tests all the time on Azure DevOps

Start-AutRun runs fine locally but fails all tests on Azure DevOps (PS 5.1 and PS 7 / pwsh).

...
VERBOSE: Ending Test Run with X errors
VERBOSE: 001
VERBOSE:    USAGE: jsondiffpatch left.json right.json
...

Reference data sets are not supported without temporal joins

If a query include a non-temporal join to a reference data set (which is the expected usage), the test will fail with random errors (Could not find file ...myoutput.json).

This is a limitation of sa.exe, the executable that currently runs the stream analytics engine for each test.

I should either update the module to invoke the new npm package instead, or deprecate asa.unittest entirely since the new package supports unit testing via the test command.

Start-AutRun does not support project names that contain a dot

In our project we have a naming convention for Visual Studio projects like X.X.X.ASA.ProjectName.
When Start a new AutRun the call gets rejected by the validation.

Is there a specific reason why dots are not included in the naming validation?

Test runs do not support higher CICD nuget version

Hello,
For our ASA project, we are using the CICD version 3.0.0. In Start-AutRun I can specify which version should be used but New-AutProject will always download the 2.4.1 version.

It would be nice to set the version which I need to use for our project in the New-AutProject command.