cycling74 / max-test Goto Github PK
View Code? Open in Web Editor NEWAutomated Test Harness for Max by Cycling '74
License: MIT License
Automated Test Harness for Max by Cycling '74
License: MIT License
In addition to the logging to a database, i’d like to see printing to the Max window, with a conclusion when all asserts in the patch has been performed about how many assertions where run, and how many failed. That would be the quickest way to see how a test fares when running it manually, in particular if the test patch contains enough assertions that they need to be embedded in sub patches.
When I'm running tests I'm seeing several patchers from distinct test projects open at the same time, although they're generally all closed at the end. This is causing issues with (for example) clashes between names/ports in send/receive pairs.
This is a bit sporadic, although on my El Capitan machine I have one checkout of max-test which always works and another which always fails. On my Sierra machine it always fails.
In Sierra, the process runs - very slowly - for a while. See screenshot. The last meaningful message is:
[0901/184516:WARNING:dns_config_service_posix.cc(217)] Failed to read DnsConfig.
(That's nearly an hour into the running time.)
should just have it assert that there is a test.terminate object in the patcher -- and fail when it is missing, and then move on. Right now the whole thing stalls waiting for a termination that isn't coming.
It seems that the test process hangs when first opening Max, until I focus and move the mouse pointer in one of Max's windows. Then it runs to completion.
OS X 10.11.6.
There are wrinkles in floating point comparison still.
Perhaps the code from Jamoma2 can be helpful here for potential improvements.
If Max is running when the tests are started, it seems that a second copy of the same Max is launched for testing. (Seems benign, so far.)
Can't run on a Max with spaces in its name. (Probable cause: adoption of IO.popen
.)
yedikule:ruby nick$ ruby test.rb "/Applications/Max bb_mxj_less.app/"
Max Automated Test Runner
Starting the OSC Server...
Launching Max...
test.rb:174:in `popen': No such file or directory - /Applications/Max (Errno::ENOENT)
from test.rb:174:in `launchMax'
from test.rb:194:in `<main>'
Currently, the test runner always returns 0:
Line 291 in 7097a11
This means that, for example, if max-test is used in GitHub Actions, the action always succeeds regardless of whether any tests fail.
A potential workaround might be to check the result of the MAXTEST_PASS
environment variable:
Lines 275 to 277 in 7097a11
However, environment variables set this way are not (and indeed can’t be) exported to the parent process, as can be seen by running:
ruby -e 'ENV["XXX_RUBY_ENV_TEST_XXX"] = "success"'
echo "XXX_RUBY_ENV_TEST_XXX is \"$XXX_RUBY_ENV_TEST_XXX\""
Hi all,
anybody has instructions on how to build max-test for 64-bit Windows? I am using VS2017 but I have progressed very little.
Regards,
Juan.
Hi folks, I followed the tutorial video and got some errors attempting to run the tests following the steps in the video.
The first issue is just that the video is missing the critical step of making the max-test-config.json file. Maybe it can be updated or something added in the description? The error is not too obvious, without doing it I got:
Result:
1: from /usr/local/lib/ruby/gems/2.5.0/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:147:in new' /usr/local/lib/ruby/gems/2.5.0/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:147:in
initialize': no such table: tests (SQLite3::SQLException)
After fixing this, I still got errors, though my tests worked. I'm not sure what happened, but it doesn't look like the clean output in the video.
Full Output:
`
MacBook-Pro-7:ruby iainduncan$ ruby test.rb "/Applications/"
Max Automated Test Runner
MAX FOLDER
/Applications
Starting the OSC Server...
Launching Max...
Establishing Communication with Max...
2020-04-18 08:31:02.840 Max[63851:40438231] packages: package name (Video and Graphics) is different from folder name (Video & Graphics)
2020-04-18 08:31:02.853 Max[63851:40438231] packages: package name (Max-SDK) is different from folder name (max-sdk-8.0.3)
2020-04-18 08:31:03.745 Max[63851:40438231] max_startmaxserver
2020-04-18 08:31:04.536 Max[63851:40438231] clang using Sandy Bridge workaround
2020-04-18 08:31:04.709 Max[63851:40438231] sqlite version 3.8.5
2020-04-18 08:31:05.376 Max[63851:40438231] Max: starting maxdb update (schema version 8.1.2b)
2020-04-18 08:31:05.410 Max[63851:40438231] code: en-CA, name English
2020-04-18 08:31:06.806 Max[63851:40438231] test.port sending on 4792
2020-04-18 08:31:06.817 Max[63851:40438231] test.port listening on 4791
2020-04-18 08:31:06.977 Max[63851:40438231] maxworkspace_restore: vol=-674 filename=maxworkspace-63851.txt fullpath=Macintosh HD:/Users/iainduncan/Library/Application Support/Cycling '74/Max 8/Crash Recovery/maxworkspace-63851.txt
Sending ping to Max.
/usr/local/lib/ruby/gems/2.5.0/gems/rosc-0.1.3/lib/osc.rb:297: warning: constant ::Fixnum is deprecated
Ping successfully returned.
2020-04-18 08:31:09.429 Max[63851:40438451] Max: maxdb update complete
Waiting for the Max database to complete...
Sending query to Max.
Telling Max to run all of the tests for us...
2020-04-18 08:31:29.599 Max[63851:40438231] doesn't understand "won't-understand-BOOGLE"
2020-04-18 08:31:30.041 Max[63851:40438231] index out of range for key test-
2020-04-18 08:31:30.515 Max[63851:40438231] @ loadbang: Kung
2020-04-18 08:31:30.515 Max[63851:40438231]
2020-04-18 08:31:30.515 Max[63851:40438231] @ set: Kung
2020-04-18 08:31:30.515 Max[63851:40438231]
2020-04-18 08:31:30.515 Max[63851:40438231] @ set: Fu
2020-04-18 08:31:30.515 Max[63851:40438231]
2020-04-18 08:31:31.303 Max[63851:40438231] ad_spin: looks like audio callbacks have stopped
2020-04-18 08:31:34.075 Max[63851:40438231] integration tests complete
RESULTS
Executed 9 Tests with 39 Assertions
All assertions passed. Congratulations!
pass
Full test results can be found @
/Volumes/Macintosh HD/Applications/Macintosh HD--Applications--maxtestresults.db3
and explored in your favorite SQLite database client.
#<Thread:[email protected]:102 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
7: from test.rb:103:in block in establishCommunication' 6: from /usr/local/lib/ruby/gems/2.5.0/gems/rosc-0.1.3/lib/osc/udp.rb:18:in
serve'
5: from /usr/local/lib/ruby/gems/2.5.0/gems/rosc-0.1.3/lib/osc/udp.rb:18:in loop' 4: from /usr/local/lib/ruby/gems/2.5.0/gems/rosc-0.1.3/lib/osc/udp.rb:20:in
block in serve'
3: from /usr/local/lib/ruby/gems/2.5.0/gems/rosc-0.1.3/lib/osc/server.rb:33:in dispatch' 2: from /usr/local/lib/ruby/gems/2.5.0/gems/rosc-0.1.3/lib/osc.rb:242:in
decode'
1: from /usr/local/lib/ruby/gems/2.5.0/gems/rosc-0.1.3/lib/osc.rb:242:in new' /usr/local/lib/ruby/gems/2.5.0/gems/rosc-0.1.3/lib/osc.rb:242:in
initialize': no implicit conversion of nil into String (TypeError)
`
Running test.rb with Ruby 2.6.3 produces errors like this:
#<Thread:0x00007ff18f05e280@packages/max-test/ruby/test.rb:102 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
7: from packages/max-test/ruby/test.rb:103:in `block in establishCommunication'
6: from /Users/benbracken/.rvm/gems/ruby-2.6.3/gems/rosc-0.1.3/lib/osc/udp.rb:18:in `serve'
5: from /Users/benbracken/.rvm/gems/ruby-2.6.3/gems/rosc-0.1.3/lib/osc/udp.rb:18:in `loop'
4: from /Users/benbracken/.rvm/gems/ruby-2.6.3/gems/rosc-0.1.3/lib/osc/udp.rb:20:in `block in serve'
3: from /Users/benbracken/.rvm/gems/ruby-2.6.3/gems/rosc-0.1.3/lib/osc/server.rb:33:in `dispatch'
2: from /Users/benbracken/.rvm/gems/ruby-2.6.3/gems/rosc-0.1.3/lib/osc.rb:242:in `decode'
1: from /Users/benbracken/.rvm/gems/ruby-2.6.3/gems/rosc-0.1.3/lib/osc.rb:242:in `new'
/Users/benbracken/.rvm/gems/ruby-2.6.3/gems/rosc-0.1.3/lib/osc.rb:242:in `initialize': no implicit conversion of nil into String (TypeError)
Perhaps newer versions of Ruby are being more strict somewhere?
The tests complete and successful logs are written, so it seems like a fairly cosmetic issue.
Hello!
after building the project I ran the ruby test.rb "/Applications/"
command, I can see that Max8 is starting correctly, but then I get an error from ruby:
Sending ping to Max.
/Users/davide/Documents/Max 8/Packages/max-test/ruby/rosc/lib/osc.rb:300:in `encode': uninitialized constant OSC::Packet::Fixnum (NameError)
when Fixnum; [o].pack 'N'
^^^^^^
from /Users/davide/Documents/Max 8/Packages/max-test/ruby/rosc/lib/osc.rb:138:in `encode'
from /Users/davide/Documents/Max 8/Packages/max-test/ruby/rosc/lib/osc/transport.rb:10:in `send'
from test.rb:104:in `establishCommunication'
from test.rb:199:in `block in <main>'
from /Users/davide/.rubies/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout'
from /Users/davide/.rubies/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:36:in `block in catch'
from /Users/davide/.rubies/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:36:in `catch'
from /Users/davide/.rubies/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:36:in `catch'
from /Users/davide/.rubies/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:198:in `timeout'
from test.rb:197:in `<main>'
working on MacOS 12.6 running Max8.5 arm64
Thanks!
e.g. when inside of an app bundle it will write someplace wonky and hostile to apple-security requirements.
currently it just covers single floats, but ideally could also cover:
thus providing a unified equality object
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.