Comments (17)
Now I have merge the e2e test code into master: #87
We should find a way to maintain examples, which will be used in web reference site and e2e test.
Maybe meta programming, aka code generation will help us: generate test cases and reference pages from examples.
from processing.r.
We need help from the community:
- How to write unit test cases for Processing.R Post on Processing Forum
- Tools to generate Processing References for Processing.R
from processing.r.
comments from the post on Processing Forum:
I am not an expert on this kind of testing, however, one idea that occurs to me:
Sketches could be tests -- running them to see if things work is a form of testing, even without automation.
Automating sketch-testing might be done with pixel-diffs: expected sketch saveFrame() output vs. actual saveFrame() output. This would be good for checking reimplementations of existing documented sketches, and good for catching regressions.
- run a canonical sketch in Processing(Java), e.g. to test rect().
- the sketch generates a known-good image with saveFrame() and/or known-good text log
- write a sketch stub, e.g. to test rect() in Processing.R
- the test will pass when running the sketch generates a matching (or almost-matching) image file to the known-good image.
I once wrote a Processing assignment checker that could also be used as a kind of automated regression tester for sketches. Essentially, the test consists of the sketch and known-good screenshot(s) of the sketch on a certain frame. The test runs the sketch, which generates a new screenshot using saveFrame(). The test then compares the known-good screenshot and the new one. A change in visual output may indicate a regression of the feature that the sketch demonstrates.
(This approach wouldn't work with everything -- it could get really complicated simulating live input to interactive sketches, and reproducibility would suffer with clock-based math because frameCount and millis() are not lockstep.)
from processing.r.
IMO, we should make the boundary clear -- which belongs to Processing.R and which belongs to Processing.
In the test cases in Processing.R, we could:
- Mock Processing from the code or assume that Processing is always right.
- Test the logic in Processing.R, such as Runner or RLangMode.
I will try to write some unit test cases. As for integration test or system test, it is a little complex. And I think it could be another GSoC project 🤔 But it has a universal significance for Processing.
from processing.r.
Sounds good to me! FYI here is a recent thread about a Processing(Java) sketch tester:
https://forum.processing.org/two/discussion/22869/pixel-test-for-simple-sketches-feedback-request
from processing.r.
@gaocegege -- might want to join the discussion there.
from processing.r.
Yeah, I will take a look tomorrow. 😄
from processing.r.
I have a glance on the post. It is really an awesome idea. It is more like end-to-end test. And I will join the discussion tomorrow, oh, today, since it is 2:00 am now.
from processing.r.
FMI: There is a issue when uploading the result to codacy in my local env: codacy/codacy-coverage-reporter#56
from processing.r.
e2e test have a good coverage for runner and RLangPApplet, other classes related to editor or mode couldn't be tested in e2e mode. We could ensure the integration between Processing and Processing.R works well.
from processing.r.
There is a new config file needed when import a new reference code:
examples/
reference/
<function-name>/
.property.yml
<function-name>1/
.test.yml
<function-name>1.rpde
<function-name>2/
...
test:
reference: https://processing.org/reference/images/rect_0.png
from processing.r.
Now the documentation and test cases could be generated from examples/reference automatically. 🎉 It is really awesome. The thing we need to do now is to add new items in examples/reference and re-generate test cases and documentation by running the script https://github.com/gaocegege/Processing.R/blob/master/hack/generate-e2e-test.py and https://github.com/Processing-R/Processing.R-docs/blob/master/scripts/generate-reference.sh
And I think the idea could be used in Processing, too.
from processing.r.
How exciting!!!
I'm happy to see that this approach will work -- and that it will generate both e2e tests and documentation from the same code base of sketch examples and metadata files.
I agree, I think that Processing might be interested in this approach for the main project and other modes.
from processing.r.
from processing.r.
Partially done, I think it is time to close the issue. There are some drawbacks of the current solution:
- No way to test editor or mode itself.
- No way to test animation.
And there are two historical problems: #111 and #103
from processing.r.
This looks fine to me!
- Animation: this seems doable, but not a necessary enhancement, as the vast majority of all Processing reference documentation is not animated.
- Testing the editor: is there any specific editor testing that needs to be done, or should that be left to Processing?
- Testing the mode itself: perhaps open a separate issue (unscheduled, no milestone) for this to come back to it in the future.
from processing.r.
Animation: this seems doable, but not a necessary enhancement, as the vast majority of all Processing reference documentation is not animated.
It works for me.
Testing the editor: is there any specific editor testing that needs to be done, or should that be left to Processing?
I am not sure which layer we should put the test in. But if Processing does it, it works for Processing.R, too. And I will file a new issue about it and try to do a research.
Testing the mode itself: perhaps open a separate issue (unscheduled, no milestone) for this to come back to it in the future.
Yeah, it is helpful. I am not sure if the mode test and the editor test are the same thing, we could put them in one issue.
from processing.r.
Related Issues (20)
- Docs: Tutorials do not list author
- Docs: resolve laDefense.jpg HOT 5
- Support PVector
- Support Table in R HOT 1
- Add GitMate Support
- R Style API HOT 1
- [maintenance] Transfer the project to processing-r and invite Jeremy to be the first official maintainer HOT 2
- Processing.R mode not detected in contributions manager HOT 5
- Build instructions HOT 13
- Update readme.md badge stack HOT 3
- Update windows build versions (processing, ant)
- Transfer codacy integration to codacy org
- Remove RLangMode.zip from repo HOT 1
- Add ribbon nav connecting other Processing sites
- get(), set() -- cast double to int
- Getting this running with Processing 4.0 HOT 5
- example SineWave fails with SketchException
- example He_Mesh/demo incorrectly packaged, fails to run
- processing.R not compatible with R version 3.6.3
- PDE forgets first object declaration in consecutive object declarations in PDE 4.3 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from processing.r.