Comments (8)
The conclusion of talking it over IRL is the following:
- We will disable left stripping by default.
- We will keep right stripping by default.
- We will add an option to the DSL to changing the strictness.
from universal-judge.
I think the argument was made to in any case only strip newlines, not all white space, as this would break some exercises (e.g. those where a grid should be made). This seems reasonable to me.
from universal-judge.
After giving this some thought, here's what I would like to suggest:
- use a parameter (suggested names
sanitize-output
orautocomplete-final-newline
) that adds\n
at the end ofstdout
andstderr
(does not apply to a returned string-value) if it doesn't already end with a newline; the default value of this parameter isTrue
to makestdout
andstderr
automatically compliant with the C-adage that all text files should end with a newline, without these newlines cluttering the test specification - because the previous silently adds newlines to
stdout
andstderr
by default, it is quite safe to migrate to exact comparison between expected and generatedstdout
andstderr
without stripping by default (without breaking too many exercises that already use TESTed)
I guess this is the better of worlds where we do exact matching by default, but with silent addition of final newlines in case they are missing from the test specification (or without the need to clutter the specification with newlines after single-line outputs. This might also alleviate the tricky ways in which YAML treats newlines in multiline strings.
from universal-judge.
Also relates to dodona-edu/dodona#5168: automatic insertion of final newlines if missing (by default) would also resolve this Python Tutor issue (although it would seem more sensible if Dodona also wouldn't ever strip the final character (newline)).
from universal-judge.
I now see that my suggestion is also what @niknetniko suggested (but didn't read the original post until the very end).
from universal-judge.
Here's another reason why I would by default append a newline to stdout
and stderr
the string included in the test specification does not end with a newline: to follow the C/POSIX convention in the expected output displayed in Dodona:
from universal-judge.
From #486:
That the expected/generated values are not the same is a big argument in favour of doing strict comparison in my opinion, so I would not be against making the validations strict (as the main argument against it previously was that the status quo caused no problems if I recall correctly).
from universal-judge.
Here's an example that shows how confusing the newline-difference is for students, as it distracts from the true issue:
from universal-judge.
Related Issues (20)
- Inconsistent naming of bash shell scripts
- Run custom oracles with test directory as its current working directory
- Remove ability to write custom oracles (or check functions) in anything but Python?
- Output fallthrough
- JSON failed to parse HOT 1
- Fetch stdin from file as an alternative to inlining it into test suite HOT 1
- Do not quote options on bash command lines generated by TESTed
- Support tests for files
- Generate "final" variables if possible
- Show stdin as a separate channel when used in combination with arguments HOT 2
- Closing newline insertion HOT 1
- Specify interface that must be implemented
- Link bash compilation error to corresponding line in code
- Support before/after in DSL
- Support attribute assignment
- Testing stdin/stdout and expressions at the same time? HOT 3
- Update node.js to latest version HOT 3
- JavaScript: consider alternative ways to create object as alternatives HOT 1
- Bug with Javascript objects
- representation of Set and Map as return value HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from universal-judge.