FasTest (Fast Test) is a cross-language/framework tool created to enable developers and testers to specify the parts of tests that are distinct, and meaningful, thereby reduce both the time to create tests, and the time to maintain them. FastTest also has the benefit of doubling as documentation for how methods are used.
FasTest is built around ==3?== principals:
- Simple tests should be simple to write
- Complicated test cases should be easier to understand
- Repetitive forms in coding should be automated
At its most basic, fast test allows developers to specify tests and expectd results as stand alone function calls, and will handle the boiler plate code required to make those tests functions. Below is an example of a test call for a string join function.
("Fast","Test") => "FastTest"
FastTest exists as custom JavaDoc directives, the primary one being @test. Code within the @test directive, along with the other directives specified by FastTest will be parsed and output as test code in files managed by FastTest. It is not recommended that these files be edited directly, as they will be erased and re-written frequently. That said, it is expected that, should changes need to be made to a test that cannot be generated by FastTest, that that test can copied out of FastTest's output, and modified directly.
FasTest is different than other languages because it is designed around exclusive parsing. This means that outside of some very simple syntax, everything the parser encounters is assumed to be a string being supplied by the user for use in the output template. For example: Any line that contains either naked parenthesis or the '=>' symbol is considered to be a test rule, and is broken apart for use in the output template, otherwise all lines are considered to be part of the test template, and are copied and pasted whole in to the template according to their context. In this way, the application of the language used to generate the tests much more closely resembles a macro based transform, as opposed to parsed language.
Given the anatomy of most tests, there is allot we can infer from the context in which a test needed. In our case, given that the language describing a test exists within the documentation describing the function or class, we can assume that:
- The user wants to create a test
- The test needs a unique name, likely something to do with the function in question
- That test will involve calling the function
- Depending on how the function is declared, we may need to instantiate the object that owns the function
Therefor, the simplest test we can write will just be to call the function file:String.js
class String{
/**
* @test
* ("Fast","Test")
**/
function join(a,b){
...
}
}
This will generate file:StringTest.js
class StringTest{
@Test
function join_test() {
var article = new String();
article.join("Fast","Test");
}
}
While it is nice to know that we can get up and running with so little effort, it would be better if we could check the outcome of our function call, perhaps something like file: String.js
class String{
/**
* @test
* ("Fast","Test") => "FastTest"
**/
function join(a,b){
...
}
}
This will generate
file:StringTest.js
class StringTest{
/**
* ensures that article.join("Fast","Test") results in "FastTest"
**/
@Test`
function join_test() {
var article = new String();
assertEquals("FastTest",article.join("Fast","Test"));
}
}
were the join method declared as static, instead of being a member function, the test that was generated would instead look like this. Note that we are no longer instantiating our method's object.
file:StringTest.js
class StringTest{
/**
* ensures that article.join("Fast","Test") results in "FastTest"
**/
@Test
function join_test(){
assertEquals("FastTest",String.join("Fast","Test"));
}
}
What if we want to test for an exception? We can indicate that we expect an exception by marking the expected result as an exception with the ( exception_class )EX call: file: String.js
class String{
/**
* @test
* ("Slow","Test") => ( ArgumentException )EX
**/
function join(a,b){
if (a=="Slow")
throw new ArgumentException("Slow tests are lame")
}
results in
file:StringTest.js
class StringTest{
/**
* ensures that article.join("Slow","Test") throws an ArgumentException
**/
@Test`
function join_test(){
assertThrows(ArgumentException.class,() -> {String.join("Slow","Test"}));
}
}
While we will use primitive types for our examples, because it keeps the examples simple, the FasTest nomenclature is made almost exclusively of symbols (parenthesis, commas, equals, greater than, exclamation point etc.) and ANYTHING else is assumed to be written in the language in which the code is being generated, therefor, any place you see something like a literal string, you can substitute a function call, constructor, or anything you like, and the text will simply be propagated to the tests.
To maintain test simplicity, its often best practice to have code that executes before and after all tests, and sometimes to have test class member variables that can persist between tests. Within FastTest this is accomplished using the custom JavaDoc directives: @testSetup, @testTeardown and @testDeclaration respectively, within the javadoc block associated with the class of which the function is a member, with the contents of each being included in methods that are called before and after each function, or as part of declaration space within the test class.
/**
* @testDeclaration
* int i;
* @testSetup
* System.out.print("Test Setup");
* @testTeardown
* System.out.print("Test Teardown");
**/
class exampleClass{ ... }
results in
public class exampleClassTest {
int i;
@Before
void setUp() {
System.out.print("Test Setup");
}
@After
void tearDown() {
System.out.print("Test Teardown");
}
...
}
The full syntax for a test rule is:
{test name} {(constructor parameters)} (parameters){method/member calls} {{!}=> result {#error message}}
This means that we can supply our test:
- A method name to use
- If multiple tests use this name, they will automatically be numbered
- Constructor parameters for the instantiation of our article under test
- Our test call
- Not optional
- Code to call on the product of our test call
- A result to expect
- Or explicitly NOT expect
- An error message for when the test fails
Meaning that with the following
class String{
/**
* @test
* simpleJoinTest ("parameter") ("Fast","Test").length() => 8 #the join resulted in the wrong number of letters
**/
function join(a,b){
...
}
}
We will generate the test
/**
* Ensuring join("Fast","Test").length() results in 8
**/
public void simpleJoinTest_test() {
var article = new Demo1("parameter");
assertEquals(
"the join resulted in the wrong number of letters",
8,
article.join("Fast", "Test").length());
}
While concise, this nomenclature isn't nearly flexible enough to allow us to create real world tests. For real world tests we need to be able to specify code that comes before and after our function call. To do this, we can simply include code before and after our test specification. Also, comments that appear before our test, and before the code that runs before our test (if any) will be the comments that go above the test in the resulting test.
`class String{
/**
* @test
* //this is an example for how to include things around your test
* var article = new String("this print statement will be included in the test!")
* //this comment will end up in your test
* ("Fast","Test") => "FastTest
* console.log("this will too!")
**/
function join(a,b){
...
}
}
will result in
class StringTest{
/**
* this is an example for how to include things around your test
**/
@Test
function join_test() {
var article = new String("this statement will be included in the test!")
//this comment will end up in your test
assertEquals("FastTest",article.join("Fast","Test"));
console.log("this will too!")
}
}
It should be noted that the inclusion of test setup code disables the inclusion of the default constructor, necessitating that the user instantiate the test article themselves
Test fragments are context free chunks of code that can be included arbitrarily and shared between tests. They are denoted by the custom Javadoc directive
@testFragment [testFragmentName]
and all text within them will replace their invocation within FastTest Javadoc directives. This includes TestFragment directives, so that @testFragments can refer to one another. @testFragment(s) are invoked using the syntax:
$testFragmentName
Test Fragments, and other FastTest data is inherited in a hierarchy, whereby local directives will override less local directives in the chain:
- Test directives
- Function directives
- Class directives
- Namespace directives
- Project directives Whereby Test directives are the most important, and Project directives are the least important
Function Directives are directives within the javadoc for the current function under test, however directives can be specified multiple times, giving way to test directives, or the most recent specification of a directive prior to a test directive.
/**
* @testFragment myFragment
* .log("hello from the first version of myFragment")
* @test
* console.$myFragment
* ()=>0
* @testFragment myFragment
* .log("hello from the second version of myFragment")
* @test
* console.$myFragment
* ()=>0
**/
Class directives are directives specified in the javadoc block of the current class
Namespaces/directories can contain files with a suffix of ".fastest" that can contain directives. These directives will be read by the FastTest parser, giving more weight to namespaces closer to the current class.
The project root can contain files with a suffix of ".fastest" that can contain directives. These directives will be read by the FastTest parser, and their directives are overridden by directives at all other levels
This system allows for many things including the provision of standardized, parameterized test articles with default values. for example, at the class level the person class can provide a standard person instantiation to use in tests, which references a fragment for the name of the person, which the class also provides
class Person{ ... }
/**
* @testFragment personName
* "bob"
* @testFragment testPerson
* new Person($personName,"address info etc");
**/
A test can now refer to the standard person object AND optionally override the first name provided to the constructor
class Person{
/**
* @test
* var myTestPerson = $person; //this person is named bob
* (myTestPerson)
*
* @testFragment personName
* "sally"
*
* @test
* var myTestPerson = $person; //this person is named sally
* (myTestPerson)
**/
function print(person){ ... }
}
This is of particular use in languages that don't support optional parameters, necessitating the use of other constructs to achieve the same ends
Data fragments are a special type of TestFragment. The operate in the same manner as a TestFragment in their declaration and usage, aside from being declared using the @testData directive except that their first use will result in their contents being used to declare a variable, which will then be used for all subsequent invocations of the data fragment within a test. This means that the following
class String{
/**
* @testData myTestData
* new StringBuilder("test data")
* @test
* console.log("we are about to test using" + $myTestData.toString())
* ($myTestData)
* console.log("the result of the test was" + $myTestData.toString())
**/
static function reverse_mutator(a){
...
}
}
will result in
class StringTest{
@Test
function reverse_mutator_test() {
var testData = new StringBuilder("test data")
console.log("we are about to test using" + testData.toString())
String.reverse_mutator(testData);
console.log("the result of the test was" + testData.toString())
}
}
==this will change before v1.0== In accordance with our principles, we need to be able to re-use code that comes before and after a test. We do this through the use of whitespace. It should be noted that between the use of (per test method) pre and post test code, common initialization code is more common. As such, the indicator as to weather or not to include pre and post test code is in the amount of whitespace surrounding a test. If there are no blank lines between 2 tests, they will share pre and post test code (though additional lines after the second test will be appended to its post test code
class addclass{
/**
* console.log("this is included before both tests")
* (1,2)=>3
* console.log("this is included after both tests")
* (2,3)=>5
* console.log("this is included only after the second test")
**/
static function add(a,b){ ... }
}
resulting in
@Test
function add_test() {
console.log("this is included before both tests")
assertEquals(3,addclass.add(1,2));
console.log("this is included after both tests")
}
@Test
function add_test_2() {
console.log("this is included before both tests")
assertEquals(5,addclass.add(2,3));
console.log("this is included after both tests")
console.log("this is included only after the second test")
}
However, leaving a blank space between tests will clear the code that is included after the first test, but still append code before the second test to the code from the first test
class addclass{
/**
* console.log("this is included before both tests")
* (1,2)=>3
* console.log("this is included only after the first test")
*
* console.log("this is included only before the second test")
* (2,3)=>5
* console.log("this is included only after the second test")
**/
static function add(a,b){ ... }
}
resulting in
@Test
function add_test() {
console.log("this is included before both tests")
assertEquals(3,addclass.add(1,2));
console.log("this is included only after the first test")
}
@Test
function add_test_2() {
console.log("this is included before both tests")
console.log("this is included only before the second test")
assertEquals(5,addclass.add(2,3));
console.log("this is included only after the second test")
}
Finally, if two tests have two consecutive blank spaces between them, then the tests will not share any pre or post test code
class addclass{
/**
* console.log("this is included only before the first")
* (1,2)=>3
* console.log("this is included only after the first test")
*
*
* console.log("this is included only before the second test")
* (2,3)=>5
* console.log("this is included only after the second test")
**/
static function add(a,b){ ... }
}
resulting in
@Test
function add_test() {
console.log("this is included only before the first test")
assertEquals(3,addclass.add(1,2));
console.log("this is included only after the first test")
}
@Test
function add_test_2() {
console.log("this is included only before the second test")
assertEquals(5,addclass.add(2,3));
console.log("this is included only after the second test")
}
Test Sets allow us to specify whole sets of tests with a single line, incorporating all of the functionality covered so far. Test Sets are specified as **(**value1, value2 ... )SET. Sets are replaced by their containing values. Meaning that the following
class addclass{
/**
* @test
* ((1,2,3)SET,2) !=> 2
**/
static function add(a,b){ ... }
}
is the same as writing
class addclass{
/**
* @test
* (1,2) !=> 2
* (2,2) !=> 2
* (3,2) !=> 2
**/
static function add(a,b){ ... }
}
and will result in
class addclass{
@Test
function add_test() {
assertNotEquals(2,addclass.add(1,2));
}
@Test
function add_test_2() {
assertEquals(2,addclass.add(2,3));
}
@Test
function add_test_3() {
assertEquals(2,addclass.add(3,3));
}
}
There are actually several set specifiers: ()SET and ()SETA-()SETZ. sets declared with the same specifier will produce tests on the same schedule.
class addclass{
/**
* @test
* ((1,2,3)SETA,2) => (3,4,5)SETA
**/
static function add(a,b){ ... }
}
is the same as writing
class addclass{
/**
* @test
* (1,2) => 3
* (2,2) => 4
* (3,2) => 5
**/
static function add(a,b){ ... }
}
However, different set specifiers operate on seperate schedules, leading to a full-join behavior between sets
class addclass{
/**
* @test
* ((1,2,3)SETA,(10,20,30)) !=> (3,4,5)SETA
**/
static function add(a,b){ ... }
}
is the same as writing
class addclass{
/**
* @test
* (1,10) !=> 3
* (2,10) !=> 4
* (3,10) !=> 5
* (1,20) !=> 3
* (2,20) !=> 4
* (3,20) !=> 5
* (1,30) !=> 3
* (2,30) !=> 4
* (3,30) !=> 5
**/
static function add(a,b){ ... }
}
and bear in mind that sets contain raw code separated by commas, so they can be used to construct arbitrary strings, sets of function calls, all kinds of things! Sets can even be embedded in other sets. What do you think the following produces
("((a,my)SETBpet,garden)SETA (could be,is)SETC a (good,great)SETB thing to (give to me, have)SETB")
Test lists are similar to test Sets, but take a more formal approach. A test list is easier to read, and easier to maintain, but requires a little more setup ahead of time. Test Lists allow us to specify a named set of possible values for each parameter in the method under test, and then filter a full join of all of those values to specify the expected outcome. We specify parameters and their values in order within our test specification using the following syntax
ParameterName : { valueName : valueLiteral , ... }
example:
@test
firstParameter : {a:new String("A"),b:"B",c:"C"}
secondParameter : {d:"D",e:"E",f:"F"}
Where parameter name is optional. Once we have our list of possible values for our parameters, we can specify how to filter them. the most basic way to do this is just to use the names of the values. For this we will use our rule syntax with one addition: we will surround our list of rules in parenthesis. This will give us something like this:
( (a,d).length() => 2
(a,e).length() !=> 3 )
This is a test list, where each line can be used to specify a different expected outcome. We can swap our our value names for filters (*(all values),multi-value, ranges and regular expressions are legal) however there is one caveat. This test list behaves like an access list, in that each set of parameter values is matched only once, to the first line (from top to bottom) with rules that match its parameters. (if no line matches the combination of parameter values, no test is generated for that set) Therefor, given that the second set of parameters contains 3 values:
( (a,d).length() => 2
(a,*).length() !=> 3 )
will produce the following tests
(new String("A"),"D").length() => 2
(new String("A"),"E").length() !=> 3
(new String("A"),"F").length() !=> 3
Matches all values
( (a,d).length() => 2
(a,*).length() !=> 3 )
A space separated list of values between commas. Each value listed is matched:
( (a,d).length() => 2
(a,e f).length() !=> 3 )
Ranges can be mixed in with multi-value matches. ranges are values surrounded by brackets, with a hyphen. Ranges assume an order, which in this case is the order in which value names are specified) This means that a range can be [e-f], however if a range includes an end of the list, that value can be excluded, meaning that [e-f] can be specified as just [e-]
Regular Expressions can be used to specify a set of value names to be used in a test rule. Regular Expressions are specified as the content of the regular expression, surrounded by 2 forward slashes. Consider the following example:
pressure : {
ERR_LOW:0,
OK_LOW:1,
OK_HIGH:1,
ERR_HIGH:3}
temperature : {
OK:91,
CRITICAL_HIGH):99,
ERR:100}
volume : {
ERR_LOW:0.1,
OK_LOW:0.91,
IDEAL:0.92,
OK_HIGH:0.93,
ERR_HIGH:100}
( (/^ERR.*/,*,*) => (PressureOutOfRange)EX
(*,/^ERR.*/,*) => (TemperatureOutOfRange)EX
(*,*,/^ERR.*/) => (VolumeOutOfRange)EX
(*,*,*) !=> (Exception)EX )
The above generates tests that call the test article with all possible combinations of parameters in cases where
- pressure is out of range
- PressureOutOfRange will be thrown
- temperature is out of range, unless pressure is out of range
- TemperatureOutOfRange will be thrown
- volume is out of range, unless pressure or temperature is out of range
- PressureOutOfRange will be thrown
- otherwise, ensure that there are no exceptions
with 16 lines of code we detail the intended results of 60 calls to our method while managing to capture our intentionality in a succinct manner! With names that establish relationships between values, using regular expressions to filter our parameter values can be an incredibly powerful tool.
FastTest is a tool that spans multiple languages as well as multiple testing frameworks. Through templating it is able to act as a single test specification nomenclature that can act as a universal testing language. Presently FasTest has templates for
In spite of its name, JavaDoc is a standard for code documentation of functions and classes that can be used in almost any language. JavaDoc provides a list of standardized names, or directives, for common parts of documentation that allow that documentation to then be machine readable so that it can be pulled out in to standalone documentation, be shown as live documentation in an IDE or any number of other uses. A common JavaDoc block might look like
/**`
* this is a function that joins 2 strings in to a single string
* @param String stringA the first string of 2 to join together
* @param String stringB the second string of 2 to join together
* @return String
**/
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent