picimako/mockitools

Mockito framework integration for the IntelliJ platform.

Welcome Dear Visitor!

I create plugins for JetBrains products based on the IntelliJ Platform. I have more than a decade experience in QA and test automation, so my main focus is on test automation related libraries.

Beside the current projects I maintain, I also offer IDE plugin development and consultation services. If you are interested, you can contact me via one of the ways below.

On JetBrains Marketplace:

• Test automation: WireMocha Paid Plugin / Citric Paid Plugin / Mockitools / Gherkin Overview / Terra Support / JBehave Support (I took over its maintenance)
• IDE plugin development: Just Kitting

On npm:

• Test automation: lasponya

Places you can find me at: Website / LinkedIn / Twitter / I(J)nspector blog / JetBrains Platform Slack / E-mail: pmplugindev at protonmail dot com

mockitools's Issues

Intention action: expand simple mock creations to their `MockSettings` based variants

Summary

#58 simplifies some of the MockSettings based mock and spy creation. An intention action could expand the simpler versions to using withSettings(). This would be useful for quick conversion when one has to add more settings to mocks.

Tasks

• Create in intention action that does the conversion in the opposite direction as #58 does.

Intention action to convert Mockito.verify() and BDDMockito.then() calls to InOrder verification

Summary

When there are multiple not-InOder verifications, users may want to do instead InOrder verification on them.

Tasks

• Create an intention action to convert Mockito.verify() and BDDMockito.then() calls to InOrder verification.
• The intention would be available when:
  • there is at least 2 of the same kind of verification within the text selection
  • there is no other content between the verifications (stubbing, variable definition, non-Mockito logic, etc.)
  • the Mockito.verify() and BDDMockito.then() are not mixed within the selection

Inspection: Suspicious call on `MockitoAnnotations.openMocks()`

Summary

There are certain JUnit4 runners and JUnit5 extensions that initialize the mocks and spies in the test class. In thoses cases calling MockitoAnnotations.openMocks() may be unnecessary, at least suspicious.

Tasks

• Create an inspection that can detect JUnit4 runners and JUnit5 extensions on a test class, used in conjunction with MockitoAnnotations.openMocks().

Questions

• Check if and how superclasses need to be taken into account from both JUnit side and calling openMocks() side.

Bulk conversion of stubbings

Summary

Similarly to how #19 and #20 introduced bulk conversion of verifications, bulk conversion of stubbing could be implemented too.

Tasks

• Migrate the current Intention Actions into an Action group hidden behind a single intention, e.g. Convert stubbing(s) to...
• Bulk conversion would be available when one or more stubbing call chains are selected in the editor, and.

To be clarified

• The bulk conversion will probably make sense only when call chains with the same stubbing approach are selected.

Inspection to report void methods stubbed with a return value

Summary

Void methods don't have a return value, thus they cannot be stubbed to return something. This rule corresponds to this Mockito reporting.

Tasks

• Create an inspection to report stubs when void methods are stubbed with return values,

Example

doReturn(20).when(mock).voidMethod();

Support `*Return()` and `*Throw()` calls in MockedStatic stubbing

Summary

The simplification of consecutive *Return() and *Throw() calls is applicable to stubbing of MockedStatic as well.

Tasks

• Extend SimplifyConsecutiveReturnCallsInspection and SimplifyConsecutiveThrowCallsInspection to support MockedStatic.when() based call chains.

Inspection: report spying on mock

Summary

Mockito 5.4.0 throws an exception when a mock object is passed into Mockito.spy(). See

• mockito/mockito#2915
• mockito/mockito#2989

Tasks

• Implement an inspection that can detect Mockito.spy() being passed in a mock object. Constructs can be like:
  • Mockito.spy(Mockito.mock(Type.class))
  • Mockito.spy(<object annotated with @Mock>)

Questions

• Should it check spies passed into Mockito.spy() as well?

Omit default answer when converting @Mock field to Mockito.mock()

Summary

When the answer attribute is set to Answers.RETURNS_DEFAULTS in the @Mock annotation, omit it and don't add that value to neither the Mockito.mock(class, Answer) call nor the .defaultAnswer() call to the MockSettings.

Related intention: ConvertMockSpyFieldToCallIntention

Introduce settings to prefer setup methods when converting mock/spy fields to calls

Summary

Currently ConvertMockSpyFieldToCallIntention offers methods in the same class to create the converted call in. It offers all existing methods in current class. However, one might prefer to move these fields into setup methods if there is any.

Task

• Extend ConvertMockSpyFieldToCallIntention with support for setup methods.
• Distinguish between moving the whole field with identifier and initialization to the selected setup method, and keeping the field and initializing it in the setup method.
• Potential setup methods: various before methods in JUnit 4, JUnit 5 and TestNG.

Inspection to report stubs when non-void methods are stubbed to do nothing

Summary

Mockito doesn't allow non-void methods to be stubbed to do nothing. This rule corresponds to this Mockito reporting.

Tasks

• Create an inspection to report stubs where non-void methods are stubbed to do nothing.

Support `strictness` for mock creation settings

Summary

mockito/mockito#2648 and mockito/mockito#2650 introduced the strictness attribute both for MockSettings and the @Mock annotation.

Tasks

• Add support for this attribute.
  • Update @Mock to Mockito.mock() and vice versa conversion.
  • Add inspections if feasible: org.mockito.internal.exceptions.Reporter#strictnessDoesNotAcceptNullParameter

Handle generic-inferred `ArgumentCaptor.captor()` method

See mockito/mockito#3144

Inspection to report `toString()` in verification

Summary

Based on Mockito error handling, Mockito cannot verify the toString() method.

Tasks

• Create an inspection that can report verification of toString()
• Include all approaches from Mockito, BDDMockito and InOrder
• Check if/how MockedStatic is affected in this case.

Support for `withoutAnnotations` attribute on `@Mock`

Summary

mockito/mockito#2965 adds the withoutAnnotations configuration to the @Mock annotation.

Tasks

• Add support for the withoutAnnotations attribute.
  • See also MockSettings.withoutAnnotations().

Intentions to move mock()/spy() calls to @Mock/@Spy fields and vice versa

In scope

• Mockito.mock() -> @Mock conversion.
  Naming logic (mock call to field):

Call	Field
MockObject mockName = mock(...);	@Mock MockObject mockName;
MockObject mockName = mock(..., "customName");	@Mock MockObject mockName;
mock(...) //with no variable assignment	@Mock MockObject mockObject; //name of the type
mock(..., "customName")	@Mock MockObject customName;

• Mockito.spy() -> @Spy conversion.
• Take into account various MockSettings.
• When converting calls to fields, let users choose the target class if there are multiple parent classes.
• When converting fields to calls
  • if there is only one method, move the field to that method
  • if there are multiple methods, let users choose the target method
• Naming logic (field to call):
  • in case of field to call conversion, it is always a local variable assignment created. The local variable gets the name of the field.

To be clarified

• Collect all the mappings between MockSettings and the annotation attributes.

Resources

Notes

This may be extended to ArgumentCaptors and @Captor fields in the future.

Extend inspection to report calls on MockedStatic.reset()

Summary

CallOnMockitoResetInspection is already in place to report Mockito.reset() calls. It should be extended with MockedStatic.reset().

Add quick fix(es) for the checked exception inspection

Summary

ThrowsCheckedExceptionStubbingInspection reports checked exceptions not defined in the stubbed method's throws clause, but doesn't provide a quick fix.

Tasks

• Create a quick fix that would add the selected exception to the stubbed method's throws clause.
• To be decided: create a quick fix that would remove the argument from the stubbing.

Intention action to convert arguments of `*Throw()` stubbing calls

Summary

*Throw() stubbing methods have Class and Throwable parameterized overloads, so it is natural that conversion could happen between them.

Tasks

• Create an intention action to convert arguments of *Throw() calls from PsiNewExpressions to PsiClassObjectAccessExpressions and vice versa.
• Methods in scope
  • Mockito.when().thenThrow()
  • BDDMockito.given().willThrow()
  • Mockito.doThrow().when()
  • Mockito.do*().doThrow().when()
  • BDDMockito.willl*).willThrow().given()
  • BDDMockito.willThrow().given()
• The intention would be available when either all arguments are PsiNewExpressions, or all are PsiClassObjectAccessExpressions.
  • PsiNewExpressions called with non-default constructor would make the intention not available.

Add support for mockMaker in `@Spy`

Summary

mockito/mockito#2740 aims to expand on mockito/mockito#2626 for the @Spy annotation.

#43 must be implemented prior to this.

Tasks

• Depending on how Mockito implements this logic, @Spy annotation related features may or may not have to be updated/implemented.

Add support for the @DoNotMock annotation

The @DoNotMock annotation is introduced in Mockito 4.1.0.

In scope

• Create inspection to report attempts to mock such types.
• Cover any type whose name ends with org.mockito.DoNotMock.
• Check the whole type hierarchy for do not mock types.
• Implement for the @Mock annotation and Mockito.mock() calls too.
• In the inspection message, include the reason specified in the @DoNotMock annotation, if it has a reason attribute at all. (This could happen in case of custom annotations.)
  • If there are multiple @DoNotMock types in a type hierarchy, take the highest one with a non-blank reason and include that reason in the message.

Questions

• Is this applicable to spying as well? (Probably yes.)
• What severity level to use: warning or error?

Resources

• https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/DoNotMock.html
• mockito/mockito@b740f99

Intentions to switch between stubbing approaches

Summary

Mockito has different workflows for stubbing behaviour, Mockito.when().thenThrow(), BDDMockito.given().willThrow(), etc. To ease conversion between them, an intention could be created.

Tasks

• Create an intention action to convert between the different stubbing approaches:
  • Mockito.when().then*()
  • BDDMockito.given().will*()
  • Mockito.do*().when()
  • BDDMockito.will*().given()

To be clarified

• When would the intention be available?
• On what element would it be available?
• Map method names to each other between the approaches.
• Should it be N separate intentions, or a single one with multiple sub-options?

Nice to have

• As an expanded feature (maybe even in a separate GitHub issue) could be a project-wide migration tool/option/action/etc. to convert all Mockito... based approaches to BDDMockito... based ones or the other way around.
  • Conversion of stubbing of certain logic should have to be taken into account, i.e. when Mockito.do*() is preferred over Mockito.when().then*().

Autocomplete parameters of reset(), verifyZeroInteractions(), etc. with mock/spy variables

Summary

Since Mockito.reset(), Mockito.verifyZeroInteractions() and similar methods take mock and spy objects as arguments, using autocomplete, it would restrict the list of objects that are suggested to users to only valid instances.

• Implement a logic that can identify mock and spy objects from the references of fields and variables.
  • This includes objects created via the @Mock and @Spy annotations, as well as Mockito.mock() and Mockito.spy().
• Create completion contributor for mock and spy variables
  • For variables that cannot be determined whether they are mocks/spies, they should not be included in the autocomplete list.
  • To optimize performance, the search for such objects should be limited to the current test class and its parent classes.
    • If possible it could even further be limited in the class hierarchy until certain base test classes are reached that certainly do not contain any such object definitions/declarations.

To be clarified

• Collect the list of methods to bind the completion contributor to.

Extend inspection to simplify MockedStatic.verify(V, times(1)) calls

Summary

Extend TimesVerificationModeInspection to report MockedStatic.verify(V, times(1)) calls, and offer a quick fix to simplify it to MockedStatic.verify(V)

Inspection for consecutive *Return() and *Throw() calls

Summary

There are multiple overloads for the *Return() and *Throw() methods, both single- and multi-parameter variants.
In case they are called multiple times, consecutively and respectively, they may be merged into one such call.

Similarly if multiple such calls are made with the same parameter(s) (or rather the subsequent call's first parameter is the same as the previous one's last one), they can be deduplicated.

Tasks

• Create an inspection to deduplicate/simplify *Return()and*Throw()` calls.
• Constructs in scope (parts between [ and ] are optional sections)
  • throw
    • Mockito.when().thenThrow()[.thenThrow()]
    • BDDMockito.given().willThrow()[.willThrow()]
    • Mockito.doThrow()[.doThrow()].when()
    • BDDMockito.willThrow()[.willThrow()].given()
  • return
    • Mockito.when().thenReturn()[.thenReturn()]
    • BDDMockito.given().willReturn()[.willReturn()]
    • Mockito.doReturn()[.doReturn()].when()
    • BDDMockito.willReturn()[.willReturn()].given()
• Combinations of *Throw calls:
  • Class + Class
  • Class + Class... (and vice versa)
  • Throwable... + Throwable...
  • Class + Throwable... or Class... + Throwable... (if each Throwable is a PsiNewExpression for a default constructor)
• Combinations of *Return calls:
  • Every one of them
• Handle multiple consecutive calls in the same call chain, e.g. doReturn().doReturn().doThrow().doReturn().doReturn().

To be clarified

• How to handle when there is no parameter passed to *Throw()? Exclude it or replace it with something?

Add support for mockMaker in `@Mock` and MockSettings

Summary

In Mockito 4.8.0 mockito/mockito#2626 introduced the mockMaker attribute in the @Mock annotation, and the mockMaker(String mockMaker) method in MockSettings.

Tasks

• Extend @Mock to field and vice versa conversion to support this setting.

To be clarified

• See if the newly introduced MockMakers class (see related PR) can be utilized in some way.

Intention action to convert InOrder verification to Mockito.verify() and BDDMockito.then() calls

Summary

Since conversion of single InOrder.verify() calls is implemented in #22, this issue would cover the bulk conversion aspect of InOder verifications.

Tasks

• Create an intention action to convert selected InOrder verifications to Mockito.verify() and BDDMockito.then() calls.
• The intention would be available:
  • when one or more InOrder.verify() call chains are selected,
  • with the same availability as other from-InOrder conversion intentions are available

Inspetion: report non-empty reified argument of Mockito.mock()

Summary

Mockito 4.9.0 introduced a new set of mock() methods to be able to create mocks and spies without passing in a class object. However the reified argument of these methods is simply a trick to achieve this feat, it must not be passed in any value.

Tasks

• Create an inspection to report null and non-empty values passed into the mock(*, T...) set of methods.

Resources

• https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mockito.html#mock_without_class

Suggest only mock fields/variables for argument values to reset(), etc.

Summary

There are a handful of methods that accept mock/spy objects, like reset() and verifyZeroInteractions(). To ease the completion of such arguments, mock/spy fields/variables could be suggested.

Tasks

• Identify methods that accept mock/spy objects only.
• Implement a logic to identify mock and spy fields
  • These are primarily @Mock, @Spy and @InjectMocks annotated fields.
    • Such fields should probably be identified in super classes as well.
  • Variables created via Mockito.mock(), Mockito.spy() or similar mechanism are not so straigthforward to identify, especially if super classes are taken into account.
• Implement code completion
  • this would either be a completion contributor,
  • or, if the platform supports such feature, some kind of filter for the annotated fields, to apply to a code completion results list.

Scope

• As the first iteration of this feature, the code completion would include only the properly annotated fields.

Inspection to report InOrder variables with a single verification call

Summary

InOrder variables with only a single verification call can be a result of multiple things:

• the user forgot to add further verification calls
• a verification started out as an InOrder but was forgotten to be converted to simple verification when a change of mind happened
• etc.

Tasks

• Create an inspect that would report InOrder variables with only a single verification call on them.
• The intention/converter introduced in #20 could act as a "quick fix", without actually assigning it as a quick fix.

Inlay hint for @InjectMock annotated fields to show injected mocks

Tasks

• Implement an inlay hint for @InjectMocks fields with the list of @Mock and @Spy fields that can be/are injected.
• The inlay should be a block type one.
• This would involve bringing in Kotlin support as well into the plugin.

To be clarified

• Look into the logic of how mocks are injected and based on what criteria.
• "Constructor injection; the biggest constructor is chosen"
  • What happens if there are multiple constructors with overlapping or completely different parameter list, with the same number of parameters?
• com.intellij.codeInsight.daemon.ImplicitUsageProvider might be useful to disable highlighting @Mock and @Spy fields as unused if they are used only as injected mocks in @InjectMocks fields.

Resources

• InjectMocks javadoc

Inspection to report mock objects not used in InOrder

Summary

When one configures mock objects in Mockito.inOrder(), each of those mock objects are expected to be used in at least one verification.

Tasks

• Create an inspect that would report mocks objects within Mockito.inOrder() that are not used in any verification of that InOrder.

Inspection: explicit initialization of `@Mock` and `@InjectMocks` annotated fields

Summary

@Mock and @InjectMocks annotated fields are designed to not be created via explicit instantiation, instead via MockitoAnnotations.openMocks(), dedicated JUnit runners/extensions, etc.

Tasks

• Create an inspection that would search for the usages of @Mock and @InjectMocks annotated fields in the current test class and its subclasses, and report an issue when those objects are explicitly assigned a value in any way.
  • Use ReferencesSearch for that.
• The highlighting should happen on the mock field's name.

Inspection: Report and convert `MockSettings` based mock creations to their simpler variants

Summary

The javadoc of some methods in org.mockito.MockSettings states that they have simpler versions of creating and configuring the mock:

• spiedInstance

  Foo foo = spy(fooInstance);
  //does exactly the same as:
  Foo foo = mock(Foo.class, withSettings().spiedInstance(fooInstance));

• name

  Foo foo = mock(Foo.class, "foo");
  //does exactly the same as:
  Foo foo = mock(Foo.class, withSettings().name("foo"));

• defaultAnswer

  Foo mockTwo = mock(Foo.class, new YourOwnAnswer());
  //does exactly the same as:
  Foo mockTwo = mock(Foo.class, withSettings().defaultAnswer(new YourOwnAnswer()));

Tasks

• Create an inspection that can report the withSettings() based mock creation.
• Provide a quick fix to convert them to their simpler variants.

Inspection to enforce Mockito/BDDMockito based stubbing and verification

Summary

Projects might use various conventions that they want to enforce to create a unified code base.
One such element may be the approach they use for stubbing and verifying mocks.

Tasks

• Create an inspection that, based on user preferences, reports either Mockito or BDDMockito based stubbing and verification.

• The inspection options panel would provide the option to select which one the user would want to enforce.

• By default the inspection would be disabled, so that no enforcement would happen.

• Quick fixes would not be included as conversion intentions will be available in #13 and #15.

• Highlighting level will probably be ERROR.

• The intentions in #13 and #15 would have an additional criteria for availability, so that they have to match the stubbing/verification that this inspection enforces.

• To integrate this inspection into #13 and #15, it might be the current active inspection profile that it would have to retrieve the status of the inspection.

Intention actions to convert between InOrder.verify and MockedStatic.verify

Summary

MockedStatic objects can be verified with InOrder.verify too.

Tasks

• Create an intention action that can convert one or more (in single and bulk mode) MockedStatic.verify() calls to InOrder.verify() ones.
  • In this case a new InOrder local variable will be created.
• Create an intention action that can convert one or more (in single and bulk mode) InOrder.verify() calls to MockedStatic.verify() ones.
  • The InOrder variable will be omitted.

TBD

• Take into account UnusedOrUnconfiguredMockInInOrderVerificationInspection reporting the mock not used in InOrder. For now, it might be just not reported for class object access expressions.
• Check how any of these intentions can fit with the current conversion intentions.

Inspection to report stubonly mocks passed into verifications

Summary

Based on Mockito reporting, stub-only mocks are not allowed to be used in verifications, and the test execution fails accordingly.

Tasks

• Create an inspection that identifies that objects passed into verifications are definitely mocks and they are configured as stub-only, and reports them.
• Take into account the @Mock annotation and MockSettings as well.

Out of scope

• Spied objects

Create developer documentation

Tasks

• Create dev docs, so that anyone who'd like to contribute to this project can have a better understanding of the project structure.

Inspection: mocked type differs from spied/delegated object's type

Summary

org.mockito.internal.exceptions.Reporter incorporates two exception handlers:

• mockedTypeIsInconsistentWithSpiedInstanceType()
• mockedTypeIsInconsistentWithDelegatedInstanceType()

These two are valid candidates for an inspection.

Tasks

• Create one or two inspections for these exception handlers.

Notes for mockedTypeIsInconsistentWithSpiedInstanceType()

To correctly validate mismatches on the provided types, the inspection would need access to hte runtime type of the spied instance, otherwise it is not possible (except in the case of constructor calls) to determine the actual type the spied instance would be.

Some examples:

//Correct spying
var spyCorrect = mock(ArrayList.class, withSettings().spiedInstance(new ArrayList()));
var spyCorrect2 = mock(ArrayList.class, withSettings().spiedInstance(new ArrayList<>()));
//Generics doesn't seem to matter
var spyGenerics = mock(new ArrayList<String>().getClass(), withSettings().spiedInstance(new ArrayList<Boolean>()));

//Incorrect - types don't match
var spyIncorrect = mock(List.class, withSettings().spiedInstance(new ArrayList<>()));
var spyIncorrect2 = mock(List.class, withSettings().spiedInstance(new HashSet<>()));
var spyIncorrect3 = mock(List.class, withSettings().spiedInstance(List.of())); //List vs. ListN

//This is why the runtime type is needed. Although, the mock type and
// the expression type of createSpiedInstance() are different, at runtime they would both have the type ArrayList,
// which is a correct construct.
var spyRuntime = mock(ArrayList.class, withSettings().spiedInstance(createSpiedInstance()));

private List<?> createSpiedInstance() {
  return new ArrayList<>();
}

Notes for mockedTypeIsInconsistentWithDelegatedInstanceType()

The MockSettings.delegatedInstance() methods that is mentioned in error message of org.mockito.internal.exceptions.Reporter#mockedTypeIsInconsistentWithDelegatedInstanceType seems that it no longer exists, thus it won't be covered by an inspection.

Project-wide migration tool

Summary

To ease migration of a whole project between different conventions, a custom menu action should be created.
Maybe the best example of such migration is the Mockito vs. BDDMockito based stubbing and verification.

Tasks

• Create a custom action in one of the menus at the top of the IDE.
• The action should display a dialog with some options to migrate from Mockito to BDDMockito or vice versa.
• Upon clicking OK, the action would probably perform a ReferencesSearch to collect everything that has to be converted.
• During conversion a status bar would be displayed (its status would probably be calculated from the number of elements returned by the ReferencesSearch), and it wouldn't let the user perform any other action, until the migration is complete.

To be clarified

• In what menu the action should be placed?
• What exact UI options should be included in the dialog?

Improve UI of target method selection list during mock conversion

Summary

The list of methods in the current class are displayed when converting mock fields to mock creation methods, but it has a lot of noise: class name, package name, and it is difficult to distinguish the type of methods they are.

Tasks

• Remove the noise from the list items.
• Modify icons to distinguish between before hooks, test methods, and the rest of the methods.

Intention to switch between verification approaches

Summary

Mockito has different workflows for verifying behaviour, Mockito.verify(), BDDMockito.then(). To ease conversion between them, an intention could be created.

Tasks

• Create an intention action to convert between the different stubbing approaches:
  • Mockito.verify()
  • BDDMockito.then()

To be clarified

• Map method names to each other between the approaches.

Inspection to report mocks that could be stub-only

Summary

Stub-only mocks are not allowed to be used in verifications. Vice versa, if a mock is not used in any verification, it might be stub-only.

Tasks

• Create an inspection that can report mocks that are not used in any verification and are not configured as stub-only.

Out of scope

• Spies

Add support for InOrder in verification conversion intention actions

Summary

Conversion of InOrder specific verification calls between InOrder.verify() and BDDMockito.then().should(InOrder) are not yet supported.

Tasks

• Create an intention action to convert InOrder.verify() calls to BDDMockito.then().should(InOrder).
  • The intention would be available on InOrder.verify().
• Create an intention action to convert BDDMockito.then().should(InOrder) calls to InOrder.verify().
  • The intention would be available on BDDMockito.then().

To be decided

• Alternatively, ConvertBDDMockitoThenToMockitoVerifyIntention and ConvertMockitoVerifyToBDDMockitoThenIntention might be extended with this logic, instead of creating two separate intentions.

Investigate and add support for `assertArg()`

Summary

mockito/mockito#2285 adds assertion arguments.

Tasks

• Investigate if there is anything that might be added for supporting this new Mockito feature.