Dynamic Json filtering library for Java/Spring REST APIs This lib will be useful in a situation when we want to filter (potentially large and complex) json object using kind of WYSIWYG filtering pattern.
E.g. we have following json and want to remove stats.agi
field:
{
"name": "orange",
"stats": {
"str": 8,
"agi": 10
}
}
Then filter would be like this:
{
"name": 1,
"stats": {
"str": 1
}
}
Removing name
while keeping stats
object intact will look like this:
{
"stats": {}
}
(more examples in javadoc of JsonFilteringService)
Let's assume we have a Spring REST API with some heavy endpoint, and we want to remove particular deeply-nested fields from the response, to make it lighter and more easily parsable for clients.
Sometimes we can change our response structure, prevent population of some fields etc. But this is not always an option.
Another possible solution would be to wrap our endpoint in a GraphQL query and let user chose desired fields for each request. But this approach also has its downsides.
This library is meant to be an "easy to plug in" configurable alternative that allows both static (based on pre-defined file-based filter) and dynamic (based on user-passed header) filtering of json response for Spring HTTP endpoints. It supports setting different filters per-clientId for the same endpoint and usage of both regular sync and async endpoints.
Of course, filtering a complete json response is not as resource-efficient as just not serializing unwanted fields at the first place, but Jsonsiever is based on fast Jackson Streaming API and resource usage drawback should be generally acceptable.
Lib is ready to use by just plugging it into SpringBoot 3.X / Java 17 application. Also, its code might be relatively easily adapted to SpringBoot 2.X / Java 8 or plain Spring / Java Servlet app.
It is also possible to plug the lib into plain Java 17 application and use it for manual json filtering (see below How to use Jsonsiever lib with plain Java application). In this case one would just use core JsonFilteringService
class to filter json in other scenarios, besides processing REST responses. This class (as well as whole io.github.liquidcake.jsonsiever.core
package it is part of) is ready to be compiled with Java 8.
Details on filters structure are in javadoc for JsonFilteringService class
Details on settings file are in javadoc for JsonFilteringSettings class
There is a simple demo SpringBoot application inside this repository - feel free to build it and play around - jsonsiever/demo_app_spring
Maven:
<dependency>
<groupId>io.github.liquidcake</groupId>
<artifactId>jsonsiever</artifactId>
<version>1.0.0</version>
</dependency>
Gradle:
implementation 'io.github.liquidcake:jsonsiever:1.0.0'
@Configuration
@Import(JsonFilteringConfig.class) //JSONSIEVER - import Jsonsiever config
@EnableWebMvc
public class AppConfig {
//JSONSIEVER - declare Jsonsiever servlet filter
@Bean
public FilterRegistrationBean<JsonResponseBodyFilter> filterRegistrationBean(
JsonResponseFilterApplier jsonResponseFilterApplier
) {
FilterRegistrationBean<JsonResponseBodyFilter> registrationBean = new FilterRegistrationBean<>();
registrationBean.setFilter(
new JsonResponseBodyFilter(jsonResponseFilterApplier)
);
registrationBean.addUrlPatterns("/*"); //JSONSIEVER - you may want to allow filter only for some base path
registrationBean.setOrder(99); //JSONSIEVER - you may want to adjust filter order but not mandatory
return registrationBean;
}
//JSONSIEVER - you may want to provide your own implementation for this cache
//(used to store header-based filters, if you are going to use them)
//See javadoc for SimpleInMemoryDynamicJsonFiltersCacheWrapper)
@Bean
public DynamicJsonFiltersCacheWrapper simpleInMemoryDynamicJsonFiltersCacheWrapper() {
return new SimpleInMemoryDynamicJsonFiltersCacheWrapper();
}
}
Create file json-filtering-settings.yml
inside classpath (e.g. resources
folder) and configure your endpoints (details in javadoc for JsonFilteringSettings)
Example from demo_app_spring
:
filterHeaderName: "X-json-filter-pattern"
clientIdHeaderName: "X-client-id"
endpoints:
- path: "GET_/get-cats"
regexpPath: false
headerPatternAllowed: true
filePatternAllowed: true
filePatternPathPerClient:
our-mobile-app: "/json-filters/GET_get-cats/our-mobile-app.json"
default: "/json-filters/GET_get-cats/default.json"
- path: "POST_/activate-cat/.+"
regexpPath: true
headerPatternAllowed: true
filePatternAllowed: true
filePatternPathPerClient:
our-mobile-app: "/json-filters/POST_activate-cat/our-mobile-app.json"
default: "/json-filters/POST_activate-cat/default.json"
Create dir json-filters
inside classpath (e.g. resources
folder) and inside it - directories for each configured endpoint.
Inside each per-endpoint directory you may have 1 or more per-client filter files.
E.g. in above example config - we have 2 endpoints, each with default filter and with separate filter for client-id our-mobile-app
which can be passed by client in HTTP header.
Example from demo_app_spring
has the following structure:
json-filters/
GET_get-cats/
default.json
our-mobile-app.json
POST_activate-cat/
default.json
our-mobile-app.json
Content of filter files is WYSIWYG json filtering pattern (mask). All details on filter pattern composition are available inside javadoc for JsonFilteringService
.
Simple example from demo_app_spring
file demo_app_spring/src/main/resources/json-filters/POST_activate-cat/our-mobile-app.json
:
{
"stats": {
"agi": 1
}
}
(this filters out stats.str
field that otherwise would be present in response)
After making above configuration - requests to configured endpoints will be filtered. Since we have both header-based and file-based filtering enabled, priority will be following:
- if user provides
X-json-filter-pattern
header with valid filter pattern json - this filter is applied. - else, if user provides
X-client-id
header with a value we have filters for (e.g.our-mobile-app
) - client-specific filter is applied. - else - default client filter file will be used (since we have it in this example).
- in case we didn't have nor header value neither any of filter files present for this endpoint, or in case any processing error happened - response would be returned as-is, without any filtering applied.
Example CURL requests (with both headers enabled):
curl --location '127.0.0.1:8080/get-cats' \
--header 'x-client-id: our-mobile-app' \
--header 'X-json-filter-pattern: [{"name": 1}]'
curl --location --request POST '127.0.0.1:8080/activate-cat/orange?fakeParam=blabla' \
--header 'x-client-id: our-mobile-app' \
--header 'X-json-filter-pattern: {"name": 1}'
Without SpringBoot, lib would just require Jackson
and Slf4j
dependencies added explicitly to your application and that's it - JsonFilteringService
will be usable directly from your code.
Example Gradle dependencies:
dependencies {
implementation 'io.github.liquidcake:jsonsiever:1.0.0'
implementation 'com.fasterxml.jackson.core:jackson-databind:2.17.1'
implementation 'org.slf4j:slf4j-api:1.7.25'
}
Usage:
ObjectMapper objectMapper = new ObjectMapper();
JsonFilteringService filteringService = new JsonFilteringService(objectMapper.getFactory());
String jsonData = "{\"field1\": 123, \"obj\": {\"f1\": 123, \"f2\": 456}}";
JsonNode filterPattern = objectMapper.readTree("{\"obj\": {\"f1\": 123}}");
byte[] filteredJson = filteringService.filterJsonFields(jsonData.getBytes(), filterPattern);
Lib is compiled with Java 17 but core sources are Java 8 compatible.
You could just manually take a source for whole io.github.liquidcake.jsonsiever.core
package (just a few classes besides JsonFilteringService
) and compile it with Java 8, adding some version of Jackson and Slf4 as dependency.
Then use JsonFilteringService
the same way as in Java 17