Comments (7)
Great to see this being adopted in log4j! Nice work 👏
threadLocal performs the best, since every instance is stored in TLAB
TLAB stands for thread local allocation buffer which is something different. You might want to change -> TLAB
ThreadLocal
I would also explain the possibility of class loader leaks when using the threadLocal
strategy with application servers.
In the ECS JSON layout, MDC values are serialized on the top-level now, as supposed to nested under labels
and may contain non-string values.
/cc @xeraa FYI
from log4j2-logstash-layout.
Thanks for the thorough review @felixbarny, really appreciated!
TLAB stands for thread local allocation buffer which is something different. You might want to change
->TLAB
ThreadLocal
☑️
I would also explain the possibility of class loader leaks when using the
threadLocal
strategy with application servers.
☑️
In the ECS JSON layout, MDC values are serialized on the top-level now, as supposed to nested under
labels
☑️
and may contain non-string values.
I've checked out the sources of log4j2-ecs-layout-0.4.0
and there EcsLayout
contains the following lines:
private final TriConsumer<String, Object, StringBuilder> WRITE_MDC = new TriConsumer<String, Object, StringBuilder>() {
@Override
public void accept(final String key, final Object value, final StringBuilder stringBuilder) {
stringBuilder.append('\"');
JsonUtils.quoteAsString(key, stringBuilder);
stringBuilder.append("\":\"");
JsonUtils.quoteAsString(EcsJsonSerializer.toNullSafeString(String.valueOf(value)), stringBuilder);
stringBuilder.append("\",");
}
};
private void serializeAdditionalFieldsAndMDC(LogEvent event, StringBuilder builder) {
final int length = additionalFields.length;
if (!event.getContextData().isEmpty() || length > 0) {
// ...
event.getContextData().forEach(WRITE_MDC, builder);
}
}
Would you mind elaborating on how come an MDC value of type, say, int
gets emitted as a JSON number
rather than a text
?
from log4j2-logstash-layout.
I reviewed the documentation and cross checked it with our current setup of the LogstashLayout. The documentation is well written and understandable. I don't see a reason why we can't migrate from LogstashLayout to JsonTemplateLayout.
There is one feature of which I don't understand completely what its function is and that is the "Additonal event template fields". Why is it useful and when should I use it? Maybe that could be elaborated a bit more in the documentation.
Another question: is $${json:exception:stackTrace:text}
from LogstashLayout the same as an exception
resolver with stringified
enabled?
I do have one suggestion and that is to add output examples of the effects of the resolvers and such. That would make it easier to understand what they do. This might make the documentation more bloated however, so I can understand if you are reluctant.
from log4j2-logstash-layout.
Would you mind elaborating on how come an MDC value of type, say, int gets emitted as a JSON number rather than a text?
What I meant is that currently the MDCs are stringified in EcsLayout.json
, which is not needed when serializing at the top level:
from log4j2-logstash-layout.
Thanks so much for taking time to review the manual @mzeijen, really appreciated!
There is one feature of which I don't understand completely what its function is and that is the "Additonal event template fields". Why is it useful and when should I use it? Maybe that could be elaborated a bit more in the documentation.
☑️ Updated the manual.
Another question: is
$${json:exception:stackTrace:text}
from LogstashLayout the same as anexception
resolver withstringified
enabled?
Correct. This will do the trick:
{
"$resolver": "exception",
"field": "stackTrace",
"stringified": true
}
I do have one suggestion and that is to add output examples of the effects of the resolvers and such. That would make it easier to understand what they do. This might make the documentation more bloated however, so I can understand if you are reluctant.
Yes... I am indeed sitting on the fence about this one. I think I will first wait and see the reaction of the users. If there pops up many of such questions, I will add examples.
from log4j2-logstash-layout.
Would you mind elaborating on how come an MDC value of type, say, int gets emitted as a JSON number rather than a text?
What I meant is that currently the MDCs are stringified in
EcsLayout.json
, which is not needed when serializing at the top level:
Ah... Now I see your point. I'd rather stick to stringified
version to match the log4j2-ecs-layout
behavior. This feels like the option with the least surprise factor for users. If needed, they can override it via additional fields anyway.
from log4j2-logstash-layout.
Log4j 2.14.0 is released! 🥳 Thanks to everyone for the feedback on the design drafts of JsonTemplateLayout
shipped with this release!
from log4j2-logstash-layout.
Related Issues (20)
- BufferOverflowException get thrown for messages of a larger size HOT 7
- LogstashLayout with log4j2-elasticsearch appender HOT 1
- Introduce properties to globally set LogstashLayout.Builder defaults
- Nullpointer exceptions at LogstashLayoutSerializationContexts$StringTruncatingGeneratorDelegate HOT 1
- Any template ready for GCP? HOT 9
- Message lost if exceeding maxStringLength and emptyPropertyExclusionEnabled is true HOT 4
- Use ByteArrayOutputStream instead of ByteBufferOutputStream to avoid message loss HOT 8
- Stacktraces larger than maxStringLength are not truncated HOT 7
- Failed serializing JSON: BufferOverflowException HOT 10
- How to output ThreadContext (ex -MDC fields) HOT 1
- How exactly is MapMessage supposed to be handled? HOT 7
- Multiple template variables in single property lead to an exception HOT 2
- Fallback for json:map: fields for libraries HOT 1
- NoClassDefFoundError: logstash.layout.resolver.TemplateResolvers (initialization failure) HOT 1
- Remove max byte count cap? HOT 1
- Reference whole MapMessage - and not just keys HOT 13
- Compiling with JDK 8 results in java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer; HOT 4
- Marker lookup subsitutes literal string when marker not available HOT 1
- BufferOverflowException occurs intermittently when sending data via socket appender HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from log4j2-logstash-layout.