About parameter `full_search_strategy` in drain match method

[logpai]

About parameter `full_search_strategy` in drain match method

[davidohana]

What you suggest makes sense to me. However I checked also the original Drain code here: https://github.com/logpai/logparser/blob/e8d96cd4de1121c5d2b517982c6028cd06e643f1/logparser/Drain/Drain.py#L172 and the same max-param-count selection logic exists, so it's not a translation bug. I also read the part of the Drain paper https://jiemingzhu.github.io/pub/pjhe_icws2017.pdf on page 4 (step 4), but unfortunately, the text does not describe what happens in a case of equal similarity score.
I think best action now is to check with original Drain authors the reason for this logic, before making any change.

[davidohana]

Actually while writing an issue to the Drain team, I think I understand the reason. Follow this example:

For the two templates:

request type <*> result <*> took <*> <*>
request type create result <*> took <*> ms

and log message:

request type delete result OK took 0.21 sec

Equal tokens to the 1st template = 4, param count in first message = 4
Equal tokens to the 2nd template = 4, param count in first message = 2

Here the message has a bettter match with 1st template.

[davidohana]

Also, something in your first example does not make sense.
You provide IPPROTO_TCP fd: 100, errno: 100, option: 100, value: 100 with : in some tokens, but the templates does not contain :, e.g IPPROTO_TCP fd <NUM> errno <NUM> option <NUM> value <NUM>. So this is not a perfect match. and therefore match should fail.

[doudoublues]

while enbale include_param=True，the similarity 1st template is (4+4)/ 8 = 1.0, the sim of 2st template is (4+2) / 8 = 0.75, it will not go to the logical comparsion cur_sim == max_sim and param_count > max_param_count.

[doudoublues]

for this case IPPROTO_TCP fd: 100, errno: 100, option: 100, value: 100, we have set extra_delimiters = [':', '@', '&', '[', ']', '(', ')', ',', '=', '{', '}', ';', '!', '?'] which include : .

[davidohana]

But the template has 9 tokens, not 8 ?
Also, you want to match against the first template so isn't this the expected behavior?

[doudoublues]

But the template has 9 tokens, not 8 ? Also, you want to match against the first template so isn't this the expected behavior?

what I want to talk about is when two template has same similarity, the one with less wildcards will be match. But in this case, when we enable include_param=True, the similariteis of 1st and 2nd template is not equal, it will not go to the logical comparsion cur_sim == max_sim and param_count > max_param_count.

[davidohana]

include_param=True by default in match(), so the behavior is fine as I understand.
Closing this issue.

[doudoublues]

include_param=True by default in match(), so the behavior is fine as I understand. Closing this issue.

I know, So that in my cases,

IPPROTO_TCP fd: 100, errno: 100, option: 100, value: 100

two template similarity is 1.0, we should choose 1st template which has no wildcard, but drain it return the 2st template.

ID=2 : size=1 : IPPROTO_TCP fd <NUM> errno <NUM> option <NUM> value <*>

[davidohana]

#68 (comment)

This is an example where more params is the better choice.

[doudoublues]

Got it, but it didn't work on my case. Is it something wrong in its logic? In other words, I think when two template both get 1.0 similarity score, the template has the less wildcard will be match, isn't it? But drain doesn't work on this case.

[davidohana]

If you want, you can provide a sample code that reproduces the problem in a branch, and I will check.