Fingerprint processor

[benthosdev]

Fingerprint processor

[Jeffail]

Hey @rucciva, if I create a general purpose hash processor you would be able to do the following:

processors:
  - type: json
    json:
      operator: set
      path: id
      value: ${!json_field:some.field}-${!json_field:another.field}
  - type: process_field
    process_field:
      path: id
      processors:
        - type: hash
          hash:
            algorithm: sha256

And then you could use the id as your ES id, would that work for you?

[rucciva]

Yes, that seems great. I hope murmur 3 will be included in the hash algorithm. Looking forward to it

[Jeffail]

101cfea, is there a specific reason that you wanted murmur 3? There's one or two libraries for it but they seem a little risky to include.

[Jeffail]

Released in https://github.com/Jeffail/benthos/releases/tag/v0.25.0, if you need murmur 3 support then feel free to open a new ticket.

[rucciva]

Hi @Jeffail , murmur3 is best fit for fingerprint case, since it has big enough space (128 bit) but quite fast. But if you thinks it's too risky then i think i could use the existing algorithm