boshrin / or-match Goto Github PK
View Code? Open in Web Editor NEWPostgres based identity match engine
License: BSD 3-Clause "New" or "Revised" License
Postgres based identity match engine
License: BSD 3-Clause "New" or "Revised" License
If a required attribute is missing, the request would be considered invalid.
The one gigantic README.md is hard to navigate and maintain. Move the documentation to a wiki somewhere, probably depending on where the project ends up.
Also review second person vs third person style. (The README switches about halfway through from "you" to "one".)
Current proposal: Reference ID Match (Service)
It might make sense to allow substr (#16) and token (#23) matches to be canonical. One way to do this would be to have a hierarchy of exact/token/substr, and whichever (if any) is first defined for an attribute would be used in determining canonical matches. Another way would be to allow canonical matches to specify search types (like potential matches), though it's not clear that it makes sense to have different canonical rules for the same attribute with these search types.
Add a utility to generate matchgrid based on the current configuration.
Currently, the match engine will allow any value for SOR, assuming the client is authorized for it. So if a client is authorized for '*', it might send 'sis', 'SIS', or 'student' all meaning the same thing. Perhaps require SORs to be defined in the [sors] section of config.in. This could then allow 'sis' while rejecting the other 2 as undefined.
Currently, reference_id is generated as a UUID. Offer a configurable option to generate it from a sequence (which can be cast to a varchar so the matchgrid column doesn't need to change).
It may make sense to use SELECT FOR UPDATE to handle a read/write request to avoid concurrency issues.
Review API documentation for missing features/requirements and implement (or create tickets) as necessary.
Create a utility of some form to define attributes as specified in config.ini.
This would handle (eg) John vs Jack.
See postgres documentation on Dictionaries (§12.6).
HR record in Mixed Case does not match STU record in ALL CAPS. (See notes for details.)
This should already be handled by conversion to an update attributes request, but apparently isn't.
Implement Update Match Attributes Request.
searchDatabase() does not ignore records where reference_id is null (ie: pending resolution).
Implement a tokenization search type, that would (eg) treat "anna marie" and "anna" as the same by ignoring everything from the space onwards. This might be doable with SQL's substring(string from pattern for escape).
Especially when using a sequence, check for an existing record with a newly assigned reference ID, and if one exists try again (up until some reasonable max). This will simplify bulk loading existing data that may exist in a conflicting range.
This may or may not be a good/useful idea, but is supported by more databases.
Individual files should reference the BSD license (as mentioned in the README and LICENSE).
These two attributes are mandatory, and so should always be defined. (Perhaps inserted into config during parsing?)
Should "hr" and "HR" be considered equivalent?
Support notification to an email address (per SOR?) when a new potential match is recorded (202, not 300).
Like the exact search type, but only operates on a substring (eg: first 5 characters, last 5, second through fifth, etc).
If an attribute consists of only blanks or zeroes (and possibly non-alphanumeric characters according to the configuration), skip it for matching purposes. eg: lastname="", dob="0000-00-00", ssn="000000000".
Perhaps have a configuration option to disable this on a per-attribute basis.
Currently invalidates operates on a simple string comparison. Should it honor modifiers such as casesensitive and alphanum?
Offer an option to return the reference ID as an identifier in the API response, with the identifier type configurable.
Add support for none / connections / logic / trace (sql)
Allow authentication via $REMOTE_USER. ie: ignore matchauth:apikey, but still use matchauth for authz.
Add the ability for an attribute to match against SORID. Something like
[attribute:employeeid]
; Match this attribute against SORID where SOR is "HR"
search['sor'] = 'HR'
Almost all of config.ini should be moved to the database (ie: database driven configuration).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.