commitd/gazetteer

Simple text markup API based on gazeteers with management UI

Committed Gazetteer

Committed's Gazetteer offers a simple API for performing text markup using a Gazetteer. The Gazetteer is often overlooked in favour of ML base NLP solutions but for high value targets a simple gazetteer can form an important part or a broader NLP extraction strategy. This service also has a user interface for configuring the Gazetteer so it can be done by none developers.

🚀 Quickstart

Use the pre-build docker container:

docker run -p 8080:8080 committed/gazetteer

You can configure the gazetteer using the UI on http://localhost:8080 and checkout the API at http://localhost:8080/swagger-ui/index.html

💻 Development

We use maven to build the project using:

mvn clean package

The server is written in java using Spring Boot and the UI is Typescript with React.

This will build the server, ui and Docker image.

For UI development, in src/main/app you will find the usual node package.json with scripts for, build, start, test and storybook as well as some other utility scripts. The scripts can be used while developing the UI, example below, but the main maven build will take care of building and packaging the UI into the jar. Not for full function during UI development you should also run the server.

yarn build

We use Storybook to develop and document the components, this is run in development using

yarn storybook

🤖 CI

Pull requests go through CI checks using Drone. Use version tags on master to deploy to Docker hub, release to maven central is currently manual, using:

mvn deploy -P release

©️ License

MIT - © Committed Software 2020 https://committed.io

gazetteer's Issues

Separate type name from id

This would allow renaming of the type and the multi config setups with the same type.

Need to be able to cope with long lists

Added a lot of examples - works fine but struggles to show the list of values

Either needs to be virtualized or paginaged

Allow types to be any word

Currently gets UPPERCASED I don't see why it can't be lower or mixed etc

Improve type view

paginate and allow sorting by value and count.

Backup query may be timing out

Large number of example used in ARGA causing backup to fail - fails at 30s every time so likely a timeout issue.

Add github actions

Allow type to specify ignore case and whole word only

case is currently taken into account and we only match whole words but this could be done on a configuration basis

Add dev container

Add new release process

What problem do you want to solve?

What do you think is the correct solution to this problem?

Add file based upload of values

could support csv, json or one line per value options.

Need to recreate the trie after a backup restore