knadh/dictpress

A stand-alone web server application for building and publishing full fledged dictionary websites and APIs for any language.

dictpress's Issues

data API

Hey

Is there any sort of API yet ?

A golang API so that golang can call would be useful.

A golang API would also allow a CTL and CLI later.

I think people use swagger around that to then gen the REST API for non golang clients / services.

My use case

This is part of a science project where we need to create specific terms in many languages.

Services need to interact with the dictionary for read, write and notifications of CRUD at the data level.

Services are holding the term refs in freetext files ( markdown, etc ) and also in their DB, so maybe i can hold the dictpress DB PK GUID, as that wont change over time.

If the API can also do change notification it would be good, so that Services can know when data changes, so then they can store the data on their side in their free text and DB, and then update it when they get a notification.

[Feature] Allow using Wikilexemes as database

Hi, In order to keep the data publicly accessible it may be a good idea to add Wikidata/Wikilexeme support, this way the terms will be retrieved with SPARQL and we can eliminate the database and increase collaboration.

Thanks.

Add support for meta in CSV importer

What I'm doing right now is put JSON in notes

-,അ,"അകം(1) അന്തരംഗം, മനസ്സ്‌",malayalam,"{""native"": ""അകം1 അന്തരംഗം മനസ്സ്‌"", ""en"": ""akam1 antharamgam manass‌"", ""ipa"": ""əkəm1 ən̪t̪əɾəmgəm mən̪əss"", ""iso"": ""akaṁ antaraṁgaṁ manass ""}",,,1-1,,

and then use Rails to put it as JSON into meta:

Entry.find_each do |entry|
  entry.update!(meta: JSON.parse(entry.notes))
rescue
  print("error #{entry.id}")
end

Better if dictpress does this directly

Make the search form more clear to understand

https://github.com/knadh/dictpress/blob/master/site/static/main.js#L16C1-L17C1

.replace(/[^a-z\u00E0-\u00FC\s]/ig, '')

This regex replace is unnecessary for a default example template. I was confused why the search was not working and ending up in a HTTP authentication required page.

The form didn't look like it had a JS backing. One more suggestion is to remove the name attribute from the search form so that it is more clear. Why need a name if the form submission is actually handled by JS.

Adding relation does not work

When I try to add a definition to an entry in the admin console, it creates a new entry for the definition, but it fails in creating the relation. An Alert pops up saying "Error Invalid IDs".

I added some print statements and found that the echo context in handleAddRelation does not have a param called toID. Instead, the relation ID is in a param called relID. This is strange since line 179 in init.go says /api/entries/:fromID/relations/:toID. The package may have a bug or something else is causing this.

Regardless, the easy way to fix it is to change line 251 of admin.go to toID, _ = strconv.Atoi(c.Param("relID"))

I am not sure which versions of the tools the project requires, but I am using Go 1.18.6 on Mac

Add Docker support

The current codebase requires the user to set up and link a Postgres instance manually. The time and effort required for this would cause many people not to give dictpress a try. Of course one could use Docker themselves, but providing a basic docker-compose file would still be useful and attractive, I believe.

Here's a basic Docker Compose support (just for the DB) that I added to try out dictpress for the first time:
https://github.com/nandedamana/dictpress/tree/add-docker-support

It also contains an option to auto-generate the postgres password env file and sync it with config.toml. However, I'm not satisfied with the changes, and that's why I'm not filing a PR now.

There are multiple things in the current state of the branch that might be frowned upon, including:

• The added docker support is partial (just for DB, not the server itself)
• Placeholder DB name and username changed to "postgres" in config.sample.toml
• Generation of pg.secret.env and its syncing with config.toml is written in the Go program itself, rather than in a shell script or something

Please share your thoughts.

More features support request

I want to support the following features for every word definition. How can I do that?

1. Synonym/Antonym of the word
2. Usage example in a sentence
3. Pronunciation audio clip (I have audio clips, I want it to play if the user clicks near the pronunciation word); where to store the audio clips, how to map, and all.

@knadh please help, if I can leverage existing columns or need to add.

Defintion entries notes', tags' are not imported from CSV

https://vimeo.com/884986954/8e3c7cd3ad?share=copy

Retrieve suggestions using longer strings

Thanks for building and open-sourcing this awesome tool !

I had a request:

Currently, the search query is used just to search a word.
It would be great if there was a feature to support search using text of arbitrary length (like a sentence), and return the list of all dictionary matches based on substrings (or words) in the text.

Example usecase: This would be helpful in cases where we need to search and retrieve the list of all glossary terms in a given sentence.

WIP: Admin dashboard

Context.

import of dictioniary

Is there any way how to import/convert existing dictionary to dictmaker without need to manually create sql file?

Allow searching by just tags without having to provide a search query

Currently it's not possible to search entries by just tags without providing a query param:

GET /api/entries/:fromLang/:toLang?tag=something

This is only possible currently:

GET /api/entries/:fromLang/:toLang/:searchQuery?tag=something

Searching by just tags without a query param is a useful feature.

Need a strategy to generate guid for entries

The sample.sql uses guid like MD5('apple-1'), etc. If we have a sorted list of words, we can predictably create guids like this.

But, when a random word is being inserted, how do we generate a guid?

One option is to make a fixed strategy for generating guid. Something like this:

# pseudocode
let newWord = 'apple'
let existingCount = count (*) from entries where word = 'apple'
let newCount = existingCount + 1
let guidKey = `newWord-${newCount}`
let guid = MD5(guidKey)

Another option is to drop the guid column if it is not important or if it can be generated from the other columns.

Related questions:

1. Do you have a link to a script which you use for inserting data into alar?
2. What's the purpose of guid?

Need a way to organize new code

Right now, queries.sql has all the SQL queries. These are used to prepare statements in search.go.

But search.go doesn't seem to be the right place to add new features related to add/update/delete/approve words.

So, if we create a manage.go, there is a problem that to initialize the queries, we will clash with search.go's initializations.

How do you propose that the queries be organized for new functionalities.

add unit/integration test

I got to know about the project from Nithin's tweet. So wanted to know more about the project. When going through the code base, I thought we could add some tests.

I added some tests here chinmayb#1 . Its just a couple of tests to begin with and contains a minor fix. We could add more in the upcoming commits/PRs. It brings up postgres as docker container during tests. Let me know your thoughts, I will raise the PR here if the approach seems fine, Thanks!

API Endpoint for uploading CSV file

Is there an API endpoint to POST a .csv file to insert it into the database? Currently the only documented methods are CLI based.
As part of the National Language Translation Mission, the team at AI4Bhārat is developing an Indic language glossary to help translators and annotators in the translation of domain-specific text. We are looking to use dictpress as an open-source solution for this task, and the API endpoint to upload a dataset would be a requirement for the same.