Comments (3)
Hi @nishant-nayak. There isn't an API to upload bulk data, but there are APIs to insert and manage individual entries and definitions. What is your usecase? To integrate with a system to continuously add data (the entry/definition APIs are sufficient for that) or to continuously add large datasets (which is typically unlikely).
An API for CSV bulk load isn't ideal because:
- Bulk loading is an expensive process and could take arbitrarily long time to complete. So an API request can't wait for the process to finish (broken request may cause partial import), instead, it'll have to queue the file for import.
- This'll either need the app to have filesystem access or an in-memory blob to store the bulk data.
- Implementing this is complex.
- Now to check the status of a queued file, there'll have to be a status check API that would have to be polled.
Dictionaries typically do not get frequent bulk loads of data. It's almost always one off, or rare, and for that, CLI is sufficient.
from dictpress.
Hi @knadh , the use case that we were looking at is to add large datasets to our dictionary. As you mentioned, it won't be a frequent operation so it should be doable with just the CLI. We have a similar upload feature on another application where we do implement the in-memory blob and status of a queued file using Django, but I believe with the glossary it may not be necessary.
Thanks for the clarification!
from dictpress.
I've implemented an asynchronous CSV->DB import system in another Go project where frequent bulk uploads are a necessity, but had decided not to bring that complexity into dictpress because of the reasons mentioned above; dictionaries seldom need continuous bulk imports.
Please do let me know if you need any help using dictpress in the glossary project. Excited to see it!
from dictpress.
Related Issues (17)
- Need a strategy to generate guid for entries HOT 2
- Need a way to organize new code HOT 2
- WIP: Admin dashboard HOT 4
- import of dictioniary HOT 4
- More features support request HOT 2
- Retrieve suggestions using longer strings HOT 1
- Add Docker support HOT 3
- [Feature] Allow using Wikilexemes as database HOT 3
- Adding relation does not work
- data API HOT 1
- Allow searching by just tags without having to provide a search query
- Make the search form more clear to understand HOT 1
- Defintion entries notes', tags' are not imported from CSV HOT 3
- Add support for meta in CSV importer
- add unit/integration test HOT 2
- Creating a new entry without adding content in Meta gives an error
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dictpress.