Comments (2)
aaccomazzi
commented on July 28, 2024
For extraction from XML sources, this issue is fixed with commit c18f29b
We should still investigate whether this is also an issue with PDF extraction and then decide how to tackle it.
from adsfulltext.
aaccomazzi
commented on July 28, 2024
To clarify: the SOLR ingestion problem had to do with a limitation in the word length (blobs of characters longer than 32k caused a failure), not text length.
Commit ee36740 modifies the PDF extraction code so that newlines are kept, greatly lowering the possibility that huge strings of nonsense will be generated.
from adsfulltext.
Related Issues (20)
- Do not remove facilities from acknowledgements HOT 1
- XML articles with valid empty body should end up with an empty body field in solr
- False positive OCR files for "NASA" queries
- Invalid empty body resulting from <!-- body endbody --> syntax
- Handle facility tags without xlink:href
- HTML extractor cannot decode certain characters
- html5lib parser failing due to less than symbol in XML files
- Facilities appear outside of acknowledgements in XML
- Fulltext query for "Mars" and "HiRISE" returns less papers than before
- Extract useful data from tables
- Some parsers are only extracting the acknowledgements from XML files
- Facilities not properly tokenized
- fulltext contains serialized json in the body
- Use database instead of filesystem HOT 2
- Improve use of file close
- Support encodings other than utf-8
- Multi-file parse HOT 5
- Update parser for Nature XML HOT 2
- Fix missing appendix text HOT 2
- Document Similarity
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from adsfulltext.