Comments (11)
I assume this indirectly caused by the TiCC::tempname() function which creates an unique named file.
The calling program may use this file/filename for some time, but at some point it is responsible for removing it.
Frog doesn't do this, nor in Server mode (around line 874 of FrogAPI.cxx ) nor in the Frogtostring() function which is made specially for python-frog. , (around line 1149, needs some refactoring to delete the file before the return)
from frog.
Addition: There are other programs that use TiCC::tempname(). They should all be investigated.
A more sustainable and C++ like solution is to build "tempstream" class around tempname() which will clean up automatically when the object gets out of scope.
Should't be that hard.
from frog.
I fixed this in frog using a new tmp_stream() class at 2 locations.
Other locations where TiCC::tempname() was used already removed the temporary file, so all is sound now.
When the code is tested enough, the new tmp_stream class might be incorporated in ticcutils for general uses
Happy testing!
from frog.
Ah! Thanks! :) I told @StergiosMorakis I'd look at it next week but you beat me to it already, he'll be happy to hear that. I'll give it a test soon.
from frog.
@StergiosMorakis Is everything running nicely still after Ko's patch? Then I'll do a release.
from frog.
Released v0.21 which fixes this issue.
from frog.
@proycon , we are facing issues with centos 7 with other dependencies . For now i am planning to use older version . I want to cherry-pic changes related to this fix to older version
Found these related changes for this fix . Can you please check if there are any other commits related to this
a24ac9c
7a49cdc
4dc4b87
from frog.
Yes, those are indeed the right commits pertaining to this issue.
from frog.
@StergiosMorakis this is deleting files in /tmp folder but python is still maintaing the file descriptor count . Ideally this reference should be deleted after closing the file
So python-frog is failing to create another file in /tmp after it processes n request ( n = number set under ulimit -n )
what's the ulimit -n set to in your machine ?
Edit:-
Is this working fine for you ?
from frog.
@padurucr7 Are you running into this problem despite the patches in those three commits?? Then we should reopen the issue I guess.
from frog.
yeah , not sure why python is maintaining file descriptors ever after closing them in c++ part .
I went through the code and everything looks fine . May b this is issue with python .
So , i am explicitly closing file descriptors related to frog after each query . This fixed the issue
Note:- This issue is no way related to those 3 commits . It's happening with out these commits too .
base = '/proc/self/fd'
file_handler_idxs =[]
for num in os.listdir(base):
path = None
try:
path = os.readlink(os.path.join(base, num))
if path is not None and "frog" in path:
file_handler_idxs.append(num)
except Exception as e:
print(e)
pass
for idx,val in enumerate(file_handler_idxs):
os.close(int(val))
from frog.
Related Issues (20)
- Frog Chunker creates invalid FoLiA HOT 2
- released frog (0.29) depends on unreleased libfolia (2.15) HOT 2
- Building on Ubuntu 22.04 LTS Pop!_OS HOT 1
- Token annotation error for XML output with non-standard rules HOT 3
- segmentation fault when invoked with a missing [[tokenizer]] section in the configuration HOT 5
- Server mode creates only 1 paragraph HOT 2
- Add JSON output as an alternative to 'tabbed' format HOT 3
- Frog breaks while processing large amount of txt data HOT 11
- Keep the deep_morph structure intact when resolving MWU's HOT 1
- Simplify option and configuration handling
- MWU output when no Parser is selected HOT 7
- Update debian package for v0.20
- Python Frog HOT 2
- Praktische vragen rondom grote datasets HOT 7
- Bug: frog server; frog-:connection lost unexpected : write to client failed HOT 2
- Segfault on FoLiA in to FoLiA out (speech data with events and utterances) HOT 7
- New release? HOT 3
- frog lemmatizer with --deep-morph misses a morpheme in FoLiA output
- [Docker] Initialization fails for nld-vnn and dum HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from frog.