Comments (10)
benibela
commented on June 12, 2024
Do you call xquery.freeThreadVars at the end of the thread?
from internettools.
murnur19
commented on June 12, 2024
It's still there.
Can you please take a look at this example.
project1.tar.gz
from internettools.
benibela
commented on June 12, 2024
That is an interesting situation.
fpc creates a temporary IXQValue variable to store the result of evaluate/the for loop, which is released at the end of the execute function.
You need to call free on the query engine after that variable has been released, i.e. have the calculation and free in different methods.
freeThreadVars was for the global query function and IXQValue.map method. If you create and free a query engine, it has no effect right now.
That still leaves two unexplained, unfreed blocks
from internettools.
murnur19
commented on June 12, 2024
So, what you suggest. I want to reuse the engine for multiple xpath query from a thread.
If I do something like this then there will be no leak. But it's kind of defeat the purpose of interface.
v:=e.evaluateXPath3(Form1.Edit1.Text,t.getLastTree);
if v.Count>0 then
for i:=1 to v.Count do
begin
x:=v.get(i);
addout(x.tostring);
end;
x:=nil;
v:=nil;
from internettools.
benibela
commented on June 12, 2024
So, what you suggest. I want to reuse the engine for multiple xpath query from a thread.
Create one engine at the beginning of the thread and free it at the end of the execute method. But do not do any calculations involving IXQValues inside the execute method.
When there are no calculations/ixqvalues in the execute method, you could put TXQueryEngine.create.free; there. The last free in a thread frees everything, except still existing ixqvalues.
I could add a test to free the interfaces immediately, if no engine exists. But it would make things slower. All this caching memory management is an optimization for people having millions of IXQValues.
That still leaves two unexplained, unfreed blocks
That seems to be the lack of freeonterminate := true; in the example.
from internettools.
murnur19
commented on June 12, 2024
Can you make an example?
I would like to handle the download by myself, examine a page source and probably spawn another thread to examine the sub page. While also running multiple thread with similar job. So there will be a lot of engine and IXQValues.
That seems to be the lack of freeonterminate := true; in the example.
Yes, I forgot about that.
from internettools.
benibela
commented on June 12, 2024
Just keep the IXQValue away from execute:
TXPath3Thread=class(tthread)
private
engine: TXQueryEngine;
procedure addoutput;
procedure addout(const s: string);
protected
procedure Calculate;
procedure Execute; override;
end;
procedure TXPath3Thread.Calculate;
var
v: IXQValue;
t: TTreeParser;
begin
t:=TTreeParser.create;
try
t.parsingModel := pmHTML;
t.repairMissingStartTags:=true;
t.repairMissingEndTags:=true;
t.parseTree('<html><a href="a"/></html>');
for v in engine.evaluateXPath3('//a/@href',t.getLastTree) do
addout(v.toString);
except
on e:exception do
addout(e.Message);
end;
t.free
end;
procedure TXPath3Thread.Execute;
begin
engine := TXQueryEngine.create;
Calculate;
engine.free;
FreeOnTerminate := true;
end;
Or with the global engine:
TXPath3Thread=class(tthread)
private
procedure addoutput;
procedure addout(const s: string);
protected
procedure Calculate;
procedure Execute; override;
end;
procedure TXPath3Thread.Calculate;
var
v: IXQValue;
t: TTreeParser;
begin
t:=TTreeParser.create;
try
t.parsingModel := pmHTML;
t.repairMissingStartTags:=true;
t.repairMissingEndTags:=true;
t.parseTree('<html><a href="a"/></html>');
for v in xqvalue(t.getLastTree).map('//a/@href') do
addout(v.toString);
except
on e:exception do
addout(e.Message);
end;
t.free
end;
procedure TXPath3Thread.Execute;
begin
Calculate;
freeThreadVars;
FreeOnTerminate := true;
end;
from internettools.
murnur19
commented on June 12, 2024
Do the global engine thread-safe?
from internettools.
benibela
commented on June 12, 2024
yes.
(and that is the reason for the memory issues. Every thread gets its own cache, and when the thread ends without freeing the entire cache, no other thread can access the cache to free or use it)
from internettools.
murnur19
commented on June 12, 2024
I see. Thanks.
from internettools.
Related Issues (20)
- Memory leak?
- Invalid symbol in hex entity: xxx HOT 1
- Memory leaks with fpc trunk? HOT 3
- Infinity memory consumption HOT 2
- Configuration proxy with username and password? HOT 1
- Xpath 3.1 update HOT 2
- Unable to compile on linux HOT 2
- Can I query from a string? HOT 5
- How to get query result to html instead of plain text? HOT 2
- How to replace Node innerHTML and save it to new string? HOT 4
- Compiling on FreeBSD HOT 3
- Android build problem HOT 9
- [Suggestion] Why do you still use travis and not GitHub actions? HOT 4
- Removing Or enhance the EInternetException HOT 2
- Failed to compile, cause Error: Incompatible type for arg no. 2 HOT 2
- Will not install using Package Manager in Lazarus 3.99 HOT 2
- 朋友，能弄个delphi版本吗 HOT 1
- declaration of TFLRESizeInt HOT 3
- Compile error HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from internettools.