Comments (6)
It also seems to not work reliably as some invocations returned correctly {count(1) => 475} and others returned wrongly {}
> my $result = $dbh.query('select count(1) from requests');
DB::SQLite::Result.new(stmt => DB::SQLite::Native::Statement.new, count => 1, sth => DB::SQLite::Statement.new(stmt => DB::SQLite::Native::Statement.new, count => 1, db => DB::SQLite::Connection.new(conn => DB::SQLite::Native.new, owner => DB::SQLite.new(filename => "data/proxy.sqlite", busy-timeout => 10000, max-connections => 5, connections => Concurrent::Stack.new))), finish => Bool::True, keys-cache => Any)
> $result.hash
{count(1) => 475}
> $result.hash
{}
> $result.hash
{count(1) => 475}
> $result.hash
{}
> $result.hash
out of memory
in block at /home/demanuel/.raku/sources/3481BCAD53FC3338AC505479B91D3AD1F5BF3724 (DB::SQLite::Result) line 42
in method row at /home/demanuel/.raku/sources/3481BCAD53FC3338AC505479B91D3AD1F5BF3724 (DB::SQLite::Result) line 27
in method hash at /home/demanuel/.raku/sources/51F6397FB5D02B2A0E15A62EC3ACF47669DBF3DC (DB::Result) line 68
in block <unit> at <unknown file> line 1
>
from raku-dbsqlite.
hashes
uses an iterator to make a Seq
with a one time pass through the data. If you want to use it multiple times, assign it to a variable. Don't keep reusing the result object -- Just get out of it what you need and let it finish (which frees the memory). Exhausting the hashes iterator by reading into a variable will finish
it automatically and free its memory.
my @hashes = $dbh.query('select * from requests limit 1').hashes;
say $_<content_type> for @hashes;
BTW, you can see memory used by SQLite objects by printing DB::SQLite::Native.memory-used
at any time. If you have hanging objects (could be cached connections or statements, or un-finished results) you will see how much memory they are holding by printing that.
from raku-dbsqlite.
Is this applicable even when it's 1 element? I had the "limit 1" clause....
> say $_<content_type> for $result.hashes
image/jpeg
> DB::SQLite::Native.memory-used
325408
> say $_<content_type> for $result.hashes
image/jpeg
> DB::SQLite::Native.memory-used
325408
> say $_<content_type> for $result.hashes
image/jpeg
> DB::SQLite::Native.memory-used
325408
> say $_<content_type> for $result.hashes
image/jpeg
> DB::SQLite::Native.memory-used
162920
> say $_<content_type> for $result.hashes
out of memory
in block at /home/demanuel/.raku/sources/3481BCAD53FC3338AC505479B91D3AD1F5BF3724 (DB::SQLite::Result) line 42
in method row at /home/demanuel/.raku/sources/3481BCAD53FC3338AC505479B91D3AD1F5BF3724 (DB::SQLite::Result) line 27
in method pull-one at /home/demanuel/.raku/sources/51F6397FB5D02B2A0E15A62EC3ACF47669DBF3DC (DB::Result) line 21
in block <unit> at <unknown file> line 1
>
Only when it decreased the error appeared. And what memory are we talking about? My system has 32GiB of memory. That value shown is not enough for getting an out of memory.
There is also the issue that if i remove the "limit 1" clause, it becames unusable:
To exit type 'exit' or '^D'
> use DB::SQLite;
Nil
> my $dbh = DB::SQLite.new(filename => 'data/proxy.sqlite');
DB::SQLite.new(filename => "data/proxy.sqlite", busy-timeout => 10000, max-connections => 5, connections => Concurrent::Stack.new)
> DB::SQLite::Native.memory-used
0
> DB::SQLite::Native.memory-used
0
> my $result = $dbh.query('select * from requests limit 1');
DB::SQLite::Result.new(stmt => DB::SQLite::Native::Statement.new, count => 10, sth => DB::SQLite::Statement.new(stmt => DB::SQLite::Native::Statement.new, count => 10, db => DB::SQLite::Connection.new(conn => DB::SQLite::Native.new, owner => DB::SQLite.new(filename => "data/proxy.sqlite", busy-timeout => 10000, max-connections => 5, connections => Concurrent::Stack.new))), finish => Bool::True, keys-cache => Any)
> DB::SQLite::Native.memory-used
162520
> my $ct= $result.hashes[0]<content_type>;
image/jpeg
> DB::SQLite::Native.memory-used
168552
> my $result = $dbh.query('select * from requests');
DB::SQLite::Result.new(stmt => DB::SQLite::Native::Statement.new, count => 10, sth => DB::SQLite::Statement.new(stmt => DB::SQLite::Native::Statement.new, count => 10, db => DB::SQLite::Connection.new(conn => DB::SQLite::Native.new, owner => DB::SQLite.new(filename => "data/proxy.sqlite", busy-timeout => 10000, max-connections => 5, connections => Concurrent::Stack.new))), finish => Bool::True, keys-cache => Any)
> DB::SQLite::Native.memory-used
331072
> my %ct= $result.hashes;
It is stuck here for at least 30 mins (until i ctrl-c).
from raku-dbsqlite.
I suspect the 'out of memory' is a spurious error. The Results object is a one-time use object. You get the results, you pull them out, then you don't use the object again. I'll try to add some more checks to the object to prevent its reuse, but the fact remains that you just can't do that. Decide what you want out of it (value/hash/array/arrays/hashes), then call it one time to get the result, then don't access the result any more.
my $ct= $result.hashes[0]<content_type>;
This says that you want a sequence of hashes, then you access the first one. The sequence then hangs around waiting for you to get the rest (doesn't matter if there is just one).
If you just want a single hash, just call hash
:
my $ct = $result.hash<content_type>;
my %ct= $result.hashes;
hashes returns a sequence of hashes -- if you want just one, call hash
:
my %ct = $result.hash;
from raku-dbsqlite.
ok, got it.
Any reason why invoking .hashes
being so slow?
from raku-dbsqlite.
Building each hash is slow. First it retrieves the array, then pairs the values with keys to make the hashes. If performance is a concern, always use arrays.
from raku-dbsqlite.
Related Issues (16)
- Cannot locate native library 'libsqlite3.so': libsqlite3.so: cannot open shared object file: No such file or directory on Ubuntu 18.10 HOT 2
- DB::SQLite doesn't like IntStr placeholders HOT 2
- doc error in Result.pm6 HOT 1
- prepare statements don't protect apostrophes HOT 13
- provide a way to properly hot back up a db [FEATURE REQUEST]
- sqlite3 column names "oid" and "_rowid_" are not returned in a results hash HOT 2
- add query method .values HOT 2
- Use of Nil in numeric context in DB::SQLite::Statement line 44 HOT 14
- How to open a db in readonly mode? HOT 4
- The module fails to install due to hard-coded NativeLibs version HOT 1
- Rakudist test fails
- Native lib name sqlite3 vs sqlite HOT 6
- In memory database support broken
- Frequent "Database is locked" errors HOT 4
- can't use a named bind parameters with execute
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from raku-dbsqlite.