Comments (5)
Running this testing code fails the assert mentioned above on the attached dataset and workload.
The assert gets triggered after creation and evaluation of multiple instances of SuRF with different parameter values.
My guess is that there is some global state or variable that maintains it's value from the previous instance without being reset.
NOTE about recreating the assertion failure: currently SuRF has debug mode turned off, CMakeLists.txt -> line 24: "add_definitions(-DNDEBUG)". To turn debug mode on, remove "add_definitions(-DNDEBUG)", recompile, and run this code to (hopefully) recreate the bug. If the following code doesn't trigger the assert(false) before the return statement, then debug mode is off.
#include <iostream>
#include <vector>
#include <algorithm>
#include "include/surf.hpp"
#include <string>
#include <fstream>
using namespace surf;
using namespace std;
vector<string> read_dataset(const string& file_path) {
vector<string> dataset;
ifstream file(file_path);
cout << "INIT reading dataset file " << file_path << endl;
int id = 0;
while (!file.eof()) {
string line;
getline(file, line);
if(!line.empty()) {
dataset.push_back(line);
}
id++;
}
file.close();
cout << "END reading dataset file " << file_path << endl;
cout << "READ " << dataset.size() << " rows" << endl;
return dataset;
}
static vector<string> split(string str, const string& delim) {
vector<string> ret;
while (str.find(delim) != std::string::npos) {
if (str.find(delim) >= 1) {
ret.push_back(str.substr(0, str.find(delim)));
}
str = str.substr(str.find(delim) + delim.size(), str.size() - (str.find(delim) + delim.size()));
}
if (!str.empty()) {
ret.push_back(str);
}
return ret;
}
vector<pair<string, string> > read_workload(const string& file_path) {
vector<pair<string, string> > workload;
ifstream file(file_path);
cout << "INIT reading workload file " << file_path << endl;
int id = 0;
while (!file.eof()) {
string line;
getline(file, line);
assert(line.find(' ') != std::string::npos);
if (!line.empty()) {
vector<string> parts = split(line, " ");
assert(parts.size() == 2);
workload.emplace_back(parts[0], parts[1]);
}
id++;
}
file.close();
cout << "END reading workload file " << file_path << endl;
cout << "READ " << workload.size() << " rows" << endl;
return workload;
}
int surf_main() {
vector<string> dataset = read_dataset("dataset.txt");
sort(dataset.begin(), dataset.end());
vector<pair<string, string> > workload = read_workload("workload.txt");
for (int trie_size = 0; trie_size <= 64; trie_size++) {
surf::SuRF *surf_real = new surf::SuRF(dataset, surf::kReal, 0, trie_size);
for (int i = 0; i < workload.size(); i++) {
bool out = surf_real->lookupRange(workload[i].first, true, workload[i].second, true);
cout << out << endl;
}
}
assert(false);
return 0;
}
from surf.
Please check out the latest commit which hopefully fixes the bug :)
from surf.
Hi! The new commit did indeed fix the issue above.
HOWEVER!
There is another issue. Same exact setup, only with the following dataset and workload:
dataset.txt
workload.txt
Fails at the exact same assert, but probably for a different reason (it's not because of initialization of multiple instances of surf).
Failure happens even with this surf_main:
int surf_main() {
vector<string> dataset = read_dataset("dataset.txt");
sort(dataset.begin(), dataset.end());
vector<pair<string, string> > workload = read_workload("workload.txt");
int trie_size = 45; {
surf::SuRF *surf_real = new surf::SuRF(dataset, surf::kReal, 0, trie_size);
int i = 0; {
bool out = surf_real->lookupRange(workload[i].first, true, workload[i].second, true);
cout << out << endl;
}
}
assert(false);
return 0;
}
from surf.
Committed with additional fixes.
from surf.
Thanks! It works now!
from surf.
Related Issues (14)
- Please clarify license HOT 1
- How Can I benchmark RockDB with SuRF? HOT 3
- Array index out of bound? HOT 2
- How can the lookUpRange function be parallel? HOT 1
- Why lookupKey function doesn't work as expected?
- the procedure of moving down the trie in LOUDS-Sparse
- Buffer overrun in bitvector HOT 2
- Potential assertion failure in louds sparse HOT 3
- Failed to lookupRange via deserialized SuRF HOT 3
- Got a "double free or corruption" when destroy a deserialized SuRF in BitvectorRank
- RocksDB measurements with hybrid filtering (Bloom filters for point queries and SuRF for range queries)
- Non-0 false negative rate! HOT 1
- deSerialize(serialize()) doesn't clone properly. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from surf.