qsft/doradus

Doradus is a REST service that extends a Cassandra NoSQL database with a graph-based data model, advanced indexing and search features, and a REST API. See also doradus-openshift-quickstart, which allows Doradus to run as an OpenShift cartridge!

doradus's Issues

unable to merge shards

Hi. im working on OLAP. im not able to merge the shards. im using the Doradus-jetty REST api's.
when i run the merge REST api it say.Application does not exist or does not have merges yet.

and when i run the api for application listing. its shows my application. can you please help me with shard merging.

Loaded data from csv file with doradus-client is not available for query

I'm trying to use Doradus server, i've setup the server correctly (it's up and running). I've a problem when I try to use the doradus-client to push a tsv file:

• It doesn't support other separators than , so I've modified the source to support \t,
• The file needs a header (even if the schema is provided) so I've added a list of column names,
• The application name is hard coded to BIM, so I've changed my app name to that,

When I launch the client I don't see a problem in the output but actually there is no data been pushed:

$ java -cp target/doradus-client-2.3.1.jar:target/dependency/* com.dell.doradus.client.utils.CSVLoader -schema BIM.json -delimiter \t
17:32:47,337 INFO CSVLoader: Deleting existing application: BIM
17:32:48,333 INFO CSVLoader: Creating application 'BIM' with schema: /Users/dzlab/Workspace/DB/Doradus/doradus-client/BIM.json
17:32:48,896 INFO CSVLoader: Starting 3 workers
17:32:48,922 INFO CSVLoader: Scanning for files in folder: /Users/dzlab/Workspace/DB/Doradus/.
17:32:48,922 INFO CSVLoader: Loading CSV file: /Users/dzlab/Workspace/DB/Doradus/./sample_withheader.csv
17:32:54,896 INFO CSVLoader: ...loaded 10000 records.
17:32:58,824 INFO CSVLoader: ...loaded 20000 records.
17:33:01,635 INFO CSVLoader: ...loaded 30000 records.
17:33:04,507 INFO CSVLoader: ...loaded 40000 records.
17:33:07,140 INFO CSVLoader: ...loaded 50000 records.
17:33:09,513 INFO CSVLoader: ...loaded 60000 records.
17:33:11,690 INFO CSVLoader: ...loaded 70000 records.
17:33:14,581 INFO CSVLoader: ...loaded 80000 records.
17:33:16,807 INFO CSVLoader: ...loaded 90000 records.
17:33:19,053 INFO CSVLoader: ...loaded 100000 records.
17:33:21,747 INFO CSVLoader: ...loaded 110000 records.
17:33:24,079 INFO CSVLoader: ...loaded 120000 records.
17:33:26,720 INFO CSVLoader: ...loaded 130000 records.
17:33:29,138 INFO CSVLoader: ...loaded 140000 records.
17:33:31,547 INFO CSVLoader: ...loaded 150000 records.
17:33:34,020 INFO CSVLoader: ...loaded 160000 records.
17:33:36,602 INFO CSVLoader: ...loaded 170000 records.
17:33:38,899 INFO CSVLoader: ...loaded 180000 records.
17:33:41,408 INFO CSVLoader: ...loaded 190000 records.
17:33:43,791 INFO CSVLoader: ...loaded 200000 records.
17:33:46,367 INFO CSVLoader: ...loaded 210000 records.
17:33:49,911 INFO CSVLoader: ...loaded 220000 records.
17:33:52,121 INFO CSVLoader: ...loaded 230000 records.
17:33:54,300 INFO CSVLoader: ...loaded 240000 records.
17:33:56,982 INFO CSVLoader: ...loaded 250000 records.
17:33:59,444 INFO CSVLoader: ...loaded 260000 records.
17:34:02,955 INFO CSVLoader: ...loaded 270000 records.
17:34:08,699 INFO CSVLoader: ...loaded 280000 records.
17:34:11,146 INFO CSVLoader: ...loaded 290000 records.
17:34:14,034 INFO CSVLoader: ...loaded 300000 records.
17:34:17,071 INFO CSVLoader: ...loaded 310000 records.
17:34:17,794 INFO CSVLoader: File '/Users/dzlab/Workspace/DB/Doradus/./sample_withheader.csv': time=88867 millis; lines=313000
17:34:18,418 INFO CSVLoader: Total files loaded:    1
17:34:18,418 INFO CSVLoader: Total lines scanned:   313000
17:34:18,418 INFO CSVLoader: Total bytes read:      158616510
17:34:18,418 INFO CSVLoader: Total time for load:   1 minute, 29 seconds
17:34:18,418 INFO CSVLoader: Average lines/sec:     3497
17:34:18,418 INFO CSVLoader: Average bytes/sec:     1772330

Process finished with exit code 0

But when I check the logs from the doradus-server, I only see the creation of schema:

...
17:32:48,342 INFO SchemaService: Defining application: BIM
17:32:48,344 INFO CassandraSchemaMgr: Creating ColumnFamily: Doradus:BIM_sample
17:32:48,454 INFO CassandraSchemaMgr: Creating ColumnFamily: Doradus:BIM_sample_Terms

When I check the urls, there is nothing:

GET http://localhost:1123/_tasks
<tasks/>
GET http://localhost:1123/_olapp
Applications    Shards  Tables

The application is actually created:

GET http://localhost:1123/_applications
<applications>
  <application name="BIM">
  <key>BIM_key</key>
  <options>
    <option name="StorageService">SpiderService</option>
    <option name="AutoTables">true</option>
  </options>
  <tables>
    <table name="sample"/>
  </tables>
</application>

But no docs:

GET http://localhost:1123/BIM/sample/_query?q=*
<results>
  <docs/>
</results>

On debug mode, when I check the BatchResult I see a 1000 ObjectResult that have the following error Invalid field name: .... This fields is a Double and it is correctly set in the JSON schema as well as in the CSV file header.
Is there any naming constraints on scalar fields?

Internal Server Error with Spider on any field query

I was trying out any field query as per: https://github.com/dell-oss/Doradus/wiki/Text-Contains-Clauses%20(Spider) and I got internal server error response from doradus. Is this intended?

500 Internal Server Error: java.lang.NullPointerException
    at com.dell.doradus.service.spider.SpiderHelper.getFields(SpiderHelper.java:569)
    at com.dell.doradus.search.builder.BuilderBinary.search(BuilderBinary.java:59)
    at com.dell.doradus.search.Searcher.search(Searcher.java:115)
    at com.dell.doradus.search.builder.BuilderAnd.search(BuilderAnd.java:42)
    at com.dell.doradus.search.Searcher.search(Searcher.java:115)
    at com.dell.doradus.search.Searcher.search(Searcher.java:97)
    at com.dell.doradus.search.QueryExecutor.search(QueryExecutor.java:119)
    at com.dell.doradus.search.QueryExecutor.search(QueryExecutor.java:113)
    at com.dell.doradus.search.QueryExecutor.search(QueryExecutor.java:84)
    at com.dell.doradus.service.spider.ObjectQuery.query(ObjectQuery.java:304)
    at com.dell.doradus.service.spider.SpiderService.objectQueryURI(SpiderService.java:194)
    at com.dell.doradus.service.spider.QueryURICmd.invokeUNodeOut(QueryURICmd.java:34)
    at com.dell.doradus.service.rest.UNodeOutCallback.invoke(UNodeOutCallback.java:39)
    at com.dell.doradus.service.rest.RESTServlet.validateAndExecuteRequest(RESTServlet.java:162)
    at com.dell.doradus.service.rest.RESTServlet.doGet(RESTServlet.java:66)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:800)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:517)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
    at org.eclipse.jetty.server.Server.handle(Server.java:497)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:245)
    at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
    at java.lang.Thread.run(Thread.java:745)

Example: _query?q=*:(sflksjf)

OLAP support in Doradus

hi all, I see OLAP mention a lot in the documentation , but most of the document talk and example about columnar db .
I wander (maybe I missed it ) if you've support for OLAP Cube!. I'm especially interested in the "roll-up" capabilities.

I would also like also to discuss on offline (my mail is agonenil at gmail...) threat a use case that we are considering to use doradus.

error using the example

Hi I've installed Doradus using docker and using the example :

curl -H "content-type: application/json" -d '{"Stuff": {}}' http://172.17.0.1:1123/_applications

i'm getting the following error:
application definition missing 'key' definition

Code examples for link querying/updates

I read the docs on links and MV fields, but I'm not sure how to use doradus session (java client) to either link one object to another or query for example: "Find me all people whose manager is Bob". Code examples would be wonderful.