wix-incubator/quix

Quix Notebook Manager

quix's Issues

add health check + monitoring endpoints

some relevant repos:

https://github.com/nestjs/terminus
https://github.com/CloudNativeJS/cloud-health-connect
https://github.com/digikare/nestjs-prom
https://github.com/uber/express-statsd

file tree improvements

• notebook move
• folder move (recursive)
• folder delete (recursive)

preserve newlines when submitting query to presto

good example :

bad example :

backend : remove mvn test phase as mvn install includes it

travis.yml includes both mvn install and mvn test commands.

but according to http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html install already includes test

also raw travis logs show that same tests are executed twice :
https://api.travis-ci.com/v3/job/194732404/log.txt

first execution of RefreshableDbTest at 20:57:08

Running quix.presto.db.RefreshableDbTest
20:57:08.473 [specs2-2] WARN quix.presto.db.RefreshableDb$ - Catalog Catalog(bi,List(),catalog)is empty
20:57:08.589 [specs2-1] INFO quix.presto.QueryExecutor - method=initClient event=start query-id=7b5735fa-3e1e-4d3b-a7c2-188886aebd00 user=quix-db-tree
20:57:08.588 [specs2-2] INFO quix.presto.QueryExecutor - method=initClient event=start query-id=3dff2531-4115-440d-b34c-12435959dcca user=quix-db-tree
20:57:08.603 [specs2-1] INFO quix.presto.QueryExecutor - method=runAsync event=start query-id=7b5735fa-3e1e-4d3b-a7c2-188886aebd00 user=quix-db-tree sql=select distinct table_cat, table_schem, table_name from system.jdbc.tables

second execution of RefreshableDbTest at 20:58:51

Running quix.presto.db.RefreshableDbTest
20:58:51.980 [specs2-2] WARN quix.presto.db.RefreshableDb$ - Catalog Catalog(bi,List(),catalog)is empty
20:58:52.137 [specs2-3] INFO quix.presto.QueryExecutor - method=initClient event=start query-id=82f26a45-46cb-47aa-b16c-2c0a9836ec67 user=quix-db-tree
20:58:52.152 [specs2-2] INFO quix.presto.QueryExecutor - method=initClient event=start query-id=8f2c27f7-f576-49fd-b873-5de3e0901242 user=quix-db-tree
20:58:52.165 [specs2-2] INFO quix.presto.QueryExecutor - method=runAsync event=start query-id=8f2c27f7-f576-49fd-b873-5de3e0901242 user=quix-db-tree sql=select distinct table_cat, table_schem, table_name from system.jdbc.tables

Add a temporary note

Add an option to quickly run a script without creating a note.

Add chart visualization

backend : generalize ActiveQuery and PrestoExecutions

frontend: add pagination in search results

cover quix-backend with E2E tests

Don't show header and menu until after login

backend : extract logging/metering into interface to allow pluggable implementations

Docs: incorrect anchor link to Configuration

Docs Installation page (https://wix.github.io/quix/docs/installation) references a link to the Configuration section incorrectly.

Link is set to: https://wix.github.io/quix/docs/installation#Configuration
Needs to be set to: https://wix.github.io/quix/docs/installation#configuration

That's #configuration spelled with a lowercase "c".

Change default favicon

right now we are using default wix favicon, can we change it to quix icon?

Create a manual login page

Don't log the user in automatically

After some time I get a message "Connection Lost"

Hi
After a night the process up and running when I try to make a query I get a message "connection lost"
Seems to be a problem with timeout of connections or something like that
I've tried to see for exception in the log container but nothing there
Tried to restart the container do not work either !?!
I join a screenshot

featuring Quix in Docusaurus

Hello, we would like to feature your project to the Docusaurus users

If you are okay with it, are you able to provide a logo we can use? Additional information can be found here

Add contextual auto complete to the sql editor

improve timeseries graph visualisation

SELECT date_trunc('minute', created) as date_created, count(1) as count
FROM system.runtime.queries
group by 1

will result in following visualisation :

• the dates on X axis are overlapping one another
• the format of tooltip doesn't match the values (it shows hour first and then date)
• two adjacent points can't be distinguished one from another (the cursor points to two different points in time)

backend : support "use" command

https://prestosql.io/docs/current/sql/use.html

backend : refactor common code to quix-core

add run and download ability to quix-backend

Add option to "like" a notebook

add /api/db/autocomplete endpoint

hide variables boilerplate from search results

right now only first 2 lines are presented in search results.

for notes with variables it looks like this:

improve error message on empty query execution

now :

Fix note share (with params)

At the moment sharing a note with params doesn't add the note id to the url.

How to use my presto cluster ?

I've succeed to access my presto cluster
I see quix making queries to this cluster now
but nothing ! the loading db keep busy and no schema is retrieved from the server
The only request made repeatedly is

select 1 from raptor.audience.monthly_users limit 1

The table name is monthly_users
Is there something that must be configured ?
Thanks

backend : handle flaky IT test DownloadControllerTest

Aws Athena support

Question: can this product works on top of Athena instead of Presto directly?

fix search to handle partial terms

frontend: implement "trash" for notebooks/notes

align start execution payload with rest of "event" payloads

backend : handle "ping" messages in execute websocket

add history of query executions

we want the ability to see in quix the history of all past executions.

users should be able to see their queries, queries of other users, all queries that were executed in some timerange.

let's try adding to quix-core module of quix-backend the trait that should handle management of such data.

Add browser notifications on note run success/error

backend : rewrite set session support

use setSessionProperties from QueryInfo

Add Pie chart

use system.jdbc tables for db-tree

UnknownHostException: presto

Hi,
I've tried installing quix on my server.. However, When I try to run a query, the backend throws the following exception:

[quix-db-tree-32] quix.presto.SingleResultBuilder : Got exception
backend_1 |
backend_1 | java.net.UnknownHostException: presto
backend_1 | at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:220) ~[na:na]
backend_1 | at java.base/java.net.Socket.connect(Socket.java:591) ~[na:na]
backend_1 | at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177) ~[na:na]
backend_1 | at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474) ~[na:na]
backend_1 | at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569) ~[na:na]
backend_1 | at java.base/sun.net.www.http.HttpClient.(HttpClient.java:242) ~[na:na]
backend_1 | at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:341) ~[na:na]
backend_1 | at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:362) ~[na:na]
backend_1 | at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1242) ~[na:na]
backend_1 | at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1181) ~[na:na]
backend_1 | at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1075) ~[na:na]
backend_1 | at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1009) ~[na:na]
backend_1 | at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1581) ~[na:na]
backend_1 | at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1509) ~[na:na]
backend_1 | at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:527) ~[na:na]
backend_1 | at scalaj.http.HttpRequest.doConnection(Http.scala:367) ~[scalaj-http_2.12-2.4.1.jar:2.4.1]
backend_1 | at scalaj.http.HttpRequest.exec(Http.scala:343) ~[scalaj-http_2.12-2.4.1.jar:2.4.1]
backend_1 | at scalaj.http.HttpRequest.asString(Http.scala:491) ~[scalaj-http_2.12-2.4.1.jar:2.4.1]
backend_1 | at quix.presto.rest.ScalaJPrestoStateClient.$anonfun$init$3(ScalaJPrestoStateClient.scala:22) ~[quix-presto-module-1.0.0-SNAPSHOT.jar:na]
backend_1 | at monix.eval.internal.TaskRunLoop$.startFull(TaskRunLoop.scala:81) ~[monix-eval_2.12-3.0.0-RC2.jar:3.0.0-RC2]
backend_1 | at monix.eval.internal.TaskRestartCallback.syncOnSuccess(TaskRestartCallback.scala:108) ~[monix-eval_2.12-3.0.0-RC2.jar:3.0.0-RC2]
backend_1 | at monix.eval.internal.TaskRestartCallback.onSuccess(TaskRestartCallback.scala:74) ~[monix-eval_2.12-3.0.0-RC2.jar:3.0.0-RC2]
backend_1 | at monix.eval.internal.TaskSleep$SleepRunnable.run(TaskSleep.scala:62) ~[monix-eval_2.12-3.0.0-RC2.jar:3.0.0-RC2]
backend_1 | at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
backend_1 | at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
backend_1 | at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

This is my backend application.properties file:

server.port=8080
presto.api=test-server01:8090
logging.level.org.springframework.web=DEBUG
logging.level.org.hibernate=ERROR

Am I missing something?

docker-compose up failed

quix docker-compose up failure.docx

frontend : progress UI doesn't match the execute socket progression

example :

socket says 30%, UI says 0%

use this query to reproduce :

WITH customer_total_return 
     AS (SELECT sr_customer_sk     AS ctr_customer_sk, 
                sr_store_sk        AS ctr_store_sk, 
                Sum(sr_return_amt) AS ctr_total_return 
         FROM   tpcds.sf1.store_returns, 
                tpcds.sf1.date_dim 
         WHERE  sr_returned_date_sk = d_date_sk 
                AND d_year = 2001 
         GROUP  BY sr_customer_sk, 
                   sr_store_sk) 
SELECT c_customer_id 
FROM   customer_total_return ctr1, 
       tpcds.sf1.store, 
       tpcds.sf1.customer 
WHERE  ctr1.ctr_total_return > (SELECT Avg(ctr_total_return) * 1.2 
                                FROM   customer_total_return ctr2 
                                WHERE  ctr1.ctr_store_sk = ctr2.ctr_store_sk) 
       AND s_store_sk = ctr1.ctr_store_sk 
       AND s_state = 'TN' 
       AND ctr1.ctr_customer_sk = c_customer_sk 
ORDER  BY c_customer_id
LIMIT 100;

add shortcut for existing full-screen edit mode (esc for example)

it would be great if we can exit the full-screen edit view by clicking esc key

@AntonPOD

search results are hidden on small screens

to reproduce :

1. search for anything
2. open dev tools, it will shrink the visible screen size

[Question] clickhouse support

do you guys have docs on how to add additional data sources?

It would be great if you supported other datasources (similar to:
https://redash.io/help/data-sources/setup/supported-data-sources
)

I'm currently interested in: https://clickhouse.yandex/
Do you have plans to support it (and similar)?

Thanks

Mandatory authentication?

Hi,
Could authentication method be required?
I've tried to run quix with basic configuration and got the following exception in backend when trying to run a query:

backend_1 | quix.api.users.RequestNotAuthenticated: Request not authenticated
backend_1 | at quix.web.auth.JwtUsers.$anonfun$auth$5(JwtUsers.scala:20) ~[classes/:na]
backend_1 | at scala.Option.getOrElse(Option.scala:121) ~[scala-library-2.12.7.jar:na]
backend_1 | at quix.web.auth.JwtUsers.auth(JwtUsers.scala:20) ~[classes/:na]
backend_1 | at quix.web.controllers.PrestoController.handleTextMessage(PrestoController.scala:23) ~[classes/:na]
backend_1 | at org.springframework.web.socket.handler.AbstractWebSocketHandler.handleMessage(AbstractWebSocketHandler.java:43) ~[spring-websocket-5.1.2.RELEASE.jar:5.1.2.RELEASE]
backend_1 | at org.springframework.web.socket.handler.WebSocketHandlerDecorator.handleMessage(WebSocketHandlerDecorator.java:75) ~[spring-websocket-5.1.2.RELEASE.jar:5.1.2.RELEASE]
backend_1 | at org.springframework.web.socket.handler.LoggingWebSocketHandlerDecorator.handleMessage(LoggingWebSocketHandlerDecorator.java:56) ~[spring-websocket-5.1.2.RELEASE.jar:5.1.2.RELEASE]
backend_1 | at org.springframework.web.socket.handler.ExceptionWebSocketHandlerDecorator.handleMessage(ExceptionWebSocketHandlerDecorator.java:58) ~[spring-websocket-5.1.2.RELEASE.jar:5.1.2.RELEASE]
backend_1 | at org.springframework.web.socket.adapter.jetty.JettyWebSocketHandlerAdapter.onWebSocketText(JettyWebSocketHandlerAdapter.java:83) ~[spring-websocket-5.1.2.RELEASE.jar:5.1.2.RELEASE]
backend_1 | at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
backend_1 | at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
backend_1 | at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
backend_1 | at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
backend_1 | at org.eclipse.jetty.websocket.common.events.annotated.CallableMethod.call(CallableMethod.java:71) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.events.annotated.OptionalSessionCallableMethod.call(OptionalSessionCallableMethod.java:72) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.events.JettyAnnotatedEventDriver.onTextMessage(JettyAnnotatedEventDriver.java:234) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:66) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.events.JettyAnnotatedEventDriver.onTextFrame(JettyAnnotatedEventDriver.java:226) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:162) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:476) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.extensions.AbstractExtension.nextIncomingFrame(AbstractExtension.java:183) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.nextIncomingFrame(PerMessageDeflateExtension.java:105) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension.forwardIncoming(CompressExtension.java:142) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.incomingFrame(PerMessageDeflateExtension.java:85) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:220) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:219) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:244) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.readParse(AbstractWebSocketConnection.java:560) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:391) ~[websocket-common-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) ~[jetty-io-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) ~[jetty-io-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) ~[jetty-io-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) ~[jetty-util-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) ~[jetty-util-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) ~[jetty-util-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) ~[jetty-util-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) ~[jetty-util-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) ~[jetty-util-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) ~[jetty-util-9.4.12.v20180830.jar:9.4.12.v20180830]
backend_1 | at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
backend_1 |

Also, When I try to view a table's details in the DB explorer, I get the following error in console:

Error: "[$resource:badcfg] Error in resource configuration for action query. Expected response to contain an array but got an object (Request: GET /api/db/explore/cassandra/keyspace1/table_test1)
https://errors.angularjs.org/1.7.8/$resource/badcfg?p0=query&p1=array&p2=object&p3=GET&p4=%2Fapi%2Fdb%2Fexplore%2Fcassandra%2Fkeyspace1%2Ftable_test1"
i http://test-server01:3000/app.bundle.min.js:1
G http://test-server01:3000/app.bundle.min.js:1
u http://test-server01:3000/app.bundle.min.js:1
u http://test-server01:3000/app.bundle.min.js:1
$digest http://test-server01:3000/app.bundle.min.js:1
$apply http://test-server01:3000/app.bundle.min.js:1
b http://test-server01:3000/app.bundle.min.js:1
y http://test-server01:3000/app.bundle.min.js:1
onload http://test-server01:3000/app.bundle.min.js:1
Possibly unhandled rejection: {}

It seems that the response is okay(and yes, an object), but not an array.

Add note reordering

An error when executing the demo query: "This connector does not support roles"

Automatically generated query for 1000 rows:

SELECT *
FROM tpch.information_schema.applicable_roles
LIMIT 1000

Gives an error:
"This connector does not support roles"

docker-compose not working

Hi
I try to start the server as described I get
Starting quix_presto_1 ...
Starting quix_presto_1 ... done
Starting quix_backend_1 ... error

ERROR: for quix_backend_1 Cannot start service backend: driver failed programming external connectivity on endpoint quix_backend_1 (279f5f64a60ebb43a0e883a02844357c2adc322412678e3af1aa1d5a68e9e72a): Error starting userland proxy: listen tcp 0.0.0.0:8080: bind: address already in use

ERROR: for backend Cannot start service backend: driver failed programming external connectivity on endpoint quix_backend_1 (279f5f64a60ebb43a0e883a02844357c2adc322412678e3af1aa1d5a68e9e72a): Error starting userland proxy: listen tcp 0.0.0.0:8080: bind: address already in use
ERROR: Encountered errors while bringing up the project.

Checked on my machine no port 8080 used
Do you've any idea ?
Thanks

add generic jdbc note + driver

sql syntax validation: handle mixed letter cases

if a keyword in written in mixed case From it will be an error in the client-side syntax checker