abhioncbr/docker-superset

Repository for Docker Image of Apache-Superset. [Docker Image: https://hub.docker.com/r/abhioncbr/docker-superset]

docker-superset's Issues

Can't load Athena JDBC plugin

I'm trying to connect Athena JDBC plugin to Superset

I get an error
ERROR: {"error": "Connection failed!\n\nThe error message returned was:\nCan't load plugin: sqlalchemy.dialects:awsathena.jdbc"}

Same with rest plugin:
ERROR: {"error": "Connection failed!\n\nThe error message returned was:\nCan't load plugin: sqlalchemy.dialects:awsathena.rest"}

Am I missing something? should I load the plugin manually?

Build

Hello, first of all I congratulate you and I thank you for the contribution of your repository, it has made it much easier for me to start with the concept tests that I must do on Superset.

I'm not the most expert in Docker so the question may sound a little stupid.

I managed to start the container using docker-compose and it works perfectly, however I must make some adjustments from the beginning so I modified the Dockerfile and generated the image with the "docker build" as it is specified in the documentation. However, I can not get the container to start, it always remains in an error state. I'm not clear about the command that I should use. It must be this:

"cd docker-superset & & docker run -p 8088: 8088 -v config: / home / superset / config / abhioncbr / docker-superset: cluster server <superset_metadata_db_url> <redis_url>"

the parameter "superset_metadata_db_url" is the host of the metadata DB with the structure user @ host .....?

When running the "docker build" does not generate the redis container, I must generate it separately and in "redis_url" I send the URL of this one?

Basically what I need is to use the Superset container but to automatically execute a Sh that modifies the Superset configuration, also creates some folders in the container and copies some monitoring routines, but that allows us to use Superset updates, that I thought about modifying the Dockerfile and executing the Sh in the "docker-entrypoint.sh", in that case the best option is to use the docker build of your container? or is there some other simpler way?

I would very much appreciate any help you can give me!

Thank you.

Dashboard Metadata in one Container

Hi Abhishek,

Thanks for your detailed blog and process to setup Apache superset.

I am deploying Superset on Container, However I am not able to import dashboard once the given image by you runs.

Also, I am trying to deploy the dashboard metadata database in one container itself.

Could you please share the process if you are aware or direct me to the link as i am working on POC and new to Superset.

Regards,
Tausif.

flask.cli.NoAppException while starting superset container in local mode using docker-compose command.

Describe the bug
With Superset image of version '0.29.0rc7', Superset gave 'flask.cli.NoAppException'
in the local mode using docker-compose.

To Reproduce
Steps to reproduce the behavior:

1. pull superset image of version '0.29.0rc7'.
2. start the container using docker-compose command and in 'local' mode.
3. access superset home page using url localhost:8088
4. 'flask.cli.NoAppException'

Expected behavior
Shouldn't be that page. The home page of the superset should be rendered.

Screenshots
If applicable, add screenshots to help explain your problem.

elasticsearch connection error

Describe the bug
error:
ERROR: {"error": "Connection failed!\n\nThe error message returned was:\nCan't load plugin: sqlalchemy.dialects:elasticsearch.https"}

when adding elasticsearch data source

To Reproduce
Steps to reproduce the behavior:

1. Go to 'Sources'
2. Click on 'Databases'
3. Click on Add Database
4. Add Elasticsearch HTTPS with Authentication
5. Click on Test Connection

Expected behavior
success with the test connection

Latest Superset install using the docker-compose

Use of environment file for docker-compose instead of passing through command

Is your feature request related to a problem? Please describe.
Currently, for running docker-superset using compose command, we need to pass environment variables (SUPERSET_ENV=local SUPERSET_VERSION=), instead of that .env file would be much better.

Describe the solution you'd like
wrap all environment variables in .env file which will be picked by docker-compose command.

Install database dependencies python plugins in docker images

Is your feature request related to a problem? Please describe.
Currently, docker image only consists dependencies plugins of MySql, Postgres and, BigQuery. For connectivity to other databases, docker image needs to be rebuilt. Install plugins for all common databases.

Describe the solution you'd like
Install the database dependencies listed in Superset manual

gevent not found when running 0.34 builds

Describe the bug
gevent lib is not installed on container and when it starts worker I see following error.
I do not understand how did you managed to run this version without seeing this error.

To Reproduce
Steps to reproduce the behavior:

1. use latest images with tag 0.34.0 or 0.34.0rc1 from dockerhub
2. Running on prod and compose parameters
3. After workers started I see log
   Started Celery workers[worker1, worker2] and Flower UI.
4. After server starts I see in the log
   File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/ggevent.py", line 22, in
   73218f80f0a8bcaaa3f5f9d2c04fcabd8fa3a4feb0619aeb716e09856a865a85
   raise RuntimeError("You need gevent installed to use this worker.")

Expected behavior
If I switch back to 0.33.0rc1 which is current latest, it works with everything else is same.

Additional context
Running on aws ECS, but should not have an effect on this error.

Support for other query result cache framework apart of Redis.

Is your feature request related to a problem? Please describe.
Superset supports multiple frameworks for query caching however in docker-superset only Redis based caching is available. Requires support of other caching frameworks.

Describe the solution you'd like
Files [superset_config.py and docker-entrypoint.sh] require enhancement for supporting other caching frameworks.

Python SSL error when connecting to Bigquery

Describe the bug
When I attempt to add a bigquery database connection I received the following error in the UI
ERROR: {"error": "Connection failed!\n\nThe error message returned was:\nHTTPSConnectionPool(host='oauth2.googleapis.com', port=443): Max retries exceeded with url: /token (Caused by SSLError(SSLError(\"bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)\",),))"}

Upon further investigation, I attached to the docker container as root:
docker exec -u 0 -it <id> /bin/bash and then ran python`

Terminal session:
`Python 3.6.8 (default, Mar 27 2019, 08:49:59)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.

import requests
requests.get('https://www.google.com')
From cffi callback <function _verify_callback at 0x7f5b1eccc378>:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 309, in wrapper
_lib.X509_up_ref(x509)
AttributeError: module 'lib' has no attribute 'X509_up_ref'
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 441, in wrap_socket
cnx.do_handshake()
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1915, in do_handshake
self._raise_ssl_error(self._ssl, result)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1647, in _raise_ssl_error
_raise_current_error()
File "/usr/local/lib/python3.6/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')]
...`

I found that I can successfully connect to bigquery or any other https connection via python by running this pip removal.

pip uninstall pyOpenSSL

To Reproduce
Steps to reproduce the behavior:

1. Go to add databases
2. Click add a bigquery://
3. see error message

Expected behavior
I install the docker-compose template and everything works without hackery.

I'm not sure what the solution is here, but thought I'd report it.

Options to edit Username, Email address and add Email Reporting

Thanks for the docker. Saved so much time and works without any issues!
Especially there are several issues with the original superset container like entering restarting loop with exit code 243, missing CSS styling. This worked like a charm without editing any files.

However, now I'm out of sync with the original superset documentation. Is there a way to edit username, email address of the user before/after the docker is deployed?

Also say for example, if I need features like email reporting, how to get it to work?

Support for celery beat to schedule email reports

https://superset.incubator.apache.org/installation.html#email-reports

Specifying the number of Celery workers in environment variables

Is your feature request related to a problem? Please describe.
I simply wanted to add more workers in a prod environment, so I had to edit the docker-entrypoint.sh starting script. Then rebuild the entire container because I couldn't change that script inside the container without root privilege.

Describe the solution you'd like
Inside the .env file, have a variable defined as:

# Number of workers
CELERY_WORKERS=2

which will automatically generate the number of workers instead of having it hard-coded to 2.

Describe alternatives you've considered
Manually changing docker-entrypoint.sh and rebuilding everything.

Update docker image to Superset 0.36

Is your feature request related to a problem? Please describe.

Superset 0.36 uses SMART_NUMBER when displaying graphs, but the current image does not support this data type.

Describe the solution you'd like

Update image to the latest Superset tag

Describe alternatives you've considered

Other than not use this solution? We haven't considered anything at this point.

Additional context

We just started using Superset, and we moved from the incubator after learning that it was not suitable outside of development.

Add default configurations in DockerFile for starting conatiner fron Kitematic.

Is your feature request related to a problem? Please describe.
Apart from running the Docker container through 'docker-compose' & 'docker run' container, many users start the container through some UI tool like Kitematic. Add the default configurations in the DockerFile so that container could start from Kitematic.

Accessing The Volumes

Hi.
I would like to know how to access the superset asset files.

Ansible script for setting up environment and starting docker-superset.

Is your feature request related to a problem? Please describe.
Currently, manually pulling of the image from docker-hub and then needs copying of config & compose files for starting the docker-superset container. Need an automation script for manual steps.

Describe the solution you'd like
Ansible based scripts will install docker in-case if it is not present and then will start the dockers-superset container.

Describe alternatives you've considered
The shell script can be an option but ansible way is a more clean and generic approach.

Additional context
Consider terraform based scripts for setting up the infrastructure.

Superset error sqlalchemy

Describe the bug
Superset error sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1054, "Unknown column 'dbs.allow_run_sync' in 'field list'")

To Reproduce
Steps to reproduce the behavior:

1. Go to 'SOURCES'
2. Click on 'DATABASES'
3. See error

Expected behavior
Open DATABASE connection page.

superset_config.py should not be the part of the docker image. It should be mounted in to the container.

Is your feature request related to a problem? Please describe.
Superset read configurations from superset_config.py file present in the python path. Minor change in the file results into the building of the new docker image. It should be easily modifiable.

Describe the solution you'd like
superset_config.py should not be the part of the docker image. It should be mounted into the container.

Describe alternatives you've considered
All those configurations variable can be provided through docker-compose.yml file but it requires a lot of logical processing in superset_config.py file.

Additional context
Mor configurable docker image for generalization.

Docker won't rebuild

Describe the bug

# docker-compose build
redis uses an image, skipping
mysql uses an image, skipping
superset uses an image, skipping

# docker-compose images
ERROR: no such image: sha256:13b0147efffc26b489f6aafda0370c3ef3afd936548a6cbe75ef9d84158fb7f1: No such image: sha256:13b0147efffc26b489f6aafda0370c3ef3afd936548a6cbe75ef9d84158fb7f1

I manually removed the image because the docker-compose build would not rebuild. Now the image is gone and it still won't build.

To Reproduce
Steps to reproduce the behavior:

1. docker-compose build won't do anything
2. docker rmi 5874cfce8261 will remove abhioncbr/docker-superset
3. docker-compose build still won't do anything

Expected behavior
Rebuilding the container if it is no longer present for whatever reason.

Desktop (please complete the following information):

• OS: Ubuntu Server 20.04
• Browser n/a
• Version latest

Additional context
Superset is not at the latest version. See issue #41.

This endpoint requires the datasource **, database or `all_datasource_access` permission

When trying to create a chart on data source I get error :
This endpoint requires the datasource , database or all_datasource_access permission

With this stacktrace

Traceback (most recent call last):
  File "/home/superset/superset/views/base.py", line 114, in wraps
    return f(self, *args, **kwargs)
  File "/home/superset/superset/views/core.py", line 1256, in explore_json
    samples=samples,
  File "/home/superset/superset/views/core.py", line 1169, in generate_json
    security_manager.assert_datasource_permission(viz_obj.datasource)
  File "/home/superset/superset/security.py", line 470, in assert_datasource_permission
    self.get_datasource_access_link(datasource),
superset.exceptions.SupersetSecurityException: This endpoint requires the datasource poc-test, database or
            `all_datasource_access` permission

Happens in the latest docker image and also in the version before: 0.29.0rc7

cannot import name 'Any'

I'm getting the below error while running the docker image.

Docker : docker pull abhioncbr/docker-superset:0.36.0
Command : docker run -p 8088:8088 -v config:/home/superset/config/ abhioncbr/docker-superset:0.36.0 cluster server localhost:3307 localhost:6379

Reproduce:

1. Execute the above Docker pull command
2. Then execute the Docker run command

• I have mysql and redis docker containers running in local

Customizing the container assets and pushing to docker-hub

Hi. Sorry for skipping the standard protocol. This isn't really a bug, but more of an inquiry. I wanted to know how to go about pushing an image or container to docker-hub after customizing the assets in the container. Any help/advice would be appreciated.

No such image

I'm getting a "ERROR: no such image: abhioncbr/docker-superset" on docker-compose up -d.
Is this a temporal error? I thought I've tried it before with more luck...

Connect superset docker to local postgresql database

Describe the bug
Deploying through kitematic I'm unable to connect to a postgres db running on local host. Pyscog seems to be successfully installed

Loaded your LOCAL configuration at [/home/superset/config/superset_config.py]
/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
Recognized Database Authentications.
Admin User admin created.
Loaded your LOCAL configuration at [/home/superset/config/superset_config.py]
/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.

To Reproduce
Steps to reproduce the behavior:
Add new database
Enter URI as
postgresql://username:@localhost:5432/dbname

Expected behavior
Test connection should be successful instead receive error

ERROR: {"error": "Connection failed!\n\nThe error message returned was:\n(psycopg2.OperationalError) could not connect to server: Connection refused\n\tIs the server running on host \"127.0.0.1\" and accepting\n\tTCP/IP connections on port 5432?

Db is accessible through that address

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: Desktop
• OS: Mac
• Browser Chrome
• Version 75.0.3770.142 (Official Build) (64-bit)

Additional context
Add any other context about the problem here.

Explore sysdig for monitoring and securing docker-superset container.

Sysding is a tool for monitoring and securing containers. As per Sysding project Sysdig is the first unified approach to monitor and secure containers across the entire software lifecycle.

Exploring Sysdig and leveraging it in the docker-superset would bring the project to be more enterprise readiness.

https://sysdig.com/

ENV not set?

I'm using your .env variable, but I get:

WARNING: The MYSQL_USER variable is not set. Defaulting to a blank string.
WARNING: The MYSQL_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The MYSQL_DATABASE variable is not set. Defaulting to a blank string.
WARNING: The MYSQL_ROOT_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The SUPERSET_VERSION variable is not set. Defaulting to a blank string.
WARNING: The MYSQL_HOST variable is not set. Defaulting to a blank string.
WARNING: The MYSQL_PORT variable is not set. Defaulting to a blank string.
WARNING: The REDIS_HOST variable is not set. Defaulting to a blank string.
WARNING: The REDIS_PORT variable is not set. Defaulting to a blank string.
WARNING: The SUPERSET_ENV variable is not set. Defaulting to a blank string.
WARNING: The GOOGLE_APPLICATION_CREDENTIALS variable is not set. Defaulting to a blank string.

Install Bigquey and Hive packages in image

Is your feature request related to a problem? Please describe.
for accessing BigQuery and Hive databases through Superset, packages needs to be installed.

Describe the solution you'd like
pip install pybigquey pyhive

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Minor update in Dokcerfile.