opencve/opencve-docker

Docker Image packaging for OpenCVE

Important

The OpenCVE v1 development is freezed as the v2 will be released soon.

Apart from using other technologies (Django & Airflow), this new version will provide new features like organizations & projects, advanced notifications or usage of multiple CVEs providers to cross the information (CVSS, CPEs, References...).

For more information, included what will change and some ETAs, you can read this blog post.

---

---

Try for free OpenCVE on https://www.opencve.io or check documentation to install it yourself.

What is OpenCVE

OpenCVE is a platform used to locally import the list of CVEs and perform searches on it (by vendors, products, CVSS, CWE...).

Users subscribe to vendors or products, and OpenCVE alerts them when a new CVE is created or when an update is done in an existing CVE.

Features

• Explore the CVE database and filter the results by Vendor, Product, CVSS score or CWE
• Subscribe to vendors and products extracted from CPE
• Be notified for new CVE and for CVE updates based on subscriptions
• Analyse all CVE details (vendors, products, CVSS score, CPE, CWE, References...)
• Create custom tags (unread, important, devteam...) to organize the CVE list
• Display the history of each CVE and see their last changes in the homepage
• Retrieve the last changes in custom reports
• Customize the notification settings (frequency, filter by CVSS score...)
• Integrate OpenCVE with your own tools using the Rest API

You can use for free these features on the public instance: https://www.opencve.io.

How does it work

OpenCVE uses the JSON feed provided by the NVD to update the local list of CVEs.

After an initial import, a background task is regularly executed to synchronize the local copy with the NVD feed. If a new CVE is added, or if a change is detected, the subscribers of the related vendors and products are alerted.

Read the How It Works guide to learn in details how OpenCVE works.

Requirements

OpenCVE works with Python >=3.7.

It uses the JSONB feature for performance, so you will need a PostgreSQL instance to store the data (CVE, Users, Vendors, Products, Subscriptions, ...). Other engines are not supported.

The pg_trgm module of PostgreSQL is required to let you search in the CVEs list. The upgrade-db command will enable it for you, but you can also do it yourself if you prefer (CREATE EXTENSION pg_trgm). From PostgreSQL 13 this module is considered as trusted, meaning it can be installed by non-superusers with the CREATE privilege.

Celery is used to periodically fetch the NVD database and update the list of CVEs. For that you will need a broker : we recommend you Redis for the ease of installation. Futhermore it is possible that future versions of OpenCVE will use a cache feature, in that case the Redis requirement will already be filled for you.

During the import of initial data OpenCVE will download and parse huge files, like the CPE dictionnary. For that we recommend you 3.5G RAM at least.

Installation

We provide 2 methods to install OpenCVE :

• manual installation
• docker installation

The second method can be useful if you don't want to manage the dependencies (like PostgreSQL, Redis or Celery).

Check these documentations for the details of each step (initial import, admin creation, etc).

opencve-docker's Issues

Updates management

Hello,

Before to use in production, I would like to know how to manage updates and keep data in databases.

Thanks.

docker images not pushed to docker hub

Expected Behavior

New releases should be listed as tags on docker hub. https://hub.docker.com/u/opencve

Actual Behavior

No tags are shown.

Steps to Reproduce the Problem

See https://hub.docker.com/u/opencve

Specifications

Screenshots (optional)

how update code and update database

how update code and update database

Probable bug while creating users

Expected Behavior

Creating a user with following command: docker exec -it webserver opencve create-user user [email protected]

Actual Behavior

docker exec -it webserver opencve create-user user [email protected]          
Password:                                                                                                               
Repeat for confirmation:                                                                                                
(trapped) error reading bcrypt version                                                                                  
Traceback (most recent call last):                                                                                        
File "/app/venv/lib/python3.8/site-packages/passlib/handlers/bcrypt.py", line 620, in _load_backend_mixin                 version = _bcrypt.__about__.__version__                                                                             
AttributeError: module 'bcrypt' has no attribute '__about__'                                                            
[*] User user created.      

But I was able to login with the created admin user.

Steps to Reproduce the Problem

• Following this document for installation of opencve-docker: https://docs.opencve.io/installation/docker/
• Create an admin

Specifications

• OpenCVE version: 1.5.0
• Platform: Ubuntu 22.04 server
• Docker version: 26.1.1
• Docker-compose version: 1.29.2

Screenshots (optional)

After upgrade to V1.4.0: Error 111 connecting to 127.0.0.1:6379. Connection refused.

Expected Behavior

Working environment after upgrade to V1.4.0

Actual Behavior

Exception and no reports on vulnerabilities

docker logs celery_worker

warnings.warn(RuntimeWarning(ROOT_DISCOURAGED.format(
[2022-11-06 14:57:22,619: ERROR/ForkPoolWorker-1] Task CVE_UPDATES[445a6120-06c2-43e0-b289-b0c1184101fd] raised unexpected: ConnectionError('Error 111 connecting to 127.0.0.1:6379. Connection refused.')
Traceback (most recent call last):
File "/app/venv/lib/python3.8/site-packages/redis/connection.py", line 539, in connect
sock = self._connect()
File "/app/venv/lib/python3.8/site-packages/redis/connection.py", line 596, in _connect
raise err
File "/app/venv/lib/python3.8/site-packages/redis/connection.py", line 584, in _connect
sock.connect(socket_address)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/venv/lib/python3.8/site-packages/celery/app/trace.py", line 385, in trace_task
R = retval = fun(*args, **kwargs)
File "/app/venv/lib/python3.8/site-packages/celery/app/trace.py", line 648, in protected_call
return self.run(*args, **kwargs)
File "/app/venv/lib/python3.8/site-packages/opencve/tasks/init.py", line 47, in cve_updates
acquired = Lock(r, "cve_updates_lock", timeout=3600 * 12, blocking=False).acquire()
File "/app/venv/lib/python3.8/site-packages/redis/lock.py", line 182, in acquire
if self.do_acquire(token):
File "/app/venv/lib/python3.8/site-packages/redis/lock.py", line 197, in do_acquire
if self.redis.set(self.name, token, nx=True, px=timeout):
File "/app/venv/lib/python3.8/site-packages/redis/client.py", line 1519, in set
return self.execute_command('SET', *pieces)
File "/app/venv/lib/python3.8/site-packages/redis/client.py", line 836, in execute_command
conn = self.connection or pool.get_connection(command_name, **options)
File "/app/venv/lib/python3.8/site-packages/redis/connection.py", line 1073, in get_connection
connection.connect()
File "/app/venv/lib/python3.8/site-packages/redis/connection.py", line 544, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:6379. Connection refused.

worker: Warm shutdown (MainProcess)

Steps to Reproduce the Problem

Check if there is no updates ongoing on the celery_worker:
$ docker logs celery_worker

If or when there is no update ongoing:
$ docker stop celery_beat celery_worker webserver

Update opencve-docker repository:
$ git pull --prune

Update your opencve.cfg file with the changes
$ vi conf/opencve.cfg

Check that you are on the correct version:
$ grep OPENCVE_VERSION .env
OPENCVE_VERSION=1.4.0

Build the new docker images:
$ docker-compose build

Start the webserver & celery_worker:
$ docker-compose up -d webserver celery_worker

Upgrade the database schema:
$ docker exec -it webserver opencve upgrade-db

Now you are good to go, you can start the beat:
$ docker-compose up -d celery_beat

Specifications

• OpenCVE version: 1.3.0
• Platform: Ubuntu 20.04.5 LTS
• Docker version: 20.10.19
• Docker-compose version: 3.4

Screenshots (optional)

Certificate adding

Hello,

What is the best way to add a certificate for the frontend component.

Thanks.

Error with JSMIN even with JSMIN 3 installed

Expected Behavior

Platform to install with requirements installed per stated with versions. Using python and pip even pinned the version. Every time I run the docker it pulls jsmin 2.2.2 even though I put 3.0.1 and syste fails to build. Running on Ubuntu 20.04 LTS with all updates.

Actual Behavior

Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'error'
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [1 lines of output]
error in jsmin setup command: use_2to3 is invalid.
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
The command '/bin/sh -c python3 -m pip install /opencve/' returned a non-zero code: 1
ERROR: Service 'webserver' failed to build : Build failed

Steps to Reproduce the Problem

Install postgres, docker, docker-compose, python3 pip3 and pip tried, pip install new requiremts list in requirements.txt then docker-compose build pointing to yml.
It fails every time. I have tried everything I can think of and continues to fail.

Specifications

• OpenCVE version: Latest as of 5/12/2023
• Platform:Ubuntu 20.04
• Docker version: 20.10.21
• Docker-compose version:1.29.2

Screenshots (optional)

Downloading SQLAlchemy-Utils-0.33.11.tar.gz (128 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 128.0/128.0 kB 19.5 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting psycopg2-binary==2.8.4 (from opencve==1.4.0)
Downloading psycopg2_binary-2.8.4-cp38-cp38-manylinux1_x86_64.whl (3.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.0/3.0 MB 30.5 MB/s eta 0:00:00
Collecting Redis==3.3.11 (from opencve==1.4.0)
Downloading redis-3.3.11-py2.py3-none-any.whl (66 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.7/66.7 kB 15.8 MB/s eta 0:00:00
Collecting arrow==0.13.1 (from opencve==1.4.0)
Downloading arrow-0.13.1-py2.py3-none-any.whl (36 kB)
Collecting celery==4.3.0 (from opencve==1.4.0)
Downloading celery-4.3.0-py2.py3-none-any.whl (413 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 413.6/413.6 kB 28.6 MB/s eta 0:00:00
Collecting cpe==1.2.1 (from opencve==1.4.0)
Downloading cpe-1.2.1.tar.gz (2.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 31.1 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting untangle==1.1.1 (from opencve==1.4.0)
Downloading untangle-1.1.1.tar.gz (3.1 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting cssmin==0.2.0 (from opencve==1.4.0)
Downloading cssmin-0.2.0.tar.gz (3.2 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting jsmin==2.2.2 (from opencve==1.4.0)
Downloading jsmin-2.2.2.tar.gz (12 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'error'
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [1 lines of output]
error in jsmin setup command: use_2to3 is invalid.
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
The command '/bin/sh -c python3 -m pip install /opencve/' returned a non-zero code: 1
ERROR: Service 'webserver' failed to build : Build failed

getting error in webserver build

Building webserver
unknown flag: --iidfile

sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL: password authentication failed for user "opencve"

$ docker-compose up -d postgres redis webserver celery_worker
is ok

 $ docker exec -it webserver opencve upgrade-db
docker exec -it webserver opencve upgrade-db
Traceback (most recent call last):
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
    return fn()
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 294, in unique_connection
    return _ConnectionFairy._checkout(self)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
    rec = pool._do_get()
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
    return self._create_connection()
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
    return _ConnectionRecord(self)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 428, in __init__
    self.__connect(first_connect_check=True)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
    connection = pool._invoke_creator(self)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
    return dialect.connect(*cargs, **cparams)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 453, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/app/venv/lib/python3.7/site-packages/psycopg2/__init__.py", line 126, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: FATAL:  password authentication failed for user "opencve"


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/venv/bin/opencve", line 11, in <module>
    load_entry_point('opencve==1.2.3', 'console_scripts', 'opencve')()
  File "/app/venv/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/app/venv/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/app/venv/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/app/venv/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/app/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/app/venv/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/app/venv/lib/python3.7/site-packages/opencve/commands/__init__.py", line 39, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/app/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/app/venv/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/app/venv/lib/python3.7/site-packages/flask/cli.py", line 412, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/app/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/app/venv/lib/python3.7/site-packages/opencve/commands/upgrade_db.py", line 16, in upgrade_db
    upgrade(directory=str(migrations_path))
  File "/app/venv/lib/python3.7/site-packages/flask_migrate/__init__.py", line 92, in wrapped
    f(*args, **kwargs)
  File "/app/venv/lib/python3.7/site-packages/flask_migrate/__init__.py", line 162, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/app/venv/lib/python3.7/site-packages/alembic/command.py", line 320, in upgrade
    script.run_env()
  File "/app/venv/lib/python3.7/site-packages/alembic/script/base.py", line 563, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/app/venv/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file
    module = load_module_py(module_id, path)
  File "/app/venv/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/app/venv/lib/python3.7/site-packages/opencve/migrations/env.py", line 94, in <module>
    run_migrations_online()
  File "/app/venv/lib/python3.7/site-packages/opencve/migrations/env.py", line 76, in run_migrations_online
    connection = engine.connect()
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2193, in connect
    return self._connection_cls(self, **kwargs)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 103, in __init__
    else engine.raw_connection()
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2293, in raw_connection
    self.pool.unique_connection, _connection
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
    e, dialect, self
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
    raise value.with_traceback(tb)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
    return fn()
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 294, in unique_connection
    return _ConnectionFairy._checkout(self)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
    rec = pool._do_get()
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
    return self._create_connection()
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
    return _ConnectionRecord(self)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 428, in __init__
    self.__connect(first_connect_check=True)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
    connection = pool._invoke_creator(self)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
    return dialect.connect(*cargs, **cparams)
  File "/app/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 453, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/app/venv/lib/python3.7/site-packages/psycopg2/__init__.py", line 126, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL:  password authentication failed for user "opencve"

(Background on this error at: http://sqlalche.me/e/e3q8)

Correct update/upgrade procedure and commands

Hi!

I am looking for a generic way to update from one release to another and want to make sure that the following approach and commands are correct (taken from https://github.com/opencve/opencve-docker/releases/tag/v1.2.0):

Check if there is no updates ongoing on the celery_worker:
$ docker logs celery_worker

If or when there is no update ongoing:
$ docker stop celery_beat celery_worker webserver

Update opencve-docker repository:
$ git pull --prune

Update your opencve.cfg file with the changes
$ vi conf/opencve.cfg

Check that you are on the correct version:
$ grep OPENCVE_VERSION .env

OPENCVE_VERSION=x.y.z

Build the new docker images:
$ docker-compose build

Start the webserver & celery_worker:
$ docker-compose up -d webserver celery_worker

Upgrade the database schema:
$ docker exec -it webserver opencve upgrade-db

INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.runtime.migration] Running upgrade f81abceece3d -> 33cd640e1112, Add indexes on cves.summary and cve.cve_id
INFO [alembic.runtime.migration] Running upgrade 33cd640e1112 -> 4195eeb432e9, Add tags tables

Now you are good to go, you can start the beat:
$ docker-compose up -d celery_beat

If something is wrong, I kindly ask for advise.

If the procedure is correct, please confirm. It may then make sense to add it to the documentation (https://docs.opencve.io/installation/docker/).

Many thanks and best regards,
Dago

celery_worker problem with updating feeds

Expected Behavior

celery_worker should retrieve information from external sources and update it in the database.

Actual Behavior

celery_worker reports errors while processing NVD NIST CVE list

Steps to Reproduce the Problem

Installation performed on February 1, 2022 according to official documentation - version 1.3.0

Logs of celery_worker container:
[2022-02-02 09:42:16,648: INFO/ForkPoolWorker-2] HANDLE_EVENTS[4474cf5e-3be2-4a54-90aa-991f7893fb99]: Checking for new events...
[2022-02-02 09:42:16,649: INFO/ForkPoolWorker-2] HANDLE_EVENTS[4474cf5e-3be2-4a54-90aa-991f7893fb99]: Downloading https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta...
[2022-02-02 09:42:17,184: ERROR/ForkPoolWorker-2] Task HANDLE_EVENTS[4474cf5e-3be2-4a54-90aa-991f7893fb99] raised unexpected: AttributeError("'NoneType' object has no attribute 'value'")
Traceback (most recent call last):
File "/app/venv/lib/python3.7/site-packages/celery/app/trace.py", line 385, in trace_task
R = retval = fun(*args, **kwargs)
File "/app/venv/lib/python3.7/site-packages/celery/app/trace.py", line 648, in protected_call
return self.run(*args, **kwargs)
File "/app/venv/lib/python3.7/site-packages/opencve/tasks/events.py", line 94, in handle_events
current_sum, new_sum = has_changed()
File "/app/venv/lib/python3.7/site-packages/opencve/tasks/events.py", line 33, in has_changed
if nvd_sha256 != last_nvd256.value:
AttributeError: 'NoneType' object has no attribute 'value'

Specifications

• OpenCVE version: 1.3.0

• Platform: CentOS 7 64bit

• Docker version:
  Version: 20.10.12
  API version: 1.41 (minimum version 1.12)
  Go version: go1.16.12
  Git commit: 459d0df
  Built: Mon Dec 13 11:44:05 2021
  OS/Arch: linux/amd64
  Experimental: false

• Docker-compose version:
  docker-compose version 1.18.0, build 8dd22a9
  docker-py version: 2.6.1
  CPython version: 3.6.8
  OpenSSL version: OpenSSL 1.0.2k-fips 26 Jan 2017

Screenshots (optional)

Docker configuration for SSL reverse proxy usage.

Context

Hi.

I'm in the process of switching the web access to https using a reverse proxy on the host on which the dockers are instantiated.
The reverse proxy, using nginx, is configured to map the host public ip address (for now) as follows:

        location /opencve {
                include proxy_params;
                proxy_pass http://localhost:8000;
        }

The opencve.cfg file is modified so that the server_name variable is the ip.ad.dr.es/opencve and set use_reverse_proxy to True. I didn't change the Dockerfile nor the docker-compose.yml file regarding to the launch of the web server command.

Expected Behavior

I expected the service would be usable over https.

Actual Behavior

A 404 error page is displayed. Changing the server_name to that of the public address and port makes opencve reachable but without being proxyfied.

Steps to Reproduce the Problem

• install opencve using dockers with the mentioned configuration
• configure nginx on the host to be a reverse proxy for opencve
• try and access the https port on the host.

Specifications

• OpenCVE version: 1.2.3
• Platform: debian 11
• Docker version: 20.10.5+dfsg1-1+b5
• Docker-compose version: 1.25.0-1

Screenshots (optional)

Changing the Server name creates problems

Expected Behavior

After changing the server_name tag in "conf/opencve.cfg" to x.x.x.x:80 and the command: to "webserver -b x.x.x.x:80" in the docker-compose.yml file, I expected to now reach the webserver over my chosen IP-adrese.

Actual Behavior

The website isn't accessible at all. The webserer container restarts every 5 seconds.

Steps to Reproduce the Problem

• conf/opencve.cfg
  • server_name = x.x.x.x:80
• docker-compose.yml
  • command: webserver -b x.x.x.x:80
  • ${OPENCVE_PORT:-8000}:80

Specifications

• OpenCVE version: 3.4
• Platform: Ubuntu 20.04 Server
• Docker version: 20.10.8
• Docker-compose version: 1.25.0

Screenshots (optional)

400 Bad Request

Expected Behavior

An account validation address in the format https://FQDN sent by email

Actual Behavior

I use OpenCVE behind a nginx reverse proxy. When registering on OpenCVE, the confirmation link sent is of the form: http://FQDN:443 which induces an error 400. Is it possible to change the format of the validation address in order to have https directly?

Specifications

• OpenCVE version: v1.3.0
• Platform: Ubuntu 20.04
• Docker version: 20.10.12
• Docker-compose version: 1.25.0

NIST NVD shutting down v1.1 APIs

Sometime in September of this year, NIST will be shutting down its legacy Version 1.1 for API requests.

Here's the URL: https://nvd.nist.gov/vuln/data-feeds

Upgrade from 1.3.0 to 1.4.0 - Lessons learned and update procedure

Hello, here are my lessons learned and procedure I have experienced when updating my docker instance from 1.3.0 to 1.4.0 Maybe someone else finds this usefull...

In general, I followed the procedure mentioned in the release 1.4.0 .

Learnings:

1. Always make a snapshot before starting any update process (e.g. via VMware).
2. Stop all containers with docker-compose down, because after the upgrade procedure I had a situation where all data was lost in openCVE and a warning regarding postgres container showed up on startup (cannot remember and did not safe). When postgres container was stopped before the build, the data was kept. Therefore, $ docker stop celery_beat celery_worker webserver postgres or docker-compose down
3. Check the opencve.cfg from the opencve repository for changes, as the example from the opencve-docker repository is outdated. Most important was to add celery_lock_url = redis://redis:6379/2 in order to avoid the error mentioned here.

Finally, my openCVE docker instance is up and running in version 1.4.0 without errors, all the data and new features. Many thanks for your amazing work and your responsiveness to tickets and problems @ncrocfer @ldurnez and all others involved in this great project.

Here is the exact procedure I have used with success:

Check if there is no updates ongoing on the celery_worker:
$ docker logs celery_worker

If or when there is no update ongoing:
$ docker stop celery_beat celery_worker webserver
$ docker-compose down

Update opencve-docker repository:
$ git pull --prune

Update the opencve.cfg file with the changes of the most recent version here
$ nano conf/opencve.cfg

Check that you are on the correct version:
$ grep OPENCVE_VERSION .env
OPENCVE_VERSION=1.4.0

Add POSTGRES_PASSWORD to .env file to the same password in conf/opencve.cfg (this is required to do before docker build!)
$ nano .env -> POSTGRES_PASSWORD=same password in conf/opencve.cfg

Build the new docker images:
$ docker-compose build

Start the webserver & celery_worker:
$ docker-compose up -d webserver celery_worker

Upgrade the database schema:
$ docker exec -it webserver opencve upgrade-db

Now you are good to go, you can start the beat:
$ docker-compose up -d celery_beat

Check how the system is doing (15-30mins)
$ docker logs -f celery_worker

providing a docker image on github

Is your feature request related to a problem?
It is not a fundamental problem, but rather an unnecessary expense if the image is built locally. An image built at the source takes current changes into account, avoids transfer errors and offers standardized version and release management. Our biggest problem is that we are currently unable to draw an image directly and our local provisioning process is disrupted.

Do you have a solution in mind, or a suggestion to improve OpenCVE-Docker?
A simple CI/CD with an image in the github internal repository and a possible push into the docker hub would be great. Especially since it was or is already represented there.

Additional comment
Link to the empty docker hub https://hub.docker.com/r/opencve/opencve

Issue with file endings

I had a Line endings problem on the current version when i git clone to a windos machine.
Error on docker run was exec ./run.sh: no such file or directory

Maybe it's good to add a fixed ending with git attributes?

.gitattributes

**.sh text eol=lf

pg_config executable not found

#0 39.46 Collecting psycopg2-binary==2.8.4
#0 39.60   Downloading psycopg2-binary-2.8.4.tar.gz (378 kB)
#0 40.40      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 378.7/378.7 kB 471.7 kB/s eta 0:00:00
#0 40.42   Preparing metadata (setup.py): started
#0 40.52   Preparing metadata (setup.py): finished with status 'error'
#0 40.53   error: subprocess-exited-with-error
#0 40.53   
#0 40.53   × python setup.py egg_info did not run successfully.
#0 40.53   │ exit code: 1
#0 40.53   ╰─> [23 lines of output]
#0 40.53       running egg_info
#0 40.53       creating /tmp/pip-pip-egg-info-wtffupsx/psycopg2_binary.egg-info
#0 40.53       writing /tmp/pip-pip-egg-info-wtffupsx/psycopg2_binary.egg-info/PKG-INFO
#0 40.53       writing dependency_links to /tmp/pip-pip-egg-info-wtffupsx/psycopg2_binary.egg-info/dependency_links.txt
#0 40.53       writing top-level names to /tmp/pip-pip-egg-info-wtffupsx/psycopg2_binary.egg-info/top_level.txt
#0 40.53       writing manifest file '/tmp/pip-pip-egg-info-wtffupsx/psycopg2_binary.egg-info/SOURCES.txt'
#0 40.53       
#0 40.53       Error: pg_config executable not found.
#0 40.53       
#0 40.53       pg_config is required to build psycopg2 from source.  Please add the directory
#0 40.53       containing pg_config to the $PATH or specify the full executable path with the
#0 40.53       option:
#0 40.53       
#0 40.53           python setup.py build_ext --pg-config /path/to/pg_config build ...
#0 40.53       
#0 40.53       or with the pg_config option in 'setup.cfg'.
#0 40.53       
#0 40.53       If you prefer to avoid building psycopg2 from source, please install the PyPI
#0 40.53       'psycopg2-binary' package instead.
#0 40.53       
#0 40.53       For further information please check the 'doc/src/install.rst' file (also at
#0 40.53       <http://initd.org/psycopg/docs/install.html>).
#0 40.53       
#0 40.53       [end of output]
#0 40.53   
#0 40.53   note: This error originates from a subprocess, and is likely not a problem with pip.
#0 40.53 error: metadata-generation-failed
#0 40.53 
#0 40.53 × Encountered error while generating package metadata.
#0 40.53 ╰─> See above for output.
#0 40.53 
#0 40.53 note: This is an issue with the package mentioned above, not pip.
#0 40.53 hint: See above for details.
------
failed to solve: process "/bin/sh -c python3 -m pip install /opencve/" did not complete successfully: exit code: 1

Error: pg_config executable not found. when Downloading psycopg2-binary-2.8.4.tar.gz on docker compose

Expected Behavior

Go to the next step docker-compose up -d postgres redis webserver celery_worker

Actual Behavior

docker compose was unsuccessful due to Error: pg_config executable not found.

Despite psycopg2 was installed and version 2.9.6.

Steps to Reproduce the Problem

following Docker installation, reach and type below command
$docker-compose build

Build will cease when it reaches Downloading psycopg2-binary-2.8.4.tar.gz. There will be an error as shown below

Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'error'

error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [35 lines of output]
running egg_info
writing psycopg2_binary.egg-info/PKG-INFO
writing dependency_links to psycopg2_binary.egg-info/dependency_links.txt
writing top-level names to psycopg2_binary.egg-info/top_level.txt
/tmp/pip-build-env-5dzbzw8b/overlay/lib/python3.8/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in setup.cfg
!!

          ********************************************************************************
          The license_file parameter is deprecated, use license_files instead.
  
          By 2023-Oct-30, you need to update your project and remove deprecated calls
          or your builds will no longer be supported.
  
          See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
          ********************************************************************************
  
  !!
    parsed = self.parsers.get(option_name, lambda x: x)(value)
  
  Error: pg_config executable not found.
  
  pg_config is required to build psycopg2 from source.  Please add the directory
  containing pg_config to the $PATH or specify the full executable path with the
  option:
  
      python setup.py build_ext --pg-config /path/to/pg_config build ...
  
  or with the pg_config option in 'setup.cfg'.
  
  If you prefer to avoid building psycopg2 from source, please install the PyPI
  'psycopg2-binary' package instead.
  
  For further information please check the 'doc/src/install.rst' file (also at
  <http://initd.org/psycopg/docs/install.html>).
  
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR:

Specifications

• OpenCVE version: pulled from Docker, 1.4.0
• Platform: ubuntu 20.04.2 ARM64
• Docker version: 23.0.5
• Docker-compose version: 1.25.0

Screenshots (optional)

bitnami opencve docker image

I've created this ticket today. Hope it will be accepted.

bitnami/containers#9473

Unable to create swapfile

Expected Behavior

As explained in the docs, swapfile should be created with no problem in the webserver container.

Actual Behavior

swapon /swapfile throws an Operation not permitted error. When checking perms by doing a swapoff -a system indicates that we are not a superuser. This kind of behavior is usual in a VPS environment when using a specific OS that limits actions, but should not happen in a container. Though I'm using Podman so I'm not sure if this is related to it, but this is unlikely.

Steps to Reproduce the Problem

• Build the containers using the steps described in the docs.
• Start the containers using the steps described in the docs.
• Open a shell in the webserver container and use the steps to create a swapfile in the docs.

Specifications

• OpenCVE version: Latest
• Platform: Windows Server 2022 (using WSL 2 Ubuntu 22.04.1 LTS)
• Docker version: Using Podman v4.9.0
• Docker-compose version: v2.24.3

[error] Configuration not found (use the 'init' command)

Expected Behavior

• webserver container will start with no issues

Actual Behavior

• webserver is fialing to start. Getting the error "[error] Configuration not found (use the 'init' command)"

Steps to Reproduce the Problem

• Run docker compose up -d
• Run docker ps -a. webserver container is showing Restarting

Specifications

• OpenCVE version: 1.5.0
• Platform: RHEL 7
• Docker version: Docker version 25.0.3, build 4debf41
• Docker-compose version: Docker Compose version v2.24.6

Screenshots (optional)

Vendor list not populated

Expected Behavior

When installing a new docker instance, data is imported and CVE can be listed. The vendor should be browsable.

Actual Behavior

No vendor can be found.

Steps to Reproduce the Problem

• install a new docker instance
• let the process populate the db
• list CVE, list vendors

Specifications

• OpenCVE version: the latest one grabbed by opencve-docker.
• Platform: Linux/Docker
• Docker version: 20.10.5+dfsg1-1+b5
• Docker-compose version: 1.25.0-1

Screenshots (optional)

error in jsmin setup command: use_2to3 is invalid

Expected Behavior

building the image works without any problems

Actual Behavior

building the image throws the following error:

#0 31.48 Collecting jsmin==2.2.2 (from opencve==1.3.0)
#0 31.49   Downloading jsmin-2.2.2.tar.gz (12 kB)
#0 31.51   Installing build dependencies: started
#0 33.44   Installing build dependencies: finished with status 'done'
#0 33.44   Getting requirements to build wheel: started
#0 33.64   Getting requirements to build wheel: finished with status 'error'
#0 33.65   error: subprocess-exited-with-error
#0 33.65
#0 33.65   × Getting requirements to build wheel did not run successfully.
#0 33.65   │ exit code: 1
#0 33.65   ╰─> [1 lines of output]
#0 33.65       error in jsmin setup command: use_2to3 is invalid.
#0 33.65       [end of output]
#0 33.65
#0 33.65   note: This error originates from a subprocess, and is likely not a problem with pip.
#0 33.66 error: subprocess-exited-with-error
#0 33.66
#0 33.66 × Getting requirements to build wheel did not run successfully.
#0 33.66 │ exit code: 1
#0 33.66 ╰─> See above for output.
#0 33.66
#0 33.66 note: This error originates from a subprocess, and is likely not a problem with pip.

Steps to Reproduce the Problem

• docker-compose build

Specifications

• OpenCVE version: 1.4.0
• Platform: ubuntu 20.04 LTS
• Docker version: 23.0.4
• Docker-compose version: 1.29.2

Screenshots (optional)

CVE database is not updated

Expected Behavior

According to the documentation, the database of CVEs and other information is supposed to be updated every 15 minutes using Celery Beat and Celery Worker.

Actual Behavior

The celery containers are running, but no tasks are being executed.

Steps to Reproduce the Problem

• clone this repo and follow the installation instructions

Specifications

• OpenCVE version: latest
• Platform: Linux (CentOS 7)
• Docker version: 20.10.6
• Docker-compose version: 1.29.1

CVEs in NVD and openCVE out of sync - manual re-import?

Hello, because my instance was not working for some time and I started again from a recent snapshot, my instance missed some CVEs.

docker exec -it webserver opencve import-data leads to [*] Import already done.

Is there a way to re-import the data manually? What would be the commands to have NVD and openCVE in sync again?

Thanks!

I wrote a helm chart to deploy opencve to kubernetes

Is your feature request related to a problem?
I wanted to deploy opencve into k8s, but did not find a helm chart.

Do you have a solution in mind, or a suggestion to improve OpenCVE-Docker?
You can find my helm-chart here: https://github.com/D3N14L/opencve-helm
The status could be described as "early alpha", but I would be happy to donate it to opencve and also help with the further development.

Additional comment
I would be happy if you could provide feedback on whether you could help with or want to take over the further development.