cerndocumentserver / cds-videos Goto Github PK
View Code? Open in Web Editor NEWAccess articles, reports and multimedia content in HEP
Home Page: https://videos.cern.ch
License: GNU General Public License v2.0
Access articles, reports and multimedia content in HEP
Home Page: https://videos.cern.ch
License: GNU General Public License v2.0
$ npm install -g node-sass clean-css requirejs uglify-js
apt-get install npm
), duh!sudo ln -s
which nodejs/usr/bin/node
)$ mkvirtualenv cds3
pip install virtualenv
)pip install virtualenvwrapper
)(cds3) $ pip install -e .
.whl
files for a package, it will compile its C component and try to find the required lib header files. Only debian linux distributions split lib-dev packages (for header files) from lib, therefore the following packages need to be individually installed.
I tried to install cds from docker and I found some issues with ElasticSearch
configuration:
/tmp
directory is not writable: JNA Warning: IOException removing temporary files: JNA temporary directory '/tmp' is not writable
ElasticSearch
:worker_1 | Test ES_PORT...tcp://172.17.0.148:9200...NO
web_1 | Test ES_PORT...tcp://172.17.0.148:9200...NO
legacy
Sometimes the text in the facet is too long and breaks the alignment, it would be nice to find a solution to that
We might also want to evaluate if include/exclude functionality should be added
After changing the basic image in docker from python:3.5
to python:3.5-slim
, docker-compose build
is failing because of:
gcc -pthread -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -Isrc/lxml/includes -I/usr/local/include/python3.5m -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-3.5/src/lxml/lxml.etree.o -w
In file included from src/lxml/lxml.etree.c:320:0:
src/lxml/includes/etree_defs.h:14:31: fatal error: libxml/xmlversion.h: No such file or directory
#include "libxml/xmlversion.h"
^
compilation terminated.
Compile failed: command 'gcc' failed with exit status 1
creating tmp
cc -I/usr/include/libxml2 -c /tmp/xmlXPathInitmhwo1u2u.c -o tmp/xmlXPathInitmhwo1u2u.o
/tmp/xmlXPathInitmhwo1u2u.c:1:26: fatal error: libxml/xpath.h: No such file or directory
#include "libxml/xpath.h"
^
compilation terminated.
*********************************************************************************
Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
*********************************************************************************
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/usr/local/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-9aj2a62m/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-ud9ohti0-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-9aj2a62m/lxml/
After changing the image back to python:3.5
it works.
After manually checking the intermediate image, it seems that lxml
is missing the libxml2-dev
package.
@ddaze, did you have issues with that ?
Generate ElasticSearch
schema for the records.
cds
has access to this path !)supervisorctl
command to manage services)index create
and index destroy
tasksRequest EOS space for cdslabs
.
I was wondering if there is an easy way to add a user-specific file with configuration for docker that would specify additional packages to be installed and some aliases/settings to be added to bashrc
?
For example, I would like to say that I want to have:
aliases: ll = ls -al
and cd ..=..
packages: vim, less, etc.
(vim
should probably go the list of packages by default :)
and someone else would like to have his own aliases/packages, so this settings would stay outside of git.
The most convenient way I see this is to point docker to a location on my machine (for example to: ~/.aliases
and ~/.docker_packages
) and make docker process those two files if they exist.
If they don't exist, simply do nothing.
@ddaze any ideas ?
Enable invenio-files-rest
on our app.
Processing
only if all videos have been uploaded)project
record type is valid
url
celery task (0.2w) @omelkonianinvenio-previewer
preview videos with a configurable player (2w) @sourabhlal #186cli
for cds (ex. cds conversion restart all
)ffmpeg
(?) #191High
format has an error.) (2d)Enable invenio-previewer
on detail view to preview listed files.
Bootstrap 4 will support only sass.
https://github.com/CERNDocumentServer/cds/blob/cdslabs/cds/config.py#L279
2016-06-07 11:24:10,216 ERROR: Exception on /login/ [GET] [in /opt/cds/lib/python2.7/site-packages/flask/app.py:1423]
Traceback (most recent call last):
File "/opt/cds/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/opt/cds/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/opt/cds/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/opt/cds/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/opt/cds/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/opt/cds/lib/python2.7/site-packages/flask_security/decorators.py", line 225, in wrapper
return f(*args, **kwargs)
File "/opt/cds/lib/python2.7/site-packages/flask_security/views.py", line 87, in login
**_ctx('login'))
File "/opt/cds/lib/python2.7/site-packages/flask_security/core.py", line 447, in render_template
return render_template(*args, **kwargs)
File "/opt/cds/lib/python2.7/site-packages/flask/templating.py", line 128, in render_template
context, ctx.app)
File "/opt/cds/lib/python2.7/site-packages/flask/templating.py", line 110, in _render
rv = template.render(context)
File "/opt/cds/lib/python2.7/site-packages/jinja2/environment.py", line 989, in render
return self.environment.handle_exception(exc_info, True)
File "/opt/cds/lib/python2.7/site-packages/jinja2/environment.py", line 754, in handle_exception
reraise(exc_type, exc_value, tb)
File "/opt/cds/lib/python2.7/site-packages/invenio_oauthclient/views/../templates/invenio_oauthclient/login_user.html", line 27, in top-level template code
{%- from "invenio_oauthclient/_macros.html" import oauth_button %}
File "/opt/cds/lib/python2.7/site-packages/invenio_accounts/templates/invenio_accounts/login_user.html", line 27, in top-level template code
{% from "invenio_accounts/_macros.html" import render_field, form_errors %}
File "/opt/cds/lib/python2.7/site-packages/invenio_theme/templates/invenio_theme/page_cover.html", line 28, in top-level template code
{%- set body_css_classes=['cover-page'] %}
File "/opt/cds/src/cds/cds/modules/theme/templates/cds_theme/page.html", line 3, in top-level template code
{%- set title = '%s | %s'|format(_(title), config.THEME_SITENAME) or 'CDS' -%}
File "/opt/cds/lib/python2.7/site-packages/invenio_theme/templates/invenio_theme/page.html", line 73, in top-level template code
{%- block body %}
File "/opt/cds/lib/python2.7/site-packages/invenio_theme/templates/invenio_theme/page_cover.html", line 41, in block "body"
{% block page_header %}
File "/opt/cds/lib/python2.7/site-packages/invenio_theme/templates/invenio_theme/page_cover.html", line 45, in block "page_header"
{%- block brand %}
File "/opt/cds/lib/python2.7/site-packages/invenio_theme/templates/invenio_theme/page_cover.html", line 46, in block "brand"
<a href="/"><img class="text-center masthead-brand" src="{{ url_for('static', filename=config.THEME_LOGO)}}" alt="{{_(config.THEME_SITENAME)}}" width="250px"/></a>
File "/opt/cds/lib/python2.7/site-packages/flask/helpers.py", line 312, in url_for
return appctx.app.handle_url_build_error(error, endpoint, values)
File "/opt/cds/lib/python2.7/site-packages/flask/app.py", line 1641, in handle_url_build_error
reraise(exc_type, exc_value, tb)
File "/opt/cds/lib/python2.7/site-packages/flask/helpers.py", line 305, in url_for
force_external=external)
File "/opt/cds/lib/python2.7/site-packages/werkzeug/routing.py", line 1758, in build
raise BuildError(endpoint, values, method, self)
BuildError: Could not build url for endpoint 'static'. Did you forget to specify values ['filename']?
As adviced by CERN, you use PT Sans
as a font on your frontend. But because it is not installed on most computers out there, it does not get loaded for most users. You should somehow load it as a webfont, e.g. by adding the following block to cds/base/tempates/page.html
:
{%- block css %}
{{ super() }}
<link href=//fonts.googleapis.com/css?family=PT+Sans:400" rel="stylesheet" type="text/css">
{%- endblock css %}
Using Google fonts seems legit, because it is also used here: https://framework.web.cern.ch/framework/2.0/cern_toolbar/cern_toolbar-anonymous.html
Our current library mapping is the default one too, it would be nice to have then separated so we can identify records that are not matching any of the mappings (cds-dojson needs to be updated accordingly)
Maybe reuse the old one. Also review if https://github.com/CERNDocumentServer/cds-backup needs update and deploy it
Update puppet profile for Invenio 3.
Update the invenio-files-js
and invenio-files-rest
to upload videos from a URL
. This will allow to use widgets from services like Dropbox
on the uploader.
The tasks includes:
urls
in the UIfile
model in the invenio-files-js
Update the fabric script for Invenio 3, installation instructions.
Enable invenio-previewer
on the CDS app.
We will need to override https://github.com/inveniosoftware/invenio-migrator/blob/master/invenio_migrator/records.py#L132 and maybe https://github.com/inveniosoftware/invenio-migrator/blob/master/invenio_migrator/records.py#L176 to port the document types https://github.com/inveniosoftware/invenio-migrator/blob/master/invenio_migrator/legacy/bibdocfile.py#L117
Probably we should also take a look at https://github.com/inveniosoftware/invenio-records-files/blob/master/invenio_records_files/api.py#L75
It might be a good opportunity to clean up a bit bibdoc.doctypes
, so far we have 158 different types - select count(distinct doctype) from bibdoc;
.
_FILES
- select distinct doctype from bibdoc where doctype like '%_FILE';
select count(*) from bibdoc where doctype='';
- and 26501are NULL
- select count(*) from bibdoc where doctype is NULL;
_
...select distinct doctype, count(id) from bibdoc group by doctype order by count(id) desc;
The list of files to preview should be smaller, we should only preview the 'main' file (so not the plots, for example: https://cdslabs.cern.ch/search?page=5&size=20&q=muon&authors=Aad,%20Georges.
Also, the height of the preview thumbnail should be fixed, so there should be no jumping around when moving between files.
Display the list of files for each record in detail and brief view.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.