bird-house / pyramid-phoenix Goto Github PK

View Code? Open in Web Editor NEW

7.0 14.0 10.0 25.12 MB

Phoenix is a Pyramid web-application to make it easy to interact with WPS services

Home Page: http://pyramid-phoenix.readthedocs.io/en/latest/

License: Apache License 2.0

Python 12.87% JavaScript 81.00% CSS 2.66% HTML 2.70% Mako 0.34% Shell 0.09% Makefile 0.34%

python javascript pyramid wps ogc-services wms client

pyramid-phoenix's Introduction

Phoenix

Phoenix (the bird): Phoenix is a long-lived bird that is cyclically regenerated or reborn. (Wikipedia). [..]

Pyramid Phoenix is a web-application build with the Python web-framework Pyramid. Phoenix has a user-interface to interact with Web Processing Services (WPS).

For installation and configuration read the documentation on ReadTheDocs.

Phoenix is part of the Birdhouse project.

pyramid-phoenix's People

Stargazers

Watchers

Forkers

katirg rmoorman tomlandry ouranosinc xhqiao89 teriforey cedadev tommygod3 sage-group climateintelligence

pyramid-phoenix's Issues

order services by type (wps, thredds, ...) in settings/services

... they might become more ;)

use celery buildout recipe

for example:

https://github.com/collective/collective.recipe.celery

add cancel button for solr index task

... in solr settings ... cancel celery task

The idea is to provide the user with an option to use cached results that have already been calculated for the exact same set of input parameters. After inputting all the parameters, the backend can check to see if this set of parameters has already been run, and the user can be notified. They can then have the option of using those results or they can also redo the calculation by submitting the job.

Will be added the phoenix web client. Maybe caching could also be a layer in the wps proxy which is then used by the phoenix client.

refactor dashboard

check examples:

"make passwd" has no effect if phoenix-password is missing in custom.cfg

... this can happen if someone updates an old installation.

configure data (esgsearch ...) filter for processes used by wizard

... some process are only usable for a subset of esgf data, for example only variable=tas etc ...

show file size in solr search

... maybe also restrict download size.

wizard: does not use freetext query as download filter

... freetext query works in wizard search but it is ignored as download filter.

update mongodb to 3.x

currently phoenix works with pymongo=2.8 ... but there is a new version pymongo>=3.0 which does not fit to the current mongodb=2.4 database

enable opendap support

some thredds services provide only openldap ...

LDAP: Test if system packages can be stripped.

I still haven't tested this one: #31 (comment)

If that test works, we have to check how to make the build work using the Makefile.

enable ldap login

job progress bar starts for each subprocess at 0 percent ...

maybe make a process calcualtion for all subprocesses or use two indicators (subprocesses, status of single process)

myproxy logon very slow ...

at least on machines with gpfs when there is a lot of traffic.

workaround: call esgf logon async

using oauth for login

oauth will be used in esgf ... may also be used as security proxy for wps ...

try sci-wms

https://github.com/sci-wms/sci-wms

Schemas as POST forms

I currently merged my ldap fork with your upstream changes. First of all, phoenix looks great! Good work! :-)
I've moved my login template as a schema to phoenix/schema/__init__.py, basically just by copying ESGFLoginSchema. It works fine, but all arguments are passed as GET values. Not so great, as one of the arguments is a password.

http://localhost:8081/account/ldap?username=admin&password=supergeheim

Do you have any idea how I can change that to be a POST form? Otherwise is there any documentary you can refer me to?

Wrong redirect from http to https.

When I connect to http://phoenix:8081, I'll be getting redirected to https://localhost:8443. (The host name has been omitted and replaced by localhost.)

Length restriction on input sequences

It seems to be implemented, but I can't see it working.
https://github.com/bird-house/pyramid-phoenix/blob/master/phoenix/schema/wps.py#L138

Plain 404 page

Unlike stated here, the 404 page currently doesn't fit into the global layout.

send user notification when job has finished

...

myjobs table does not update status when process has failed

... the final update of the jobs table is missing ... one has to reload the page manually.

use the latest ncwms2

http://reading-escience-centre.github.io/edal-java/ncWMS_user_guide.html

handle literal type "AnyValue"

this is used for example on badc/cows wps. example:

<LiteralData>
   <ows:AnyValue>
      fldmin fldmax fldsum 
    </ows:AnyValue>
</LiteralData>

attach log file to finished job

title and abstract should not be shown in process execution page as parameter

The process execution page shows title and abstract as input parameter. But they should only be used to display some info about the process.

refactor job view

... might look similar to binstar/build:
https://binstar.org/birdhouse/ocgis/builds

Render of forbidden.pt on login failure is not working

In phoenix/views/account.py the lines 142 to 144 are resulting in TypeErrors.

Easy test: http://localhost:8081/account/openid?id=example.com%2Ftest

There is a problem generating the logging string:

2015-06-01 15:07:28,414 ERROR [phoenix.views][MainThread] unknown failure
Traceback (most recent call last):
  File "/home/fklemme/Repositories/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/tweens.py", line 21, in excview_tween
    response = handler(request)
  File "/home/fklemme/Repositories/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/router.py", line 163, in handle_request
    response = view_callable(context, request)
  File "/home/fklemme/Repositories/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 329, in attr_view
    return view(context, request)
  File "/home/fklemme/Repositories/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 305, in predicate_wrapper
    return view(context, request)
  File "/home/fklemme/Repositories/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 245, in _secured_view
    return view(context, request)
  File "/home/fklemme/Repositories/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 385, in viewresult_to_response
    result = view(context, request)
  File "/home/fklemme/Repositories/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 477, in _class_requestonly_view
    response = getattr(inst, attr)()
  File "/home/fklemme/Repositories/pyramid-phoenix/phoenix/views/account.py", line 142, in openid
    self.session.flash('Sorry, login failed: %s', queue='danger' % (result.error.message))
TypeError: not all arguments converted during string formatting

If you comment out lines 142 and 143, there is still a problem rendering forbidden.pt:

2015-06-01 15:10:12,942 ERROR [phoenix.views][MainThread] unknown failure
Traceback (most recent call last):
  File "/home/fklemme/Repositories/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/tweens.py", line 21, in excview_tween
    response = handler(request)
  File "/home/fklemme/Repositories/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/router.py", line 163, in handle_request
    response = view_callable(context, request)
  File "/home/fklemme/Repositories/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 329, in attr_view
    return view(context, request)
  File "/home/fklemme/Repositories/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 305, in predicate_wrapper
    return view(context, request)
  File "/home/fklemme/Repositories/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 245, in _secured_view
    return view(context, request)
  File "/home/fklemme/Repositories/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 385, in viewresult_to_response
    result = view(context, request)
  File "/home/fklemme/Repositories/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 477, in _class_requestonly_view
    response = getattr(inst, attr)()
  File "/home/fklemme/Repositories/pyramid-phoenix/phoenix/views/account.py", line 144, in openid
    response.text = render('phoenix:templates/account/forbidden.pt', request=self.request)
TypeError: render() takes at least 2 arguments (2 given)

The message is a little bit confusing. I assume he's missing a dictionary as the second positional argument.

auto fill x509 certificate field

some processes need a x509 certificate. When running them with Processes/Execute one has to fill the url to the x509 certificate manually. This should be automized.

run process again ...

run process with same input parameters again ...

currently one can only choose a favorite.

clear wizard favorites ...

... currently not possible ... need a clear button in MyAccount

fix timestamp render in swiftbrowser

File "/home/phoenix/birdhouse/pyramid-phoenix/phoenix/views/wizard/swiftbrowser.py", line 135, in created_td
return self.render_timestamp_td(item.get('last_modified'))
File "/home/phoenix/birdhouse/pyramid-phoenix/phoenix/grid.py", line 59, in render_timestamp_td
c=HTML.literal(mytimestamp.strftime('%Y-%m-%d %H:%M:%S')),

AttributeError: 'unicode' object has no attribute 'strftime'

Wizard gets stuck on "Choose Input Parameter"

When I test the wizard with Emu, I always get stuck on this page.

need better container/prefix for swift upload

maybe like wps_outputs/wps_id/process_id/job_id

store auth settings: ldap, oauth2 ...

... its a bit confusing if the auth settings dont show the current configuration.

upgrade to bootstrap 3

deform has now support for bootstrap 3.

wizard: using previous button causes more than one process execution

Currently one should not use the previous button. If so, then more than one processes are executed (there should only be one!).

IPython requires 'notebook' folder in docs/

The title says it all. After a fresh installation of malleefowl and pyramid, followed the typical make start, IPython will die a few times before Supervisor gives up on retrying to bring up the process.

The reason of the problem is shown in the log file:

[fklemme@null-080027f84ec6 ~]$ tail .conda/envs/birdhouse/var/log/supervisor/ipython-stdout---supervisor-djeztv.log 
supervisor: couldn't chdir to /home/fklemme/pyramid-phoenix/docs/notebook: ENOENT
supervisor: child process was not spawned
supervisor: couldn't chdir to /home/fklemme/pyramid-phoenix/docs/notebook: ENOENT
supervisor: child process was not spawned
supervisor: couldn't chdir to /home/fklemme/pyramid-phoenix/docs/notebook: ENOENT
supervisor: child process was not spawned

The workaround is quite easy:

mkdir -p ~/pyramid-phoenix/docs/notebook

Tested on CentOS 6, but likely to affect other platforms as well.

No new SSL cert with every 'make install'

It seems that every make install generates a new certificate for https.

I think it would be an improvement if make install would preserve the old certificate, if one is already there.

link help in phoenix ui to docs on readthedocs

docs are available here:

http://pyramid-phoenix.readthedocs.org/en/latest/

show version/build number on phoenix footer/about page

don't use process identifier in url path

current path: wps/helloworld/execute
should be: wps/execute?identifier=helloworld

OpenID login failure on clean installation

Just caught this one. Quick guess: I haven't added any admin e-mails to custom.cfg. (Haven't touched anything at all, just cloned and ran phoenix.)

2015-06-19 16:18:18,349 INFO  [140319115413200][MainThread] authomatic: OpenID: Starting OpenID authentication procedure.
2015-06-19 16:18:18,418 DEBUG [140319115413200][MainThread] authomatic: OpenID: SessionOpenIDStore: Association not found.
2015-06-19 16:18:18,851 DEBUG [140319115413200][MainThread] authomatic: OpenID: SessionOpenIDStore: Storing association to session.
2015-06-19 16:18:18,852 INFO  [140319115413200][MainThread] authomatic: OpenID: Service discovery for identifier http://florianklemme.de successful.
2015-06-19 16:18:18,853 INFO  [140319115413200][MainThread] authomatic: OpenID: Writing an auto-submit HTML form to the response.
2015-06-19 16:18:21,882 INFO  [140319115123728][MainThread] authomatic: OpenID: Continuing OpenID authentication procedure after redirect.
2015-06-19 16:18:21,883 DEBUG [140319115123728][MainThread] authomatic: OpenID: SessionOpenIDStore: Association found.
2015-06-19 16:18:21,887 INFO  [140319115123728][MainThread] authomatic: OpenID: Authentication successful.
2015-06-19 16:18:21,887 INFO  [140319115123728][MainThread] authomatic: OpenID: Got AX data.
2015-06-19 16:18:21,887 INFO  [140319115123728][MainThread] authomatic: OpenID: Got SREG data.
2015-06-19 16:18:21,887 INFO  [140319115123728][MainThread] authomatic: OpenID: Procedure finished.
2015-06-19 16:18:21,887 INFO  [phoenix.views.account][MainThread] openid login successful for user [email protected]
2015-06-19 16:18:21,887 INFO  [phoenix.views.account][MainThread] openid login successful for user [email protected]
2015-06-19 16:18:21,888 WARNI [phoenix.views.account][MainThread] openid login: new user [email protected]
2015-06-19 16:18:21,888 WARNI [phoenix.views.account][MainThread] openid login: new user [email protected]
2015-06-19 16:18:22,445 ERROR [phoenix.views][MainThread] unknown failure
Traceback (most recent call last):
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/tweens.py", line 21, in excview_tween
    response = handler(request)
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/router.py", line 163, in handle_request
    response = view_callable(context, request)
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 329, in attr_view
    return view(context, request)
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 305, in predicate_wrapper
    return view(context, request)
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 245, in _secured_view
    return view(context, request)
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 385, in viewresult_to_response
    result = view(context, request)
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 477, in _class_requestonly_view
    response = getattr(inst, attr)()
  File "/home/kruecke/pyramid-phoenix/phoenix/views/account.py", line 166, in openid
    self.login_success(email=result.user.email, openid=result.user.id, name=result.user.name)
  File "/home/kruecke/pyramid-phoenix/phoenix/views/account.py", line 98, in login_success
    self.send_notification(email, subject, message)
  File "/home/kruecke/pyramid-phoenix/phoenix/views/account.py", line 87, in send_notification
    mailer.send_immediately(message, fail_silently=True)
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid_mailer-0.14.1-py2.7.egg/pyramid_mailer/mailer.py", line 275, in send_immediately
    return self.smtp_mailer.send(*self._message_args(message))
  File "/home/kruecke/pyramid-phoenix/eggs/repoze.sendmail-4.2-py2.7.egg/repoze/sendmail/mailer.py", line 100, in send
    connection.sendmail(fromaddr, toaddrs, message)
  File "/home/kruecke/.conda/envs/birdhouse/lib/python2.7/smtplib.py", line 742, in sendmail
    raise SMTPRecipientsRefused(senderrs)
SMTPRecipientsRefused: {'': (501, '5.1.3 Bad recipient address syntax')}
2015-06-19 16:18:22,445 ERROR [phoenix.views][MainThread] unknown failure
Traceback (most recent call last):
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/tweens.py", line 21, in excview_tween
    response = handler(request)
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/router.py", line 163, in handle_request
    response = view_callable(context, request)
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 329, in attr_view
    return view(context, request)
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 305, in predicate_wrapper
    return view(context, request)
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 245, in _secured_view
    return view(context, request)
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 385, in viewresult_to_response
    result = view(context, request)
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid-1.5.7-py2.7.egg/pyramid/config/views.py", line 477, in _class_requestonly_view
    response = getattr(inst, attr)()
  File "/home/kruecke/pyramid-phoenix/phoenix/views/account.py", line 166, in openid
    self.login_success(email=result.user.email, openid=result.user.id, name=result.user.name)
  File "/home/kruecke/pyramid-phoenix/phoenix/views/account.py", line 98, in login_success
    self.send_notification(email, subject, message)
  File "/home/kruecke/pyramid-phoenix/phoenix/views/account.py", line 87, in send_notification
    mailer.send_immediately(message, fail_silently=True)
  File "/home/kruecke/pyramid-phoenix/eggs/pyramid_mailer-0.14.1-py2.7.egg/pyramid_mailer/mailer.py", line 275, in send_immediately
    return self.smtp_mailer.send(*self._message_args(message))
  File "/home/kruecke/pyramid-phoenix/eggs/repoze.sendmail-4.2-py2.7.egg/repoze/sendmail/mailer.py", line 100, in send
    connection.sendmail(fromaddr, toaddrs, message)
  File "/home/kruecke/.conda/envs/birdhouse/lib/python2.7/smtplib.py", line 742, in sendmail
    raise SMTPRecipientsRefused(senderrs)
SMTPRecipientsRefused: {'': (501, '5.1.3 Bad recipient address syntax')}

wizard: solr view fails if result.abstract is not available

exception in phoenix.log:

  File "/home/pingu/.conda/envs/birdhouse/lib/python2.7/site-packages/chameleon/py26.py", line 15, in lookup_attr
    raise exc
AttributeError: 'dict' object has no attribute 'abstract'

 - Expression: "
                ${result.abstract}
                "
 - Filename:   ... id-phoenix/phoenix/wizard/templates/wizard/solrsearch.pt

mongodb: many 2GB files kombu_default.n

phoenix uses celery with mongodb as broker. There are growing number of 2GB kombu_default files in var/lib/mongodb. This consumes too much disk space especially on virtual machines ...

Probably there is a config to change this ... or use a different broker ....

wizard favorites seen by all users

each user should see only its own favorites ... maybe something wrong with userids ...

Already logged in - on a fresh installation.

While testing, I experienced this strange artefact quite a few times. When I've been logged in into Phoenix and then completely purge and reinstall my Phoenix installation, I'm still logged in (as nobody?) into Phoenix.

I always thought I just missed some files when I deleted the local installation, but I also experienced this behaviour with a virtual host which has been completely reinstalled (including partitions and operating system). Just the hostname remained the same after the host has been reinstalled again.

I haven't tagged this one as a bug, as it doesn't really seem to cause any problems and it's just very unlikely to be ever seen in a relevant, non-developement system. I just wanted to post this issue to have it in mind - maybe it becomes relevant sometime.

abstract of complex type is not shown in processes and wizard view

... so now help/doc for input data (netcdf etc ...) is available.