phoenixctms/ctsms

Phoenix CTMS - the ultimative CTMS/PRS/CDMS.

Phoenix CTMS

Phoenix CTMS is a large web application combining capabilities of database software used in clinical research in one modular system.

• PRS (Patient Recruitment System)
• CTMS (Clinical Trial Management System)
• CDMS (Clinical Data Management System)

This unmatched feature set is geared to support all operational and regulatory requirements of the clinical front end in academic research, at CROs (Contract Research Organisations) and hospitals conducting clinical studies of any phase.

After years of collaborative development with trial sites at the Medical University of Graz, the Phoenix CTMS now becomes publicly available (LGPL 2.1). It could be the perfect choice if you e.g.

• need a private, encrypted subject database for PII (personally identifyable information), to complement your existing CDMS
• need a solution to comply with upcoming EU-GDPR (General Data Protection Regulation)
• want to operate a secure online signup portal for subject candidates
• want a CDMS that gives you unlimited Javascript form scripting support (server- and browserside)
• need to deal with large eCRFs (electronic case report forms)
• want to try a serious open source alternative for eCRFs
• need to formulate complex ad-hoc database queries to list matching subject candidates using set operations
• conduct several trials in parallel and need to organize site staff and resources
• need a software to implement various processes for ICH GCP (good clinical practice) compliance
• want a turn-key system that’s operational out-of-the-box, instead of integrating multiple systems

This GitHub repository contains all tiers of the main JEE web application. Transient artifacts generated using AndroMDA are excluded.

Build and Install

1. Prepare a vanilla Debian Bullseye Linux instance (ie. from debian-11.2.0-amd64-netinst.iso):

• 2 vCPUs, 4-8 GB RAM and 10-20 GB disk should be fine for a test environment
• Select basic packages only (no database or webserver)
• The procedure was also successfully tested with latest Ubuntu

2. Open a terminal and run the installer to automatically download, build and configure your Phoenix CTMS:

   sudo apt-get -y install wget ca-certificates
   wget https://raw.githubusercontent.com/phoenixctms/install-debian/master/install.sh -O ~/install.sh
   chmod 755 ~/install.sh
   sudo ~/install.sh

   The procedure will require internet connectivity to download from debian, maven, cpan mirros and github. It will take around 1-2 hours to complete.

3. You can now open a browser and log into the system using the link and credentials printed at last.

4. If desired, populate the system with demo data:

   sudo -u ctsms /ctsms/dbtool.sh --load_demo_data

Eclipse IDE

The following was tested on a vanilla Windows 10 VM.

1. Prerequisites:

• download and install a recent Java JDK (eg. Oracle JDK1.8.0 u201 64bit)
• download and install git
• download a recent Apache Maven (eg. apache-maven-3.6.0-bin.zip, extract to C:\apache-maven-3.6.0)

2. Setup environment variables:

• add C:\Program Files\Java\jdk1.8.0_201\bin and C:\apache-maven-3.6.0\bin to "Path" variable
• add new environment variable "JAVA_HOME" and set it to C:\Program Files\Java\jdk1.8.0_201

3. Clone the Phoenix CTMS java webapp source code:

• create folder C:\workspaces
• open Command Prompt, change to C:\workspaces and run

git clone https://github.com/phoenixctms/ctsms.git

• change to C:\workspaces\ctsms and run

mvn install -DskipTests

4. Eclipse IDE:

• download and run Eclipse Installer (eg. 2020-06), choose flavour "Eclipse IDE for Java EE Developers"
• launch Eclipse IDE and select workspace C:\workspaces
• goto Window -> Preferences -> Installed JRE and add and select JDK -JRE (=JAVA_HOME)
• goto File -> Import... -> Maven -> Existing Maven Project select C:\workspaces\ctsms folder and press "Finish"
• done. you can now start debugging, ie. select DBTool.java and goto Run -> Debug

ctsms's Issues

Replace jquery.base64.js with native functions

No need to carry around a base64 encoder/decoder:
https://developer.mozilla.org/en-US/docs/Web/API/WindowBase64/Base64_encoding_and_decoding#Solution_2_%E2%80%93_escaping_the_string_before_encoding_it appears to be easier

Navigation & Primefaces upgrade

A couple of very loose thoughts on the matter of Navigation and upgrading Primefaces.

• Currently primefaces 3.3.1 is used which was release 2012. In the meantime primefaces has seen many updates (current version 7) including new features and a lot of bugfixes.

• Refactoring the layout of the many page is out of scope due to the sheer amount.

• Upgrading primefaces risks to loose the battle-tested stability.

• Refactoring the navigation might be benefital.

• An upgrade path

• Certain navigational elements aren't user friendly:

  • TagClouds feel a little bit misguided, except the nice visual impression they don't offer much utility compared to other solutions.
  • Top menu with context actions:
    • "Search" on the "Search Trial" - Unclear to why it's necessary if you are already on the trial search page.
    • "New Trial" Quick Link on Search Trial and in the Home Portal - New trials are created very rarely, so they don't warrant a quick link.
    • "Recent" entries - Although known as an idiom in other applications it begs the question how often this is of relevance in a CTSMS setup, where users generally do not operate on the same item repeatedly.
    • The same location is navigated by different names (E.g. "Trials DB" from trial item, "Search" from Portal and others).

• New-Tab Navigation behaviour irritated a couple of users. It might be the case that power users will embrace and enjoy this ui pattern, but for certain tasks it's rather surprising. (especially, if actions/links change the behaviour in different tabs). Settings exist to change this to single window application, but I feel that a designed and coherent navigation will be better than offering multiple configurable ways to use the application.

• There are some primefaces overrides, but it's no evident, which are still necessary on later primefaces version

export "inquiry forms" data

while there is support to export eCRF data, there is no option to export data entered into the inquiry form of a trial yet. it is desired to see such option in the new "jobs" tab of the trial.

Update/Assess javascript resources

Some javascript resources are not up to date, which could mean that there compatiblity with new browsers might be a problem.
It would be helpful to minimize the usage of third party libraries (~software of unknown provenance). A javascript audit would be a good starting point. For corner cases, where a small feature might introduce a lot of javascript code the necessity of the features needs to be thoroughly reasoned.

change a proband's department

the department of a proband record matches the department of the user that created it.
any personally identifyable information (PII) is encrypted using that department's passphrase.

therefore the department never was supposed to be changed, nor creating a proband for a department other than the user's department

enable cron jobs

after a vanilla installation, no cron jobs are defined out-of-the-box yet, eg.

-subject duplicate detection
-process deferred delete operations
-mass mail queue
-notification mail queue
-nightly jobs to validate large eCRFs
-execute monthly/weekly/daily jobs
-db backup

possibility to override inputfield prompts

when re-using the same input field in a eCRF form a lot of times, it is already possible to specify a specific comment each.

however the field prompt is the same for each. so it is currently requried to specify a somewhat generic prompt (eg. "grade (1-5):") and provide the detailed question in the prompt eg.

• "What was the overall constitution before diagnosis"
• "What was the overall constitution since diagnosis"
• ...

it would be nice to be able to override the field prompt directly for an input field added to the eCRF.

enable/disable module per user

the visibility of the system modules (inventory, courses, trials, ...) has to be set per user.

migrate encrpyted "enrollment status reason" to a non-encrypted database field

the "enrollment status reason" is an encrpyted field to protect usage of plain names in messages like "Mr. John Doe rejected IC".

but this implies the drawback that only a user matching the department of the user who created the enrollment status message can read it.

in preparation for the multi-center monitoring, these message should be visible to anyone, by sacrificing the field encryption.

randomization list upload, blinded randomization details and "code break"

randomization means assigning a trial subject to a group (eg. A,B,C). phoenix supports different randomization types, that either use a predefined list (eg. A,C,B,B,A,C, ..) generated in advance or the ad-hoc result from a built-in random generator. stratification works with separate predefined lists, one per subjects having properties in common (ie. gender). the stratified randomization is the standard mode in most trials and can ensure balanced assignments to the groups for each male and female participants.

Status Quo

with the "stratification randomization lists" tab of a trial it is possible to paste the randomization list (as comma delimited text, eg. "A,C,B,B,A,C" ..) per stratification (eg. "male").

So the first male subject will have group A randomized, the next C and so on. Because subject and/or trial staff may whistleblow group A is the placebo group, this is considered "unblinded".

Requirement 1

In contrast, "blinded" trials enumerate the randomization list, so randomizing yields a number or code (eg. 001) while the group can be looked up in the secret mapping, eg.

• 001 .. A .. placebo
• 002 .. C .. reference treatment
• 003 .. B .. novel treatment
• 004 .. B .. novel treatment
• 005 .. A .. placebo
• 006 .. C .. reference treatment
• ...

A randomization list generated by eg. randomizer.at usually comes as a single spreadsheet with ordered rows like above.

It is desired to be able to upload and process such spreadsheet, which will automatically identify the stratification used (if any) and finally populate the "stratification randomization lists". The spreadsheet format to support follows the convention below:

• the first row represents a header row with column names
• the first column is always the column for the randomized code/number
• randomization codes/numbers are strings, which have to be unique per stratification
• a column is considered as stratification property, if the column name matches the input field name of a stratification probadn list attribute attribute (eg. "gender")
• every remaining column is considered "blinded" data (see requirement 2)

Requirement 2

The spreadsheet will contain the secret (blinded) details, see eg. "Treatment" column in the example. This information should be also imported, but has to be hidden in the UI.

Requirement 3

A global view must allow to search for a particular randomization code/number across all trials. The blinded data of a choosen code can be revealed (unblinded) in case of an ermergency (aka "code-break"). A reason for unblinding has to be provided in this case and will be recorded, including user and timestamp. The view should indicate if a subject was randomized to the code yet. It should also indicate the latest code-break, if any.

For now the code-break operation is permitted for any phoenix user allowed to edit the trial's details.

Requirement 4

An imported spreadsheet cannot be downloaded again. It must be possible to re-upload the spreadsheet (without errors or skipping rows) at any time - no matter if randomizations or codebreaks took place already. If the identical data is uploaded again, the exact state is restored (ie. recorded reason for unblinding must not be lost). it will be safe if an updated randomization spreadsheet amends the previous content, ie. show new codes appended at the end only.

problems compiling on java 11

compiling on debian 9 and openjdk-8 works, but fails on new debian 10 (buster) which comes with a different openjdk package (based on java 11). there are various compile errors because of moved/removed packages in the new version, ie. "package javax.annotations not found".

possibility to override PDFPainter implemenations

phoenix can render various PDF documents. While labels, background template, font types and margins/positions are all externalized to .properties files already, the actual rendering is implemented using pdfbox primitive drawing operations. Therefore eg. the CV PDF renderer implementation typically consists of 5 files (residing in the org.phoenixctms.ctsms.pdf package):

• https://github.com/phoenixctms/ctsms/blob/master/core/src/main/java/org/phoenixctms/ctsms/pdf/CVPDFBlock.java
• https://github.com/phoenixctms/ctsms/blob/master/core/src/main/java/org/phoenixctms/ctsms/pdf/CVPDFBlockCursor.java
• https://github.com/phoenixctms/ctsms/blob/master/core/src/main/java/org/phoenixctms/ctsms/pdf/CVPDFDefaultSettings.java
• https://github.com/phoenixctms/ctsms/blob/master/core/src/main/java/org/phoenixctms/ctsms/pdf/CVPDFLabelCodes.java
• https://github.com/phoenixctms/ctsms/blob/master/core/src/main/java/org/phoenixctms/ctsms/pdf/CVPDFSettingCodes.java
• https://github.com/phoenixctms/ctsms/blob/master/core/src/main/java/org/phoenixctms/ctsms/pdf/CVPDFPainter.java

it is desired to be able to configure a custom implementation by specifying external .java (or .class) files. if feasable, the same technique could also be used for org.phoenixctms.ctsms.excel.

UI spinner element executes an ajax call for each interaction

When incrementing a value using a ui spinner element, the element executes ajax calls for each increment/decrement.

1. Is this really necessary? Text input fields do not share this behaviour (e.g. when entering text no ajax call is registered)

2. If it is necessary, it might be better to apply deferred event execution. eg. to wait for a certain amount of time to make the ajax call (~250ms). If a new event is triggered within this time frame, the waiting starts anew.

does not work with google compute cloud?

I have troubles with deploying app on google cloud instance.
I can run it only on localhost without being able to get to it via external ip.
Could You offer some help or documentation on how to do that.
FYI I’m data scientist not web developer od server admin so that explains my lack of knowlegde i hope

It shows that i don’t have permisson on https traffic and timeout on http traffic

(Question) Does it support some form of an API?

Hi does the system support some form of API communication to connect it to other programs?

A lot of code is commented out

In the line of this blog post (https://kentcdodds.com/blog/please-dont-commit-commented-out-code) it would be good to get rid of unused code, which is currently commented out.
This concerns Java/Javascript/xhtml/CSS and even config files (e.g. pom.xml).
In any case there are going to be situations where its justified to leave the commented code.

cannot sort/filter a trial's subject list by dept. (site)

a trial's subject list may contain subjects of different department (sites).

a column for the subject department (inlcuding search+filter capabilities) is missing.
#41 removes the limitations regarding column number nowadays.

no new timestmap/incremented version when saving a unchanged form value

some users requested 44581a5 to no longer save (ecrf) form fields when the entered value was not changed.

at least for inquiry forms, this conflicts with requirements form other users who expect the timestamp to be updated with re-saving, eg. in order to track that they already asked the subject for a recent value.

so this needs to be either
-reverted
-configurable, ie. enabled for inquiriy forms only
-covered by a separate savebutton that enforces saving

Prepare for cross platform development

To ease cross platform development (windows/linux) .gitattributes should be used in the repo.

Following:
https://git-scm.com/docs/gitattributes
the .gitattributes file should look like this

# template from https://github.com/alexkaratarakis/gitattributes/blob/master/Java.gitattributes

# Handle line endings automatically for files detected as text
# and leave all files detected as binary untouched.
* text=auto

# These files are text and should be normalized (Convert crlf => lf)
*.css           text
*.df            text
*.htm           text
*.html          text
*.java          text
*.js            text
*.json          text
*.jsp           text
*.jspf          text
*.jspx          text
*.properties    text
*.sh            text
*.tld           text
*.txt           text
*.tag           text
*.tagx          text
*.xml           text
*.yml           text

# Added to template
*.xhtml         text
*.md            text
*.sql           text
*.uml           text


# Declare files that will always have CRLF line endings on checkout.
*.bat text eol=crlf

# These files are binary and should be left untouched
# (binary is a macro for -text -diff)
*.class         binary
*.dll           binary
*.ear           binary
*.gif           binary
*.ico           binary
*.jar           binary
*.jpg           binary
*.jpeg          binary
*.png           binary
*.so            binary
*.war           binary

and then renormalize all files:

git add --renormalize . 
git status # check changed files
git commit -m "fix line endings and introduce .gitattributes"

I've tested the change on a branch in my fork: https://github.com/laborg/ctsms/tree/lineendings .
It compiles and runs without problems.

User selectable theming might cause problems

Currently there is the possibility for a user to select the theme. Not all themes appear usable and validation effort grows with the amount of themes. Further on it adds 16Kb markup to each page.

• It should be possible to completely disable user theming (e.g. with a property setting).
• And if its enabled it might be better positioned on the extended user page (where timezone and locales are configured)

generate pseudomized proband record automatically when enrolling

it should be possible implicitly create new subject records when adding a new enrollment in the "proband list" tab.

the subject records should be pseudomized ("blinded") and the alias field should be auto-generated following a format spec set up for the trial, similar to ie.

patidmask="STUDYCODE,00,000,-"> <!--- 1: Prefix, 2: format for site, 3: format for patient, 4: separator --->
eg. patid= 'STUDYCODE-01-001'

only users with "Create/Edit/Delete proband" module-level permission can manually edit a subject's alias field. regular users with "Edit proband details" or "View proband" permissions cannot.

Don't find a way to register a visit for a given proband

Hi. I am testing Phoenix with fake data in our little hospital research center, and I have not found the way to register a visit (defined in the "Visits" tab and planned in the "Visit schedule" tab) performed at a given date for a given proband. Most of trials in our center are indeed promoted by external research centers, and in that case, the CTMS only aims to list probands associated with their visits without registering any research data (destined for external eCRF).
Regards.

CV pdf - no page break with large CV sections

CV section containing a lot of positions wont fit on a A4 pdf page. the page break logic for rendering CV pdfs need to split such section accordingly.

Authentication error after first successful login

I have logged once into the system using the provided data. but then I got logged out and the login information does not work anymore (captcha is showing up now with the second loggin attempt). Is this an intended behaviour? I did not change any passwords after logging in.

Trapped to login.jsf when already logged in

In case the user is already logged and navigates to login.jsf he isn't redirected to the portal page, but gets only presented a welcome message.

Possible fixes:

• make the redirection to portal.jsf robust (preferred)
• add link to portal.jsf

Timeout after captcha entering

When entering the captcha a timeout occured.

This is the log for this error:

12-Apr-2019 06:56:08.128 SEVERE [ajp-nio-127.0.0.1-8009-exec-4] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [Faces Servlet] in context with path [] threw exception
 java.net.ConnectException: Connection timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673)
	at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
	at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
	at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
	at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
	at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1334)
	at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1309)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:259)
	at org.phoenixctms.ctsms.web.component.captcha.Captcha.validateValue(Captcha.java:185)
	at javax.faces.component.UIInput.validate(UIInput.java:971)
	at javax.faces.component.UIInput.executeValidate(UIInput.java:1237)
	at javax.faces.component.UIInput.processValidators(UIInput.java:702)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at org.primefaces.component.dialog.Dialog.processValidators(Dialog.java:359)
	at javax.faces.component.UIForm.processValidators(UIForm.java:253)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at javax.faces.component.UIViewRoot.processValidators(UIViewRoot.java:1164)
	at com.sun.faces.lifecycle.ProcessValidationsPhase.execute(ProcessValidationsPhase.java:76)
	at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
	at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
	at javax.faces.webapp.FacesServlet.service(FacesServlet.java:593)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.primefaces.webapp.filter.FileUploadFilter.doFilter(FileUploadFilter.java:79)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.phoenixctms.ctsms.web.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:21)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:624)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
	at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:486)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1458)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)

12-Apr-2019 06:56:53.182 SEVERE [ajp-nio-127.0.0.1-8009-exec-12] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [Faces Servlet] in context with path [] threw exception
 java.net.ConnectException: Connection timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673)
	at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
	at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
	at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
	at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
	at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1334)
	at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1309)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:259)
	at org.phoenixctms.ctsms.web.component.captcha.Captcha.validateValue(Captcha.java:185)
	at javax.faces.component.UIInput.validate(UIInput.java:971)
	at javax.faces.component.UIInput.executeValidate(UIInput.java:1237)
	at javax.faces.component.UIInput.processValidators(UIInput.java:702)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at org.primefaces.component.dialog.Dialog.processValidators(Dialog.java:359)
	at javax.faces.component.UIForm.processValidators(UIForm.java:253)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at javax.faces.component.UIViewRoot.processValidators(UIViewRoot.java:1164)
	at com.sun.faces.lifecycle.ProcessValidationsPhase.execute(ProcessValidationsPhase.java:76)
	at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
	at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
	at javax.faces.webapp.FacesServlet.service(FacesServlet.java:593)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.primefaces.webapp.filter.FileUploadFilter.doFilter(FileUploadFilter.java:79)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.phoenixctms.ctsms.web.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:21)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:624)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
	at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:486)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1458)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)

12.04.2019 06:57:01 org.hibernate.hql.ast.ASTQueryTranslatorFactory [ajp-nio-127.0.0.1-8009-exec-10]
INFO  Using ASTQueryTranslatorFactory
12.04.2019 06:57:01 org.hibernate.hql.ast.ASTQueryTranslatorFactory [ajp-nio-127.0.0.1-8009-exec-10]
INFO  Using ASTQueryTranslatorFactory
12.04.2019 06:57:09 org.hibernate.hql.ast.ASTQueryTranslatorFactory [ajp-nio-127.0.0.1-8009-exec-4]
INFO  Using ASTQueryTranslatorFactory
12.04.2019 06:57:09 org.hibernate.hql.ast.ASTQueryTranslatorFactory [ajp-nio-127.0.0.1-8009-exec-4]
INFO  Using ASTQueryTranslatorFactory
12.04.2019 06:57:21 org.hibernate.hql.ast.ASTQueryTranslatorFactory [ajp-nio-127.0.0.1-8009-exec-4]
INFO  Using ASTQueryTranslatorFactory
12.04.2019 06:57:21 org.hibernate.hql.ast.ASTQueryTranslatorFactory [ajp-nio-127.0.0.1-8009-exec-4]
INFO  Using ASTQueryTranslatorFactory
12.04.2019 06:57:34 org.hibernate.hql.ast.ASTQueryTranslatorFactory [ajp-nio-127.0.0.1-8009-exec-9]
INFO  Using ASTQueryTranslatorFactory
12.04.2019 06:57:34 org.hibernate.hql.ast.ASTQueryTranslatorFactory [ajp-nio-127.0.0.1-8009-exec-9]
INFO  Using ASTQueryTranslatorFactory
12.04.2019 06:59:43 org.hibernate.hql.ast.ASTQueryTranslatorFactory [ajp-nio-127.0.0.1-8009-exec-5]
INFO  Using ASTQueryTranslatorFactory
12.04.2019 06:59:44 org.hibernate.hql.ast.ASTQueryTranslatorFactory [ajp-nio-127.0.0.1-8009-exec-5]
INFO  Using ASTQueryTranslatorFactory
12-Apr-2019 07:00:17.982 SEVERE [ajp-nio-127.0.0.1-8009-exec-2] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [Faces Servlet] in context with path [] threw exception
 java.net.ConnectException: Connection timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673)
	at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
	at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
	at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
	at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
	at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1334)
	at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1309)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:259)
	at org.phoenixctms.ctsms.web.component.captcha.Captcha.validateValue(Captcha.java:185)
	at javax.faces.component.UIInput.validate(UIInput.java:971)
	at javax.faces.component.UIInput.executeValidate(UIInput.java:1237)
	at javax.faces.component.UIInput.processValidators(UIInput.java:702)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at org.primefaces.component.dialog.Dialog.processValidators(Dialog.java:359)
	at javax.faces.component.UIForm.processValidators(UIForm.java:253)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1216)
	at javax.faces.component.UIViewRoot.processValidators(UIViewRoot.java:1164)
	at com.sun.faces.lifecycle.ProcessValidationsPhase.execute(ProcessValidationsPhase.java:76)
	at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
	at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
	at javax.faces.webapp.FacesServlet.service(FacesServlet.java:593)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.primefaces.webapp.filter.FileUploadFilter.doFilter(FileUploadFilter.java:79)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.phoenixctms.ctsms.web.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:21)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:624)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
	at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:486)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1458)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)

12.04.2019 07:01:26 org.hibernate.hql.ast.ASTQueryTranslatorFactory [ajp-nio-127.0.0.1-8009-exec-11]
INFO  Using ASTQueryTranslatorFactory
12.04.2019 07:01:26 org.hibernate.hql.ast.ASTQueryTranslatorFactory [ajp-nio-127.0.0.1-8009-exec-11]
INFO  Using ASTQueryTranslatorFactory
12.04.2019 07:01:35 org.hibernate.hql.ast.ASTQueryTranslatorFactory [ajp-nio-127.0.0.1-8009-exec-11]
INFO  Using ASTQueryTranslatorFactory
12.04.2019 07:01:35 org.hibernate.hql.ast.ASTQueryTranslatorFactory [ajp-nio-127.0.0.1-8009-exec-11]
INFO  Using ASTQueryTranslatorFactory

Field picker synchronization problem

When adding multiple unique fields to an ecrf in rapid succession (~3 per second), the page to where the fields are directed to states:

ID 601244 already picked

Doing the same in a slow fashion the behaviour isn't observed. It seams that there is some kind of synchronization/queuing problem with the field picker.

data entry tab: search for a dedicated form field

if a ecrf is large enough it is cumpersome to find a particular form field (to eg. update it).
it would be nice to see a filter for fields by name, to quickly jump to it.

Separate build and deploy

The install.sh script includes a lot of different tasks:

• adding of phoenix ctsms user
• installing necessary deb packages
• installing necessary perl packages
• fetching, building and installation of bulkprocessor
• fetching, building and installation of bulkprocessor
• database setup and configuration
• apache setup and configuration
• creating the server certificate
• initialization of phoenix
• ...

These tasks need to be split up to

• make the use case of having one system per study viable efficient/automizable.
• integrate phoenix into a continuous integration system.

Process/Tasks

Building Phoenix

• maven based build
• output: .war, db_schema_changes

Building Bulkprocessor

• packaging perl modules as executable with minimum prerequisites
• output: ideally something like a jar/war

System/OS Setup

• once per system / no configuration per study
• installation, configuration and validation of
  • postgres
  • apache
  • tomcat
  • java
• output: installation log file

StudyGroup Setup

• once per system / configuration per StudyGroup certificate, branding, "master data"
• creation and deployment of server certificate
• deployment of Phoenix/Bulkprocessor
• initialization of Phoenix

Update

• multiple times per system / no configuration
• deployment of updated war
• application of any upgrade/downgrade SQL scripts

Open questions:

• Bulkprocessor is used to render workflow state images after data has been loaded. This tight integration of build/deployment/operation complicates the matter. Maybe its better to replace the perl rendering with java (https://github.com/nidi3/graphviz-java), and call this on server start.
• Bulkprocessor, as a perl application, depends on a mix of debian packages, third party package management system(s) and tools. To allow validation of a system, it is necessary to have a way to reproducible build and deploy the Bulkprocessor. My perl knowledge is limited, but maybe this could be used: https://metacpan.org/pod/PAR::Tutorial

user welcome mail w. setpassword link

currently the admin user sets an initial password for a new user, which is communicated over phone afterwards. there also is a welcome mail sent when switching a user form local to a ldap auth method.

for local users it is desired to have a welcome mail with a link to set the password himself.

Excel export of audit trail is empty if there is only one entry

After entering data the first time into an ecrf field, the audit trail (via shows one entry, as expected, for this field:

When creating an excel export however this entry is missing.
After changing the value of the field, both audit trail entries are exported.

Is this expected?

possibility to dynamically show/hide datatable columns

some datatables get too wide because of numerous columns.
others historically lack columns with relvent data to save space.

both could be circumvented by user-defined visibility of columns.

postal addresses need an additional "state/province" field

Does not install correctly

I have downloaded the VirtualBox image and tried to make it work, and also installed VirtualBox images of the Debian and Ubuntu versions and followed the instruction to build and make it work. But I have not been successful in doing so. The latter two seemed to have compile errors and even though at the end they said I had to connect to this URL, that URL was not responsive.

change a user's department

it is currently not possible to change a user's department, or create new users with a department other than the creating user (except with dbtool.sh).

Milestones for invoicing

An important component of a complete CTMS seems to be missing. The system should be setup to help keep on top of site payments. e.g., when an eCRF is completed, a corresponding tally of amount owed is updated. Have I missed this feature, or is it still in development?

Changing Tooltip on/off results in navigation jump

Firefox 66.0.5 on Ubuntu 19.04

• Navigate to a trial
• select eCRFs tab, Tooltip is on:
  
• switch Tooltip off and wait for a couple of seconds.
• the following page is loaded automatically
  

That's rather unexpected.

"terms of conditions" prompt upon first login

a user must accept a "terms and contions" text upon initial login.
it must be possible to disable this feature globally.

a user cannot change all of his/her UI-related settings

in recent changes, the dropdown menu in the left-top corner was narrowed. it currently allows

• language selection
• UI theme selection
• show/hide tooltip texts

only admins can edit the full set of a user's UI-related settings, which additionally comprise of

• dateformat
• timezone
• decimal separator
• reset datatable column visibility

a dedicated page (similar to password change page) is required, allowing non-priviledged users to change settings on their own.

import/export clinical data via webfrontend

Q: Using the command-line "dbtool" to import/export as described in https://www.phoenixctms.org/features/modules/trials/ecrf/ is somewhat inconvenient and cannot be exposed to regular application users. Is it possible to achieve this via the Web-UI?

Aside downloading eCRF data, we would like to see a possibility to upload and download the eCRF form definition spreadsheets.

Dynamic locale and timezone setting entails a large overhead on portal.jsf

While analysing the performance of pages with Chrome devtools and Firefox, I stumbled upon the a large amount of html/js code that is generated, transferred, and to my knowledge not cached, in every page due to the locale and timezone selection.

Example:

Page	Size
ecrfprogressoverview portal.jsf currently	~329K
ecrfprogressoverview portal.jsf without locale&timezone	~54K 6-fold decrease!

Options how to deal with this:

• Moving this feature to a separate user page (preferred).
  • makes the ui cleaner
  • existing users might be confused if they lose functionality
• Javascript function to reload locale/time-changing-function on demand
  • less intrusive for current users
  • still a slight overhead for a feature which is rarely used? (at least that's the impression)

Are there good reasons for having this feature? e.g. how many times are users expected to change their timezone?

blink() might be unused

The function blink() (basetemplate.xhtml) is set on each page to search for blink-tags and toggle their visibility every second. I couldn't find any use of a blink tag, so it might be possible to just remove this repeated function call. Besides the blink tag isn't endorsed any more. (https://developer.mozilla.org/de/docs/Web/HTML/Element/blink)

refactor ctsms-web to use java8 multi-catch syntax

Java 7 offers https://docs.oracle.com/javase/8/docs/technotes/guides/language/catch-multiple.html

catch (IllegalArgumentException|AuthorisationException|ServiceException e) {...

(Question) Deploying ctsms on amazon AWS

I have deployed the app on aws instance micro EC2. I have installed it using the install.sh script and opened ports 443 and 80.

The problem is that I get the service unavailible error 503 from apache.

Is this a permissions issue?
if so any tips what to look for to fix this?
website: https://ec2-54-187-152-16.us-west-2.compute.amazonaws.com/portal.jsf

Javascript and CSS resources aren't minified

Many Javascript/CSS resources are neither minified nor distributed in a packed fashion, which leads to longer loading times than necessary.

For often used combinations of Javascript resources, minifed and packed versions would be a welcomed performance improvement. The same holds for CSS files.

This addition should only be done after #12

It could be implemented using a maven plugin: https://samaxes.github.io/minify-maven-plugin/ .

subject self-signup

Q: When creating a trial, the screen show options for subject candidate signup.

How does this work, where to find it?

more than one "responsible person" for maintenance reminder entry

maintenance reminders items can be created for inventory objects, each of which currently allows to specify one "responsible person" that will receive notifications (and having personalized filters applied eg. in the inventoryMaintenanceOverview.jsf).

Few options were investiagted; one possibility is to use a Person/Organisation with multiple mail adresses as the "responsible person". Since no account will be linked to it, the UI filters wont apply; so that would cover sending email notifications only.

it therefore is desired to implement additional "responsible persons" for the maintenance reminders. Supporting an unlimited number of responsible persons will imply a more complex change to the UI and is not needed.

ecrf list in dataentry tab is empty when sorting by group

sorting causes an sql error (for SELECT DISTINCT, ORDER BY expressions must appear in select list), and the datatables is empty.

problems with deletion of eCRF fields

During an attempt to delete one of the eCRF fields, an error message was raised:
"reson for deletion required".
Where this reason should be specified? An attempt to to use "Comment" field did not help, though may be some special format of the input is required for the deletion reason.

Create state diagram images at runtime

Currently the creation of state diagrams is done with help of the Bulk-Processor either during installation (it's part of install.sh) or at a later point in time. This entanglement of build steps makes the separation between building and deploying the application complicated.

If the state diagrams could be created from within Phoenix (on startup or on first request) the installation would be easier.

Possible solutions:

• Use java to create the diagrams as pngs (preferred)
• Use javascript to generate them as SVG