Comments (14)
Projection solution available by 2023-10-08
from ctrdata.
Very helpful, thanks @machado-t, will provide a solution
from ctrdata.
-
I cannot confirm that "the package is not retrieving the field E.8.4 if it falls under the '/3RD' protocol". The field E.8.4 is retrieved and converted, but has a different name for 3rd country trials than for trials with a site in the EU. Please mention the ID of a trial for which is not working, thanks.
-
Would the following changes to fields be helpful? In bold, fields that have been changed, in order to have different names to reflect different meaning (e84_, e840_) or to have the same name as they mean the same (e83_, e863_):
2014-002605-38-DE | 2015-004898-32-3RD |
---|---|
"e83_single_site_trial": "No" (was before: "e83_the_trial_involves_single_site_ in_the_member_state_concerned": "No") | "e83_single_site_trial": "No" (was before: "e83_will_this_trial_be_conducted_ at_a_single_site_globally": "No") |
"e840_multiple_sites_globally": "Yes" (was before: "e84_will_this_trial_be_conducted_ at_multiple_sites_globally": "Yes")) | |
"e84_multiple_sites_in_member_state": "Yes" (was before: "e84_the_trial_involves_multiple_sites_ in_the_member_state_concerned": "Yes") | |
"e841_number_of_sites_anticipated_ in_member_state_concerned": "3", | |
"e85_the_trial_involves_multiple_member_states": "Yes", | |
"e851_number_of_sites_anticipated_in_the_eea": "26", | |
"e863_trial_sites_planned_in": "Switzerland Australia Austria Canada Denmark Finland France Germany Italy Japan Netherlands Norway Spain United Kingdom United States" (was before: "e863_specify_the_regions_in_which_ trial_sites_are_planned": ...) | "e863_trial_sites_planned_in": "Argentina Colombia Costa Rica Ecuador Mexico Peru Turkey Venezuela, Bolivarian Republic of" (was before: "e863_specify_the_countries_outside_ of_the_eea_in_which_ trial_sites_are_planned": ...) |
from ctrdata.
The field E.8.4 is retrieved and converted, but has a different name for 3rd country trials than for trials with a site in the EU.
I now realize what caused me to make a mistake: for some reason, my field list retrieved by dbFindFields() seems to be incomplete.
I would expect dbFindFields to output both versions of E.8.4 (and other variable fields for that matter), but it is only outputting one of them. This led me to believe the same E.8.4 field would output whichever version depending on the trial type (EEA vs 3RD).
I have now noticed, that the last time I run dbFindFields, it only outputted the fields for 3rd country trials, while in a previous time I've run it, it only outputted the fields for EEA trials. I've tested this again with the same results.
Here's the code I've just used to test this:
library(ctrdata)
db <- nodbi::src_sqlite(
dbname = "test.sqlite",
collection = "test"
)
ctrLoadQueryIntoDb(
queryterm = "https://www.clinicaltrialsregister.eu/ctr-search/search?query=&age=under-18",
con = db
)
fieldlist <- dbFindFields(namepart = ".*", con = db, verbose = TRUE)
And here's the output list of fields:
"","x"
"1","a2_eudract_number"
"2","a3_full_title_of_the_trial"
"3","a31_title_of_the_trial_for_lay_people_in_easily_understood_ie_nontechnical_language"
"4","a41_sponsors_protocol_code_number"
"5","a52_us_nct_clinicaltrialsgov_registry_number"
"6","a7_trial_is_part_of_a_paediatric_investigation_plan"
"7","a8_ema_decision_number_of_paediatric_investigation_plan"
"8","b1_sponsor"
"9","b1_sponsor._b1_sponsor"
"10","b1_sponsor.b11_name_of_sponsor"
"11","b1_sponsor.b134_country"
"12","b1_sponsor.b31_and_b32_status_of_the_sponsor"
"13","b1_sponsor.b4_sources_of_monetary_or_material_support"
"14","b1_sponsor.b4_sources_of_monetary_or_material_support.b41_name_of_organisation_providing_support"
"15","b1_sponsor.b4_sources_of_monetary_or_material_support.b42_country"
"16","b1_sponsor.b51_name_of_organisation"
"17","b1_sponsor.b52_functional_name_of_contact_point"
"18","b1_sponsor.b531_street_address"
"19","b1_sponsor.b532_town_city"
"20","b1_sponsor.b533_post_code"
"21","b1_sponsor.b534_country"
"22","b1_sponsor.b56_email"
"23","ctrname"
"24","dimp"
"25","dimp._dimp"
"26","dimp.d12_and_d13_imp_role"
"27","dimp.d21_imp_to_be_used_in_the_trial_has_a_marketing_authorisation"
"28","dimp.d25_the_imp_has_been_designated_in_this_indication_as_an_orphan_drug_in_the_community"
"29","dimp.d31_product_name"
"30","dimp.d3111_active_substance_of_chemical_origin"
"31","dimp.d31110_medicinal_product_containing_genetically_modified_organisms"
"32","dimp.d31111_herbal_medicinal_product"
"33","dimp.d31112_homeopathic_medicinal_product"
"34","dimp.d31113_another_type_of_medicinal_product"
"35","dimp.d3112_active_substance_of_biological_biotechnological_origin_other_than_advanced_therapy_imp_atimp"
"36","dimp.d3113_advanced_therapy_imp_atimp"
"37","dimp.d31131_somatic_cell_therapy_medicinal_product"
"38","dimp.d31132_gene_therapy_medical_product"
"39","dimp.d31133_tissue_engineered_product"
"40","dimp.d31134_combination_atimp_ie_one_involving_a_medical_device"
"41","dimp.d31135_committee_on_advanced_therapies_cat_has_issued_a_classification_for_this_product"
"42","dimp.d3114_combination_product_that_includes_a_device_but_does_not_involve_an_advanced_therapy"
"43","dimp.d3115_radiopharmaceutical_medicinal_product"
"44","dimp.d3116_immunological_medicinal_product_such_as_vaccine_allergen_immune_serum"
"45","dimp.d3117_plasma_derived_medicinal_product"
"46","dimp.d3118_extractive_medicinal_product"
"47","dimp.d3119_recombinant_medicinal_product"
"48","dimp.d32_product_code"
"49","dimp.d34_pharmaceutical_form"
"50","dimp.d341_specific_paediatric_formulation"
"51","dimp.d37_routes_of_administration_for_this_imp"
"52","dimp.d38_imp_identification_details"
"53","dimp.d38_imp_identification_details.d3101_concentration_unit"
"54","dimp.d38_imp_identification_details.d3102_concentration_type"
"55","dimp.d38_imp_identification_details.d3103_concentration_number"
"56","dimp.d38_imp_identification_details.d38_inn__proposed_inn"
"57","dimp.d38_imp_identification_details.d392_current_sponsor_code"
"58","dimp.d38_imp_identification_details.d393_other_descriptive_name"
"59","dimp.d38_imp_identification_details.d394_ev_substance_code"
"60","e11_medical_conditions_being_investigated"
"61","e111_medical_condition_in_easily_understood_language"
"62","e112_therapeutic_area"
"63","e13_condition_being_studied_is_a_rare_disease"
"64","e21_main_objective_of_the_trial"
"65","e22_secondary_objectives_of_the_trial"
"66","e23_trial_contains_a_substudy"
"67","e3_principal_inclusion_criteria"
"68","e4_principal_exclusion_criteria"
"69","e51_primary_end_points"
"70","e511_timepoints_of_evaluation_of_this_end_point"
"71","e52_secondary_end_points"
"72","e521_timepoints_of_evaluation_of_this_end_point"
"73","e61_diagnosis"
"74","e610_pharmacogenetic"
"75","e611_pharmacogenomic"
"76","e612_pharmacoeconomic"
"77","e613_others"
"78","e62_prophylaxis"
"79","e63_therapy"
"80","e64_safety"
"81","e65_efficacy"
"82","e66_pharmacokinetic"
"83","e67_pharmacodynamic"
"84","e68_bioequivalence"
"85","e69_dose_response"
"86","e71_human_pharmacology_phase_i"
"87","e711_first_administration_to_humans"
"88","e712_bioequivalence_study"
"89","e713_other"
"90","e7131_other_trial_type_description"
"91","e72_therapeutic_exploratory_phase_ii"
"92","e73_therapeutic_confirmatory_phase_iii"
"93","e74_therapeutic_use_phase_iv"
"94","e81_controlled"
"95","e811_randomised"
"96","e812_open"
"97","e813_single_blind"
"98","e814_double_blind"
"99","e815_parallel_group"
"100","e816_cross_over"
"101","e817_other"
"102","e821_other_medicinal_products"
"103","e822_placebo"
"104","e823_other"
"105","e824_number_of_treatment_arms_in_the_trial"
"106","e83_will_this_trial_be_conducted_at_a_single_site_globally"
"107","e84_will_this_trial_be_conducted_at_multiple_sites_globally"
"108","e862_trial_being_conducted_completely_outside_of_the_eea"
"109","e863_specify_the_countries_outside_of_the_eea_in_which_trial_sites_are_planned"
"110","e87_trial_has_a_data_monitoring_committee"
"111","e88_definition_of_the_end_of_the_trial_and_justification_where_it_is_not_the_last_visit_of_the_last_subject_undergoing_the_trial"
"112","e892_in_all_countries_concerned_by_the_trial_months"
"113","f11_number_of_subjects_for_this_age_range"
"114","f11_trial_has_subjects_under_18"
"115","f111_in_utero"
"116","f112_preterm_newborn_infants_up_to_gestational_age__37_weeks"
"117","f113_newborns_027_days"
"118","f114_infants_and_toddlers_28_days23_months"
"119","f115_children_211years"
"120","f1151_number_of_subjects_for_this_age_range"
"121","f116_adolescents_1217_years"
"122","f12_adults_1864_years"
"123","f13_elderly_65_years"
"124","f21_female"
"125","f22_male"
"126","f31_healthy_volunteers"
"127","f32_patients"
"128","f33_specific_vulnerable_populations"
"129","f331_women_of_childbearing_potential_not_using_contraception_"
"130","f332_women_of_childbearing_potential_using_contraception"
"131","f333_pregnant_women"
"132","f334_nursing_women"
"133","f335_emergency_situation"
"134","f336_subjects_incapable_of_giving_consent_personally"
"135","f3361_details_of_subjects_incapable_of_giving_consent"
"136","f337_others"
"137","f3371_details_of_other_specific_vulnerable_populations"
"138","f422_in_the_whole_clinical_trial"
"139","f5_plans_for_treatment_or_care_after_the_subject_has_ended_the_participation_in_the_trial_if_it_is_different_from_the_expected_normal_treatment_of_that_condition"
"140","h41_third_country_in_which_the_trial_was_first_authorised"
"141","record_last_import"
"142","x4_clinical_trial_type"
"143","x6_date_on_which_this_record_was_first_entered_in_the_eudract_database"
Would the following changes to fields be helpful?
To answer your question, I don't think it is needed to change the fields.
Maybe I'm using dbFindFields or something else wrong.
Thanks again for your work and help on this issue.
from ctrdata.
I have just noticed that other fields are also missing from that list:
- D.2.5.1 Orphan drug designation number
- D.8 Placebo (and all sub-fields)
- E.1.2 Medical condition or disease under investigation (and all MedDRA sub-fields)
- F.1.X.1 Number of subjects for this age range (appearing only for some age groups and missing for others)
- G. Investigator Networks to be involved in the Trial (entire section)
It seems that dbFindFields may be only retrieving the fields for one trial and not the whole collection.
I hope this is helpful. Please let me know if there's any other way I can help.
from ctrdata.
- Improved
dbFindFields()
to return from EUCTR fields from both EU and 3rd country trials - Please try 4bb519d with
devtools::install_github("rfhb/ctrdata", build_vignettes = TRUE)
- Harmonisation of EUCTR fields as mentioned here is maintained
- Have clarified in the documentation that
dbFindFields()
may not return all fields, for two reasons: only samples of records are used and only fields that have values exist in the collection; see??dbFindFields
.
from ctrdata.
Please try 4bb519d with devtools::install_github("rfhb/ctrdata", build_vignettes = TRUE)
After running the same code again, I'm now able to find 193 fields, which I believe account for most of the available fields. However, I notice some are still absent, particularly all fields from the G section. I understand this limitation may be hard to overcome if the function must rely on a sample of the records. As a workaround, is there anyway to the dbGetFieldsIntoDf function to output all fields available, instead of listing all the desired fields in the fields argument?
from ctrdata.
dbFindFields()
is a helper function, and package ctrdata
is designed to work in the same way across four different database backends. Of course it is possible to directly access the databases, as follows (note this does not provide the name of the register as function dbFindFields()
does):
# 1) use demo database provided in package ctrdata
dbc <- nodbi::src_sqlite(
dbname = system.file("extdata", "demo.sqlite", package = "ctrdata"),
collection = "my_trials")
#
# the following syntax works for nodbi database
# connection objects to PostgreSQL, DuckDB, SQLite
nodbi::docdb_query(src = dbc, key = dbc$collection, query = '{}', listfields = TRUE)
# 2) for MongoDB database connection objects such as
dbm <- nodbi::src_mongo(collection = "dbperm", db = "dbperm", url = "mongodb://localhost")
ctrLoadQueryIntoDb(queryterm = "query=&age=newborn&phase=phase-three&dateFrom=2015-03-01&dateTo=2015-03-31", register = "EUCTR", con = dbm)
#
# define a mongolite connection
con <- mongolite::mongo(collection = dbm$collection, db = dbm$db, url = dbm$url)
#
# then apply a MongoDb function (caveat, user rights have to be set to permit this)
sort(con$mapreduce(
map = "function() {
obj = this;
return searchInObj(obj, '');
function searchInObj(obj, pth) {
for(var key in obj) {
if(typeof obj[key] == 'object' && obj[key] !== null) {
if(pth != '') {pth = pth + '.'}
searchInObj(obj[key], pth + key);
}else{
key = pth + '.' + key;
key = key.replace(/[.][0-9]+[.]/g, '.');
key = key.replace(/[.][0-9]+$/, '');
key = key.replace(/[.][.]+/g, '.');
key = key.replace(/^[.]/, '');
emit(key, 1);
}}}}",
reduce = "function(id, counts) {return Array.sum(counts)}"
)[["_id"]])
from ctrdata.
Unfortunately, I'm an amateur and not familiar with MongoDB or any other of those databases, so I couldn't test or use that.
Considering this limitation of dbFindFields, i believe that a simple list of all the fields available in each platform could be very useful for new users.
Thank you for your help
from ctrdata.
Thanks, I appreciate the suggestion to provide a list of all data fields from (all) registers. These are the reasons why I cannot provide it and why this is not a limitation of dbFindFields()
: fields change in registers often enough to make manual maintenance unreliable (see for example https://prsinfo.clinicaltrials.gov/ProtocolRecordSchema.xsd); fields are listed in different formats, not always readily readable; dbFindFields()
was intended to list those fields for which at least one record in the collection has data.
Definitions of fields are linked from help("ctrdata-registers")
(see https://rfhb.github.io/ctrdata/reference/ctrdata-registers.html). Perhaps this is a good starting point, not sure what exactly is sought. I can think further with more background on the question or motivation.
from ctrdata.
Thanks for explaining. I understand now.
For my particular case, this issue is solved now. I was not able to find all the fields I needed using dbFindFields() but i happened to find them in the code provided in your blog.
from ctrdata.
Thanks, closing the issue, after code changes e.g. in 0d965d3 to improve field detection and documentation, yet adding it to the roadmap for ctrdata: https://github.com/users/rfhb/projects/1/views/1?pane=issue&itemId=42248978.
from ctrdata.
That's great. Thanks for your work
from ctrdata.
With ctrdata
v1.17.0, it is now possible to obtain names of fields
that are either derived from a sample of trials, or from all trial records - the latter will take (much) more time but is comprehensive, see parameter sample
in https://rfhb.github.io/ctrdata/reference/dbFindFields.html
from ctrdata.
Related Issues (20)
- Error message printed with ctrLoadQueryIntoDb() HOT 1
- Trial records not imported into database - stack error
- CTGOV changed website layout HOT 9
- Manual input of trial IDs instead of search functions HOT 5
- Feature request: extract EUCTR results data HOT 1
- When downloading results from EUCTR, sometimes fails due to error with invalid argument HOT 2
- When downloading result PDFs from EUCTR, sometimes get error of file not found HOT 1
- Certain result documents are causing an error: "No such file or directory" HOT 3
- invalid numeric literal when accumulating information in database HOT 1
- CTGOV2 API call does not filter by study sponsor HOT 7
- Shiny app HOT 4
- Invalid numeric literal HOT 6
- How to cite this package? HOT 3
- Error: Invalid numeric literal at line 1, column 913610 HOT 2
- Recruitment fields for trial on ClinicalTrials.Gov not available through ctrdata? HOT 2
- Incorrect number of trials found with date variables in query URLs for ClinicalTrials.gov HOT 3
- Not working with relaunched CTIS HOT 2
- CTGOV2 query for location returning incorrect number of studies HOT 3
- CTIS query seems not to function HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ctrdata.