Giter Site home page Giter Site logo

ropensci / jsonvalidate Goto Github PK

View Code? Open in Web Editor NEW
48.0 7.0 14.0 798 KB

:heavy_check_mark::interrobang: Validate JSON

Home Page: https://docs.ropensci.org/jsonvalidate

License: Other

Makefile 1.51% R 85.00% Shell 0.18% JavaScript 11.83% Dockerfile 1.48%
r rstats jsonvalidate json json-validation r-package

jsonvalidate's Introduction

jsonvalidate

Project Status: Active – The project has reached a stable, usable state and is being actively developed. R-CMD-check codecov.io

Validate JSON against a schema using is-my-json-valid or ajv. This package is a thin wrapper around these node libraries, using the V8 package.

Usage

Directly validate json against schema

jsonvalidate::json_validate(json, schema)

or create a validator for multiple uses

validate <- jsonvalidate::json_validator(schema)
validate(json)
validate(json2) # etc

See the package vignette for complete examples.

Installation

Install from CRAN with

install.packages("jsonvalidate")

Alternatively, the current development version can be installed from GitHub with

devtools::install_github("ropensci/jsonvalidate")

License

MIT + file LICENSE © Rich FitzJohn.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

ropensci_footer

jsonvalidate's People

Contributors

aliciaschep avatar amrrs avatar dependabot[bot] avatar hillalex avatar ijlyttle avatar jcohen02 avatar jeroen avatar karawoo avatar krlmlr avatar maelle avatar plietar avatar r-ash avatar richfitz avatar salim-b avatar sckott avatar shbrief avatar xvrdm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

jsonvalidate's Issues

Appveyor webhook

👋 @richfitz!

Appveyor deliveries are failing cf

You'll need to email [email protected] with the following info about the latest delivery (that's what they asked me for for other repos)
X-GitHub-Delivery: 94473386-1c0f-11e8-838a-23e941ac6c29
Time: 2018-02-27 23:43:07

Thanks!

Not sure how else to put this, it doesn't appear to work.

A simple example:

jsonvalidate::json_validate(
  '{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "title": "Test",
    "type": "object",
    "properties": {
      "test": {
        "type": "integer"
      }
    }
  }',' {"test": "string"}', engine = "ajv"
)

test is a string, the schema expects integer, yet it returns TRUE.

session_info ``` ─ Session info ──────────────────────────────────────────────────────── setting value version R version 4.3.0 (2023-04-21) os macOS 14.0 system aarch64, darwin20 ui RStudio language (EN) collate en_US.UTF-8 ctype en_US.UTF-8 tz America/New_York date 2023-11-15 rstudio 2023.09.1+494 Desert Sunflower (desktop) pandoc NA

─ Packages ────────────────────────────────────────────────────────────
! package * version date (UTC) lib source
cachem 1.0.8 2023-05-01 [1] CRAN (R 4.3.0)
callr 3.7.3 2022-11-02 [1] CRAN (R 4.3.0)
P cli 3.6.1 2023-03-23 [?] CRAN (R 4.3.0)
P crayon 1.5.2 2022-09-29 [?] CRAN (R 4.3.0)
P curl 5.1.0 2023-10-02 [?] CRAN (R 4.3.0)
devtools 2.4.5 2022-10-11 [1] CRAN (R 4.3.0)
P digest 0.6.33 2023-07-07 [?] CRAN (R 4.3.0)
ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.3.0)
VP fansi 1.0.4 2023-10-08 [?] CRAN (R 4.3.1) (on disk 1.0.5)
fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.3.0)
P fs 1.6.3 2023-07-20 [?] CRAN (R 4.3.0)
P glue 1.6.2 2022-02-24 [?] CRAN (R 4.3.0)
P htmltools 0.5.7 2023-11-03 [?] CRAN (R 4.3.1)
htmlwidgets 1.6.2 2023-03-17 [1] CRAN (R 4.3.0)
P httpuv 1.6.12 2023-10-23 [?] CRAN (R 4.3.1)
jsonlite 1.8.7 2023-06-29 [1] CRAN (R 4.3.0)
P jsonvalidate 1.3.2 2021-11-03 [?] CRAN (R 4.3.0)
later 1.3.1 2023-05-02 [1] CRAN (R 4.3.0)
P lifecycle 1.0.3 2022-10-07 [?] CRAN (R 4.3.0)
P magrittr 2.0.3 2022-03-30 [?] CRAN (R 4.3.0)
memoise 2.0.1 2021-11-26 [1] CRAN (R 4.3.0)
mime 0.12 2021-09-28 [1] CRAN (R 4.3.0)
miniUI 0.1.1.1 2018-05-18 [1] CRAN (R 4.3.0)
P pillar 1.9.0 2023-03-22 [?] CRAN (R 4.3.0)
P pkgbuild 1.4.2 2023-06-26 [?] CRAN (R 4.3.0)
P pkgconfig 2.0.3 2019-09-22 [?] CRAN (R 4.3.0)
P pkgload 1.3.3 2023-09-22 [?] CRAN (R 4.3.1)
VP plyr 1.8.8 2023-10-02 [?] CRAN (R 4.3.1) (on disk 1.8.9)
prettyunits 1.2.0 2023-09-24 [1] CRAN (R 4.3.1)
P processx 3.8.2 2023-06-30 [?] CRAN (R 4.3.0)
P profvis 0.3.8 2023-05-02 [?] CRAN (R 4.3.0)
promises 1.2.1 2023-08-10 [1] CRAN (R 4.3.0)
P ps 1.7.5 2023-04-18 [?] CRAN (R 4.3.0)
P purrr 1.0.2 2023-08-10 [?] CRAN (R 4.3.0)
P R6 2.5.1 2021-08-19 [?] CRAN (R 4.3.0)
P Rcpp 1.0.11 2023-07-06 [?] CRAN (R 4.3.0)
P remotes 2.4.2.1 2023-07-18 [?] CRAN (R 4.3.0)
P renv 1.0.3 2023-09-19 [?] CRAN (R 4.3.1)
P rlang 1.1.1 2023-04-28 [?] CRAN (R 4.3.0)
P rstudioapi 0.15.0 2023-07-07 [?] CRAN (R 4.3.0)
sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.0)
P shiny 1.7.5.1 2023-10-14 [?] CRAN (R 4.3.1)
P stringi 1.7.12 2023-01-11 [?] CRAN (R 4.3.0)
P stringr 1.5.0 2022-12-02 [?] CRAN (R 4.3.0)
P tibble 3.2.1 2023-03-20 [?] CRAN (R 4.3.0)
urlchecker 1.0.1 2021-11-30 [1] CRAN (R 4.3.0)
usethis 2.2.2 2023-07-06 [1] CRAN (R 4.3.0)
VP utf8 1.2.3 2023-10-22 [?] CRAN (R 4.3.1) (on disk 1.2.4)
V8 4.3.3 2023-07-18 [1] CRAN (R 4.3.0)
VP vctrs 0.6.3 2023-10-12 [?] CRAN (R 4.3.1) (on disk 0.6.4)
xtable 1.8-4 2019-04-21 [1] CRAN (R 4.3.0)

[1] /Users/stephenholsenbeck/Library/Caches/org.R-project.R/R/renv/library/dmdu-4393acb2/R-4.3/aarch64-apple-darwin20
[2] /Users/stephenholsenbeck/Library/Caches/org.R-project.R/R/renv/sandbox/R-4.3/aarch64-apple-darwin20/ac5c2659

V ── Loaded and on-disk version mismatch.
P ── Loaded and on-disk path mismatch.

</details>

CRAN?

Hi @richfitz - do you have plans to push this to CRAN? @sckott and I are working on a package that supplies a few different methods for validating geojson (geojsonlint), and currently have jsonvalidate as a dependency. We are hoping to push to CRAN relatively soon - we can remove the function that depends on jsonvalidate for now if you're not planning to submit to CRAN, but if you have plans to do that in the near future, we will hold off. Thanks!

Deprecate and remove dataPath from errors

After #41 is merged, our verbose error format deviates from that of ajv, which has renamed dataPath to instancePath (see migration notes).

  • add a ajv_errors class or similar to the errors element in the verbose output
  • remove the dataPath element
  • add a $.ajv_errors and [[.ajv_errors method that converts dataPath to instancePath with a warning

Fatal error in ../src/platform-linux.cc, line 776

Arch Linux, with Arch User Repository package v8-3.14.5 installed:

> devtools::install_github("ropensci/jsonvalidate")
Downloading GitHub repo ropensci/jsonvalidate@master
✔  checking for file ‘/tmp/Rtmpu2Ayok/remotes5832f83a615/ropensci-jsonvalidate-6ec811e/DESCRIPTION’ ...
─  preparing ‘jsonvalidate’:
✔  checking DESCRIPTION meta-information ...
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘jsonvalidate_1.0.0.tar.gz’
   
Installing package into ‘/home/znmeb/R/x86_64-pc-linux-gnu-library/3.5’
(as ‘lib’ is unspecified)
* installing *source* package ‘jsonvalidate’ ...
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
sh: line 1:  2036 Trace/breakpoint trap   (core dumped) '/usr/lib64/R/bin/R' --no-save --slave 2>&1 < '/home/TMPDIR/RtmpoTO5a2/file7db65937624'


#
# Fatal error in ../src/platform-linux.cc, line 776
# CHECK_EQ(0, result) failed
#   Expected: 0
#   Found: 22
#

ERROR: loading failed
* removing ‘/home/znmeb/R/x86_64-pc-linux-gnu-library/3.5/jsonvalidate’
Error in i.p(...) : 
  (converted from warning) installation of package ‘/tmp/Rtmpu2Ayok/file5835fbe218b/jsonvalidate_1.0.0.tar.gz’ had non-zero exit status

I'm not sure what's going on here - this used to install but I don't remember how long ago that was.

Support for protocol-based sub-schemas

When attempting to validate against a schema that uses sub-schemas referenced by a URL, we get an error:

 Don't yet support protocol-based sub schemas

Consider adding support for this? Is there a natural workaround meanwhile? (e.g. manually downloading the sub-schema first?)

Resolve pointers (references) in json being validated prior to validation

Hello!

We're developing a framework for setting up forecasting hubs, using json configuration files to specify the config for them and are trying to use jsonvalidate to validate config files against schema.

An issue we are having is that the json config files themselves contain references to elements in a $defs section and need to be resolved prior to validation, both because they throw validation errors (see below) but also to ensure the definitions have been correctly specified according to the schema.

At the minute, because we have not found any such functionality in R, we are getting around this in a somewhat hacky way (reading the json config files into R, resolving pointers with a custom function and then reserialising to JSON in order to perform validation with jsonvalidate::json_validate() which we are having a few issues with also (see issue #65 ))

We were wondering if it would be possible to also resolve pointers in the json being validated as well as the schema? I appreciate this might be deemed outside the scope of the package and I'm not sure how much work it would be to implement but given it is an important step prior to validation perhaps it could be considered within scope?

Reproducible example

# CREATE DIR & PATHS -----
# Create temp dir
tmp_dir <- tempdir()

schema_path <- file.path(tmp_dir, "tasks-schema.json")
json_path <- file.path(tmp_dir, "tasks.json")

# DOWNLOAD NECESSARY FILES -----
# Download file and add newline to end
download.file(
    "https://raw.githubusercontent.com/Infectious-Disease-Modeling-Hubs/schemas/add-tasks-docs/tasks-schema.json",
    destfile = schema_path)
write("", file = schema_path, append = TRUE)

# Download json file and add newline to end
download.file(
    "https://raw.githubusercontent.com/annakrystalli/hub-infrastructure-experiments/json-schema-refs/json-schema/modified-hubmeta-examples/complex-hubmeta-mod.json",
    destfile = json_path)


#  SCHEMA ----
# Create schema to validate and serialise
schema <- jsonvalidate::json_schema$new(
    schema = schema_path,
    engine = "ajv")


# Validate unresolved json. Only 3 errors arising from unresolved pointers
schema$validate(json_path, verbose = TRUE) |>
    attr("errors")
#>                                         instancePath
#> 1 /rounds/0/model_tasks/0/task_ids/location/required
#> 2 /rounds/0/model_tasks/1/task_ids/location/required
#> 3 /rounds/1/model_tasks/0/task_ids/location/required
#>                                                                                                                schemaPath
#> 1 #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/location/properties/required/type
#> 2 #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/location/properties/required/type
#> 3 #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/location/properties/required/type
#>   keyword        type            message      schema
#> 1    type array, null must be array,null array, null
#> 2    type array, null must be array,null array, null
#> 3    type array, null must be array,null array, null
#>                                                                                                                                                                         parentSchema.description
#> 1 Array of location unique identifiers that must be present for submission to be valid. Can be null if no locations are required and all valid locations are specified in the optional property.
#> 2 Array of location unique identifiers that must be present for submission to be valid. Can be null if no locations are required and all valid locations are specified in the optional property.
#> 3 Array of location unique identifiers that must be present for submission to be valid. Can be null if no locations are required and all valid locations are specified in the optional property.
#>   parentSchema.type parentSchema.type                                $ref
#> 1       array, null            string #/$defs/task_ids/location/us_states
#> 2       array, null            string #/$defs/task_ids/location/us_states
#> 3       array, null            string #/$defs/task_ids/location/us_states
#>                                             dataPath
#> 1 /rounds/0/model_tasks/0/task_ids/location/required
#> 2 /rounds/0/model_tasks/1/task_ids/location/required
#> 3 /rounds/1/model_tasks/0/task_ids/location/required

Created on 2022-11-21 with reprex v2.0.2

Using draft-06 const with schema draft-04 not throwing error

I was expecting this to throw an error

> schema <- '
{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "properties": {
    "country": {
      "const": "UK"
    }
  },
  "required": [ "country" ]
}'
> jsonvalidate::json_validate('{"country": "NOT UK"}', schema, engine = "ajv")
[1] TRUE

whereas

> schema <- '
{
  "$schema": "http://json-schema.org/draft-06/schema#",
  "properties": {
    "country": {
      "const": "UK"
    }
  },
  "required": [ "country" ]
}'
> jsonvalidate::json_validate('{"country": "NOT UK"}', schema, engine = "ajv")
[1] FALSE

I was expecting call to jsonvalidate::json_validate('{"country": "NOT UK"}', schema, engine = "ajv") with draft-04 schema to throw an error for invalid schema. But perhaps that is not the case?

Latest jsonvalidate throws "SyntaxError: Use of const in strict mode."

The recent jsonvalidate update (1.3.1) on CRAN introduces... something... that causes this error:

jsonvalidate::json_validate("asdasdasd")
## Error in context_eval(join(src), private$context, serialize) :
##  SyntaxError: Use of const in strict mode.

traceback()
## 8: stop(structure(list(message = "SyntaxError: Use of const in strict mode.",
##        call = context_eval(join(src), private$context, serialize),
##        cppstack = NULL), class = c("std::invalid_argument", "C++Error",
##    "error", "condition")))
## 7: context_eval(join(src), private$context, serialize)
## 6: get_str_output(context_eval(join(src), private$context, serialize))
## 5: evaluate_js(readLines(file, encoding = "UTF-8", warn = FALSE))
## 4: ct$source(system.file("bundle.js", package = "jsonvalidate"))
## 3: jsonvalidate_js()
## 2: json_validator(schema, engine, reference = reference, strict = strict)
## 1: jsonvalidate::json_validate("asdasdasd")

It occurs regardless of what the arguments are, because it's happening during the loading of jsonvalidate's bundle.js. I'd also guess that it depends on the version of the V8 library; this problem occurs on my HPC servers where V8::engine_info reports 3.14.5.9, while my laptop reports 9.1.269.36 and has no such problems.

Best way to save and load a validator function?

Hi there,

What is the best way to save the validator function for later use?
It returns: Error in context_eval(join(src), private$context) : ReferenceError: jv_21774e4676e1 is not defined when I try to use a saved validator function.

Thanks

/Anders

Port to use ajv?

Apparently even faster than is-my-json-valid and also compatible with browserify. The interface looks richer and might offer nicer error reporting.

Nesting an array inside an object in JSON schema always returns valid (even when it's not)

Directory structure is as follows:

├── main.json
├── objects.json
└── test.json

main.json:

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "$id": "1.1.0",
    "title": "My Main Schema",
    "description": "reprex schema",
    "type": "object",
    "properties": {
      "the_object": {
        "type": "object",
        "properties": {
            "my_nested_object": {
                "$ref": "./objects.json#/properties/objects"               
            }
        }
      }
    }
}

objects.json:

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "$id": "objects",
    "title": "My Main Schema",
    "description": "reprex schema",
    "type": "object",
    "properties": {
      "objects": {
        "type": "array",
            "items": {
                "type": "string",
                "enum": ["a","b","c"]
            }
      }
    }
}

test.json:

{
    "the_object": {
        "my_nested_object": ["d", "e"]
    }
}
jsonvalidate::json_validate("test.json", "main.json")

This will always evaluate to TRUE, disregarding the mismatch between the values in the array in test_json/the_object/my_nested_object and the enum specified in the objects reference. It seems like nested objects disregard the specified constraints of referenced schema?

QA improvements to the package

  • Add code coverage measurement and deal with inevitable coverage holes
  • Implement garbage collection for the validators
  • Backport changes in 760d970 / #16 into the in.js file
  • Restructure code to make future changes easier
  • Move README.md to README.Rmd and generate
  • Classed errors for imjv

More options for serialising `NULL` & `NA` when serialising?

Issue related to #64

We're using schema$serialise() to serialise an R list, created by reading in a json config file, using R to resolve pointers within in order to be able to then validate it against a schema. We are using schema$serialise() in order to get around the problems created by unboxing (with TRUE or FALSE) when using jsonlite::toJSON to re-serialise.

The problem we are having is how NA and null values are being serialised with schema$serialise().

NAs are being converted to null in the resulting JSON while NULLs are being converted to empty values which seems to be the default behaviour of toJSON. I was wondering if it would be possible to expose toJSON arguments na and null to allow for controlling the behaviour during serialisation?

Reproducible example

# FUNCTION -----
# substitute function
library(magrittr)
substitute_refs <- function(x, defs) {
    if (is.list(x)) {
        is_ref <- names(x) == "$ref"

        if (any(is_ref)) {
            def_location <- x[[is_ref]]

            def_idx <- gsub("#/", "", def_location, fixed = TRUE) %>%
                strsplit(split = "/") %>%
                unlist() %>%
                as.list()

            replace <- purrr::pluck(defs, !!!def_idx)

            if (is.null(replace)) {
                cli::cli_abort(
                    c(x = "definition for def {.val { def_location }} returning {.var NULL} ")
                )
            } else {
                x <- replace
            }
        } else {
            lapply(x, substitute_refs, defs = defs)
        }
    } else {
        x
    }
}

# CREATE DIR & PATHS -----
# Create temp dir
tmp_dir <- tempdir()

schema_path <- file.path(tmp_dir, "tasks-schema.json")
json_path <- file.path(tmp_dir, "tasks.json")

# DOWNLOAD NECESSARY FILES -----
# Download file and add newline to end
download.file(
    "https://raw.githubusercontent.com/Infectious-Disease-Modeling-Hubs/schemas/add-tasks-docs/tasks-schema.json",
    destfile = schema_path)
write("", file = schema_path, append = TRUE)

# Download json file and add newline to end
download.file(
    "https://raw.githubusercontent.com/annakrystalli/hub-infrastructure-experiments/json-schema-refs/json-schema/modified-hubmeta-examples/complex-hubmeta-mod.json",
    destfile = json_path)


#  SCHEMA ----
# Create schema to validate and serialise
schema <- jsonvalidate::json_schema$new(
    schema = schema_path,
    engine = "ajv")


# Validate unresolved json. Only 3 errors arising from unresolved pointers
schema$validate(json_path, verbose = TRUE) |>
    attr("errors")
#>                                         instancePath
#> 1 /rounds/0/model_tasks/0/task_ids/location/required
#> 2 /rounds/0/model_tasks/1/task_ids/location/required
#> 3 /rounds/1/model_tasks/0/task_ids/location/required
#>                                                                                                                schemaPath
#> 1 #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/location/properties/required/type
#> 2 #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/location/properties/required/type
#> 3 #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/location/properties/required/type
#>   keyword        type            message      schema
#> 1    type array, null must be array,null array, null
#> 2    type array, null must be array,null array, null
#> 3    type array, null must be array,null array, null
#>                                                                                                                                                                         parentSchema.description
#> 1 Array of location unique identifiers that must be present for submission to be valid. Can be null if no locations are required and all valid locations are specified in the optional property.
#> 2 Array of location unique identifiers that must be present for submission to be valid. Can be null if no locations are required and all valid locations are specified in the optional property.
#> 3 Array of location unique identifiers that must be present for submission to be valid. Can be null if no locations are required and all valid locations are specified in the optional property.
#>   parentSchema.type parentSchema.type                                $ref
#> 1       array, null            string #/$defs/task_ids/location/us_states
#> 2       array, null            string #/$defs/task_ids/location/us_states
#> 3       array, null            string #/$defs/task_ids/location/us_states
#>                                             dataPath
#> 1 /rounds/0/model_tasks/0/task_ids/location/required
#> 2 /rounds/0/model_tasks/1/task_ids/location/required
#> 3 /rounds/1/model_tasks/0/task_ids/location/required

# RESOLVE POINTERS ----
# Resolve pointers manually.
json_list <- jsonlite::read_json(json_path,
                                 simplifyVector = TRUE,
                                 simplifyDataFrame = FALSE
)

json_resolved <- substitute_refs(json_list, json_list["$defs"])
json_resolved
#> $rounds
#> $rounds[[1]]
#> $rounds[[1]]$round_id
#> [1] "round-1"
#> 
#> $rounds[[1]]$model_tasks
#> $rounds[[1]]$model_tasks[[1]]
#> $rounds[[1]]$model_tasks[[1]]$task_ids
#> $rounds[[1]]$model_tasks[[1]]$task_ids$origin_date
#> $rounds[[1]]$model_tasks[[1]]$task_ids$origin_date$required
#> [1] "2022-09-03"
#> 
#> $rounds[[1]]$model_tasks[[1]]$task_ids$origin_date$optional
#> NULL
#> 
#> 
#> $rounds[[1]]$model_tasks[[1]]$task_ids$scenario_id
#> $rounds[[1]]$model_tasks[[1]]$task_ids$scenario_id$required
#> [1] 1
#> 
#> $rounds[[1]]$model_tasks[[1]]$task_ids$scenario_id$optional
#> NULL
#> 
#> 
#> $rounds[[1]]$model_tasks[[1]]$task_ids$location
#> $rounds[[1]]$model_tasks[[1]]$task_ids$location$required
#>  [1] "01" "02" "04" "05" "06" "08" "09" "10" "11" "12" "13" "15" "16" "17" "18"
#> [16] "19" "20" "21" "22" "23" "24" "25" "26" "27" "28" "29" "30" "31" "32" "33"
#> [31] "34" "35" "36" "37" "38" "39" "40" "41" "42" "44" "45" "46" "47" "48" "49"
#> [46] "50" "51" "53" "54" "55" "56"
#> 
#> $rounds[[1]]$model_tasks[[1]]$task_ids$location$optional
#> [1] "US"
#> 
#> 
#> $rounds[[1]]$model_tasks[[1]]$task_ids$target
#> $rounds[[1]]$model_tasks[[1]]$task_ids$target$required
#> NULL
#> 
#> $rounds[[1]]$model_tasks[[1]]$task_ids$target$optional
#> [1] "weekly rate"
#> 
#> 
#> $rounds[[1]]$model_tasks[[1]]$task_ids$horizon
#> $rounds[[1]]$model_tasks[[1]]$task_ids$horizon$required
#> NULL
#> 
#> $rounds[[1]]$model_tasks[[1]]$task_ids$horizon$optional
#> [1] 1 2
#> 
#> 
#> 
#> $rounds[[1]]$model_tasks[[1]]$output_types
#> $rounds[[1]]$model_tasks[[1]]$output_types$mean
#> $rounds[[1]]$model_tasks[[1]]$output_types$mean$type_id
#> [1] NA
#> 
#> $rounds[[1]]$model_tasks[[1]]$output_types$mean$value
#> $rounds[[1]]$model_tasks[[1]]$output_types$mean$value$type
#> [1] "integer"
#> 
#> $rounds[[1]]$model_tasks[[1]]$output_types$mean$value$minimum
#> [1] 0
#> 
#> 
#> 
#> $rounds[[1]]$model_tasks[[1]]$output_types$quantile
#> $rounds[[1]]$model_tasks[[1]]$output_types$quantile$type_id
#> $rounds[[1]]$model_tasks[[1]]$output_types$quantile$type_id$required
#> [1] 0.25 0.50 0.75
#> 
#> $rounds[[1]]$model_tasks[[1]]$output_types$quantile$type_id$optional
#> [1] 0.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9
#> 
#> 
#> $rounds[[1]]$model_tasks[[1]]$output_types$quantile$value
#> $rounds[[1]]$model_tasks[[1]]$output_types$quantile$value$type
#> [1] "numeric"
#> 
#> $rounds[[1]]$model_tasks[[1]]$output_types$quantile$value$minimum
#> [1] 0
#> 
#> $rounds[[1]]$model_tasks[[1]]$output_types$quantile$value$maximum
#> [1] 1
#> 
#> 
#> 
#> $rounds[[1]]$model_tasks[[1]]$output_types$cdf
#> $rounds[[1]]$model_tasks[[1]]$output_types$cdf$type_id
#> $rounds[[1]]$model_tasks[[1]]$output_types$cdf$type_id$required
#> [1] 10 20
#> 
#> $rounds[[1]]$model_tasks[[1]]$output_types$cdf$type_id$optional
#> NULL
#> 
#> 
#> $rounds[[1]]$model_tasks[[1]]$output_types$cdf$value
#> $rounds[[1]]$model_tasks[[1]]$output_types$cdf$value$type
#> [1] "numeric"
#> 
#> $rounds[[1]]$model_tasks[[1]]$output_types$cdf$value$minimum
#> [1] 0
#> 
#> $rounds[[1]]$model_tasks[[1]]$output_types$cdf$value$maximum
#> [1] 1
#> 
#> 
#> 
#> 
#> 
#> $rounds[[1]]$model_tasks[[2]]
#> $rounds[[1]]$model_tasks[[2]]$task_ids
#> $rounds[[1]]$model_tasks[[2]]$task_ids$origin_date
#> $rounds[[1]]$model_tasks[[2]]$task_ids$origin_date$required
#> [1] "2022-09-03"
#> 
#> $rounds[[1]]$model_tasks[[2]]$task_ids$origin_date$optional
#> NULL
#> 
#> 
#> $rounds[[1]]$model_tasks[[2]]$task_ids$scenario_id
#> $rounds[[1]]$model_tasks[[2]]$task_ids$scenario_id$required
#> [1] 1
#> 
#> $rounds[[1]]$model_tasks[[2]]$task_ids$scenario_id$optional
#> NULL
#> 
#> 
#> $rounds[[1]]$model_tasks[[2]]$task_ids$location
#> $rounds[[1]]$model_tasks[[2]]$task_ids$location$required
#>  [1] "01" "02" "04" "05" "06" "08" "09" "10" "11" "12" "13" "15" "16" "17" "18"
#> [16] "19" "20" "21" "22" "23" "24" "25" "26" "27" "28" "29" "30" "31" "32" "33"
#> [31] "34" "35" "36" "37" "38" "39" "40" "41" "42" "44" "45" "46" "47" "48" "49"
#> [46] "50" "51" "53" "54" "55" "56"
#> 
#> $rounds[[1]]$model_tasks[[2]]$task_ids$location$optional
#> [1] "US"
#> 
#> 
#> $rounds[[1]]$model_tasks[[2]]$task_ids$target
#> $rounds[[1]]$model_tasks[[2]]$task_ids$target$required
#> NULL
#> 
#> $rounds[[1]]$model_tasks[[2]]$task_ids$target$optional
#> [1] "peak week"
#> 
#> 
#> $rounds[[1]]$model_tasks[[2]]$task_ids$horizon
#> $rounds[[1]]$model_tasks[[2]]$task_ids$horizon$required
#> NULL
#> 
#> $rounds[[1]]$model_tasks[[2]]$task_ids$horizon$optional
#> [1] NA
#> 
#> 
#> 
#> $rounds[[1]]$model_tasks[[2]]$output_types
#> $rounds[[1]]$model_tasks[[2]]$output_types$cdf
#> $rounds[[1]]$model_tasks[[2]]$output_types$cdf$type_id
#> $rounds[[1]]$model_tasks[[2]]$output_types$cdf$type_id$required
#>  [1] "EW202240" "EW202241" "EW202242" "EW202243" "EW202244" "EW202245"
#>  [7] "EW202246" "EW202247" "EW202248" "EW202249" "EW202250" "EW202251"
#> [13] "EW202252" "EW202301" "EW202302" "EW202303" "EW202304" "EW202305"
#> [19] "EW202306" "EW202307" "EW202308" "EW202309" "EW202310" "EW202311"
#> [25] "EW202312" "EW202313" "EW202314" "EW202315" "EW202316" "EW202317"
#> [31] "EW202318" "EW202319" "EW202320"
#> 
#> $rounds[[1]]$model_tasks[[2]]$output_types$cdf$type_id$optional
#> NULL
#> 
#> 
#> $rounds[[1]]$model_tasks[[2]]$output_types$cdf$value
#> $rounds[[1]]$model_tasks[[2]]$output_types$cdf$value$type
#> [1] "numeric"
#> 
#> $rounds[[1]]$model_tasks[[2]]$output_types$cdf$value$minimum
#> [1] 0
#> 
#> $rounds[[1]]$model_tasks[[2]]$output_types$cdf$value$maximum
#> [1] 1
#> 
#> 
#> 
#> 
#> 
#> 
#> $rounds[[1]]$submissions_due
#> $rounds[[1]]$submissions_due$start
#> [1] "2022-09-01"
#> 
#> $rounds[[1]]$submissions_due$end
#> [1] "2022-09-05"
#> 
#> 
#> 
#> $rounds[[2]]
#> $rounds[[2]]$round_id
#> [1] "round-2"
#> 
#> $rounds[[2]]$model_tasks
#> $rounds[[2]]$model_tasks[[1]]
#> $rounds[[2]]$model_tasks[[1]]$task_ids
#> $rounds[[2]]$model_tasks[[1]]$task_ids$origin_date
#> $rounds[[2]]$model_tasks[[1]]$task_ids$origin_date$required
#> [1] "2022-10-01"
#> 
#> $rounds[[2]]$model_tasks[[1]]$task_ids$origin_date$optional
#> NULL
#> 
#> 
#> $rounds[[2]]$model_tasks[[1]]$task_ids$scenario_id
#> $rounds[[2]]$model_tasks[[1]]$task_ids$scenario_id$required
#> NULL
#> 
#> $rounds[[2]]$model_tasks[[1]]$task_ids$scenario_id$optional
#> [1] 2 3
#> 
#> 
#> $rounds[[2]]$model_tasks[[1]]$task_ids$location
#> $rounds[[2]]$model_tasks[[1]]$task_ids$location$required
#>  [1] "01" "02" "04" "05" "06" "08" "09" "10" "11" "12" "13" "15" "16" "17" "18"
#> [16] "19" "20" "21" "22" "23" "24" "25" "26" "27" "28" "29" "30" "31" "32" "33"
#> [31] "34" "35" "36" "37" "38" "39" "40" "41" "42" "44" "45" "46" "47" "48" "49"
#> [46] "50" "51" "53" "54" "55" "56"
#> 
#> $rounds[[2]]$model_tasks[[1]]$task_ids$location$optional
#> [1] "US"
#> 
#> 
#> $rounds[[2]]$model_tasks[[1]]$task_ids$target
#> $rounds[[2]]$model_tasks[[1]]$task_ids$target$required
#> NULL
#> 
#> $rounds[[2]]$model_tasks[[1]]$task_ids$target$optional
#> [1] "weekly rate"
#> 
#> 
#> $rounds[[2]]$model_tasks[[1]]$task_ids$age_group
#> $rounds[[2]]$model_tasks[[1]]$task_ids$age_group$required
#> NULL
#> 
#> $rounds[[2]]$model_tasks[[1]]$task_ids$age_group$optional
#> [1] "0-5"   "6-18"  "19-24" "25-64" "65+"  
#> 
#> 
#> $rounds[[2]]$model_tasks[[1]]$task_ids$horizon
#> $rounds[[2]]$model_tasks[[1]]$task_ids$horizon$required
#> NULL
#> 
#> $rounds[[2]]$model_tasks[[1]]$task_ids$horizon$optional
#> [1] 1 2
#> 
#> 
#> 
#> $rounds[[2]]$model_tasks[[1]]$output_types
#> $rounds[[2]]$model_tasks[[1]]$output_types$quantile
#> $rounds[[2]]$model_tasks[[1]]$output_types$quantile$type_id
#> $rounds[[2]]$model_tasks[[1]]$output_types$quantile$type_id$required
#> [1] 0.25 0.50 0.75
#> 
#> $rounds[[2]]$model_tasks[[1]]$output_types$quantile$type_id$optional
#> [1] 0.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9
#> 
#> 
#> $rounds[[2]]$model_tasks[[1]]$output_types$quantile$value
#> $rounds[[2]]$model_tasks[[1]]$output_types$quantile$value$type
#> [1] "integer"
#> 
#> $rounds[[2]]$model_tasks[[1]]$output_types$quantile$value$minimum
#> [1] 0
#> 
#> $rounds[[2]]$model_tasks[[1]]$output_types$quantile$value$maximum
#> [1] 1
#> 
#> 
#> 
#> 
#> 
#> 
#> $rounds[[2]]$submissions_due
#> $rounds[[2]]$submissions_due$start
#> [1] "2022-09-28"
#> 
#> $rounds[[2]]$submissions_due$end
#> [1] "2022-10-01"
#> 
#> 
#> $rounds[[2]]$last_data_date
#> [1] "2022-09-30"
#> 
#> 
#> 
#> $`$defs`
#> $`$defs`$task_ids
#> $`$defs`$task_ids$location
#> $`$defs`$task_ids$location$us_states
#>  [1] "01" "02" "04" "05" "06" "08" "09" "10" "11" "12" "13" "15" "16" "17" "18"
#> [16] "19" "20" "21" "22" "23" "24" "25" "26" "27" "28" "29" "30" "31" "32" "33"
#> [31] "34" "35" "36" "37" "38" "39" "40" "41" "42" "44" "45" "46" "47" "48" "49"
#> [46] "50" "51" "53" "54" "55" "56"


# CONVERT TO JSON TO SERIALSE FOR VALIDATION
# Attempt at serialising with unboxing.
json <- json_resolved |> jsonlite::toJSON(
    null = "null",
    na = "string",
    pretty = TRUE,
    auto_unbox = TRUE
)
# All vectors serialised as arrays so generating schema errors where arrays
# expected
schema$validate(json, verbose = TRUE) |>
    attr("errors")
#>                                             instancePath
#> 1  /rounds/0/model_tasks/0/task_ids/origin_date/required
#> 2  /rounds/0/model_tasks/0/task_ids/scenario_id/required
#> 3     /rounds/0/model_tasks/0/task_ids/location/optional
#> 4       /rounds/0/model_tasks/0/task_ids/target/optional
#> 5      /rounds/0/model_tasks/0/output_types/mean/type_id
#> 6  /rounds/0/model_tasks/1/task_ids/origin_date/required
#> 7  /rounds/0/model_tasks/1/task_ids/scenario_id/required
#> 8     /rounds/0/model_tasks/1/task_ids/location/optional
#> 9       /rounds/0/model_tasks/1/task_ids/target/optional
#> 10     /rounds/0/model_tasks/1/task_ids/horizon/optional
#> 11 /rounds/1/model_tasks/0/task_ids/origin_date/required
#> 12    /rounds/1/model_tasks/0/task_ids/location/optional
#> 13      /rounds/1/model_tasks/0/task_ids/target/optional
#>                                                                                                                    schemaPath
#> 1  #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/origin_date/properties/required/type
#> 2  #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/scenario_id/properties/required/type
#> 3     #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/location/properties/optional/type
#> 4       #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/target/properties/optional/type
#> 5      #/properties/rounds/items/properties/model_tasks/items/properties/output_types/properties/mean/properties/type_id/type
#> 6  #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/origin_date/properties/required/type
#> 7  #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/scenario_id/properties/required/type
#> 8     #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/location/properties/optional/type
#> 9       #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/target/properties/optional/type
#> 10     #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/horizon/properties/optional/type
#> 11 #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/origin_date/properties/required/type
#> 12    #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/location/properties/optional/type
#> 13      #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/target/properties/optional/type
#>    keyword        type            message      schema
#> 1     type array, null must be array,null array, null
#> 2     type array, null must be array,null array, null
#> 3     type array, null must be array,null array, null
#> 4     type array, null must be array,null array, null
#> 5     type       array      must be array       array
#> 6     type array, null must be array,null array, null
#> 7     type array, null must be array,null array, null
#> 8     type array, null must be array,null array, null
#> 9     type array, null must be array,null array, null
#> 10    type null, array must be null,array null, array
#> 11    type array, null must be array,null array, null
#> 12    type array, null must be array,null array, null
#> 13    type array, null must be array,null array, null
#>                                                                                                                                                                                   parentSchema.description
#> 1  Array of origin date unique identifiers that must be present for submission to be valid. Can be null if no origin dates are required and all valid origin dates are specified in the optional property.
#> 2                      Array of identifiers of scenarios that must be present in a valid submission. Can be null if no scenario ids are required and all valid ids are specified in the optional property.
#> 3                                                       Array of valid but not required unique location identifiers. Can be null if all locations are required and are specified in the required property.
#> 4                                                           Array of valid but not required unique target identifiers. Can be null if all targets are required and are specified in the required property.
#> 5                                                                                                                      Not used for mean output type. Must be an array containing the single element 'NA'.
#> 6  Array of origin date unique identifiers that must be present for submission to be valid. Can be null if no origin dates are required and all valid origin dates are specified in the optional property.
#> 7                      Array of identifiers of scenarios that must be present in a valid submission. Can be null if no scenario ids are required and all valid ids are specified in the optional property.
#> 8                                                       Array of valid but not required unique location identifiers. Can be null if all locations are required and are specified in the required property.
#> 9                                                           Array of valid but not required unique target identifiers. Can be null if all targets are required and are specified in the required property.
#> 10                                                        Array of valid but not required unique horizon identifiers. Can be null if all horizons are required and are specified in the required property.
#> 11 Array of origin date unique identifiers that must be present for submission to be valid. Can be null if no origin dates are required and all valid origin dates are specified in the optional property.
#> 12                                                      Array of valid but not required unique location identifiers. Can be null if all locations are required and are specified in the required property.
#> 13                                                          Array of valid but not required unique target identifiers. Can be null if all targets are required and are specified in the required property.
#>    parentSchema.type parentSchema.items.type parentSchema.items.format
#> 1        array, null                  string                      date
#> 2        array, null         integer, string                      <NA>
#> 3        array, null                  string                      <NA>
#> 4        array, null                  string                      <NA>
#> 5              array                    NULL                      <NA>
#> 6        array, null                  string                      date
#> 7        array, null         integer, string                      <NA>
#> 8        array, null                  string                      <NA>
#> 9        array, null                  string                      <NA>
#> 10       null, array         integer, string                      <NA>
#> 11       array, null                  string                      date
#> 12       array, null                  string                      <NA>
#> 13       array, null                  string                      <NA>
#>    parentSchema.items.enum parentSchema.items.maxItems parentSchema.default
#> 1                     NULL                          NA                 NULL
#> 2                     NULL                          NA                 NULL
#> 3                     NULL                          NA                 NULL
#> 4                     NULL                          NA                 NULL
#> 5                       NA                           1                   NA
#> 6                     NULL                          NA                 NULL
#> 7                     NULL                          NA                 NULL
#> 8                     NULL                          NA                 NULL
#> 9                     NULL                          NA                 NULL
#> 10                    NULL                          NA                 NULL
#> 11                    NULL                          NA                 NULL
#> 12                    NULL                          NA                 NULL
#> 13                    NULL                          NA                 NULL
#>           data                                              dataPath
#> 1   2022-09-03 /rounds/0/model_tasks/0/task_ids/origin_date/required
#> 2            1 /rounds/0/model_tasks/0/task_ids/scenario_id/required
#> 3           US    /rounds/0/model_tasks/0/task_ids/location/optional
#> 4  weekly rate      /rounds/0/model_tasks/0/task_ids/target/optional
#> 5           NA     /rounds/0/model_tasks/0/output_types/mean/type_id
#> 6   2022-09-03 /rounds/0/model_tasks/1/task_ids/origin_date/required
#> 7            1 /rounds/0/model_tasks/1/task_ids/scenario_id/required
#> 8           US    /rounds/0/model_tasks/1/task_ids/location/optional
#> 9    peak week      /rounds/0/model_tasks/1/task_ids/target/optional
#> 10          NA     /rounds/0/model_tasks/1/task_ids/horizon/optional
#> 11  2022-10-01 /rounds/1/model_tasks/0/task_ids/origin_date/required
#> 12          US    /rounds/1/model_tasks/0/task_ids/location/optional
#> 13 weekly rate      /rounds/1/model_tasks/0/task_ids/target/optional



json <- json_resolved |>
    schema$serialise()


# Use Schema to validate JSON
schema$validate(json, verbose = TRUE) |>
    attr("errors")
#>                                                 instancePath
#> 1      /rounds/0/model_tasks/0/task_ids/origin_date/optional
#> 2      /rounds/0/model_tasks/0/task_ids/scenario_id/optional
#> 3           /rounds/0/model_tasks/0/task_ids/target/required
#> 4          /rounds/0/model_tasks/0/task_ids/horizon/required
#> 5        /rounds/0/model_tasks/0/output_types/mean/type_id/0
#> 6  /rounds/0/model_tasks/0/output_types/cdf/type_id/optional
#> 7        /rounds/0/model_tasks/0/output_types/cdf/value/type
#> 8      /rounds/0/model_tasks/1/task_ids/origin_date/optional
#> 9      /rounds/0/model_tasks/1/task_ids/scenario_id/optional
#> 10          /rounds/0/model_tasks/1/task_ids/target/required
#> 11         /rounds/0/model_tasks/1/task_ids/horizon/required
#> 12       /rounds/0/model_tasks/1/task_ids/horizon/optional/0
#> 13 /rounds/0/model_tasks/1/output_types/cdf/type_id/optional
#> 14       /rounds/0/model_tasks/1/output_types/cdf/value/type
#> 15     /rounds/1/model_tasks/0/task_ids/origin_date/optional
#> 16     /rounds/1/model_tasks/0/task_ids/scenario_id/required
#> 17          /rounds/1/model_tasks/0/task_ids/target/required
#> 18         /rounds/1/model_tasks/0/task_ids/horizon/required
#> 19       /rounds/1/model_tasks/0/task_ids/age_group/required
#>                                                                                                                                   schemaPath
#> 1                 #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/origin_date/properties/optional/type
#> 2                 #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/scenario_id/properties/optional/type
#> 3                      #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/target/properties/required/type
#> 4                     #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/horizon/properties/required/type
#> 5               #/properties/rounds/items/properties/model_tasks/items/properties/output_types/properties/mean/properties/type_id/items/enum
#> 6  #/properties/rounds/items/properties/model_tasks/items/properties/output_types/properties/cdf/properties/type_id/properties/optional/type
#> 7        #/properties/rounds/items/properties/model_tasks/items/properties/output_types/properties/cdf/properties/value/properties/type/enum
#> 8                 #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/origin_date/properties/optional/type
#> 9                 #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/scenario_id/properties/optional/type
#> 10                     #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/target/properties/required/type
#> 11                    #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/horizon/properties/required/type
#> 12              #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/horizon/properties/optional/items/type
#> 13 #/properties/rounds/items/properties/model_tasks/items/properties/output_types/properties/cdf/properties/type_id/properties/optional/type
#> 14       #/properties/rounds/items/properties/model_tasks/items/properties/output_types/properties/cdf/properties/value/properties/type/enum
#> 15                #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/origin_date/properties/optional/type
#> 16                #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/scenario_id/properties/required/type
#> 17                     #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/target/properties/required/type
#> 18                    #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/horizon/properties/required/type
#> 19                  #/properties/rounds/items/properties/model_tasks/items/properties/task_ids/properties/age_group/properties/required/type
#>    keyword     params.type     params.allowedValues
#> 1     type     array, null                     NULL
#> 2     type     null, array                     NULL
#> 3     type     array, null                     NULL
#> 4     type     array, null                     NULL
#> 5     enum            NULL                       NA
#> 6     type     array, null                     NULL
#> 7     enum            NULL numeric, integer, double
#> 8     type     array, null                     NULL
#> 9     type     null, array                     NULL
#> 10    type     array, null                     NULL
#> 11    type     array, null                     NULL
#> 12    type integer, string                     NULL
#> 13    type     array, null                     NULL
#> 14    enum            NULL numeric, integer, double
#> 15    type     array, null                     NULL
#> 16    type     array, null                     NULL
#> 17    type     array, null                     NULL
#> 18    type     array, null                     NULL
#> 19    type     array, null                     NULL
#>                                       message                   schema
#> 1                          must be array,null              array, null
#> 2                          must be null,array              null, array
#> 3                          must be array,null              array, null
#> 4                          must be array,null              array, null
#> 5  must be equal to one of the allowed values                       NA
#> 6                          must be array,null              array, null
#> 7  must be equal to one of the allowed values numeric, integer, double
#> 8                          must be array,null              array, null
#> 9                          must be null,array              null, array
#> 10                         must be array,null              array, null
#> 11                         must be array,null              array, null
#> 12                     must be integer,string          integer, string
#> 13                         must be array,null              array, null
#> 14 must be equal to one of the allowed values numeric, integer, double
#> 15                         must be array,null              array, null
#> 16                         must be array,null              array, null
#> 17                         must be array,null              array, null
#> 18                         must be array,null              array, null
#> 19                         must be array,null              array, null
#>                                                                                                                                                                             parentSchema.description
#> 1                                           Array of valid but not required unique origin date identifiers. Can be null if all origin dates are required and are specified in the required property.
#> 2                                                    Array of identifiers of valid but not required scenarios. Can be null if all scenarios are required and are specified in the required property.
#> 3           Array of target unique identifiers that must be present for submission to be valid. Can be null if no targets are required and all valid targets are specified in the optional property.
#> 4        Array of horizon unique identifiers that must be present for submission to be valid. Can be null if no horizons are required and all valid horizons are specified in the optional property.
#> 5                                                                                                                                                                                               <NA>
#> 6                                                   Array of valid but not required unique target values. Can be null if all ptarget values are required and are specified in the required property.
#> 7                                                                                                                                              Data type of cumulative distribution function values.
#> 8                                           Array of valid but not required unique origin date identifiers. Can be null if all origin dates are required and are specified in the required property.
#> 9                                                    Array of identifiers of valid but not required scenarios. Can be null if all scenarios are required and are specified in the required property.
#> 10          Array of target unique identifiers that must be present for submission to be valid. Can be null if no targets are required and all valid targets are specified in the optional property.
#> 11       Array of horizon unique identifiers that must be present for submission to be valid. Can be null if no horizons are required and all valid horizons are specified in the optional property.
#> 12                                                                                                                                                                                              <NA>
#> 13                                                  Array of valid but not required unique target values. Can be null if all ptarget values are required and are specified in the required property.
#> 14                                                                                                                                             Data type of cumulative distribution function values.
#> 15                                          Array of valid but not required unique origin date identifiers. Can be null if all origin dates are required and are specified in the required property.
#> 16               Array of identifiers of scenarios that must be present in a valid submission. Can be null if no scenario ids are required and all valid ids are specified in the optional property.
#> 17          Array of target unique identifiers that must be present for submission to be valid. Can be null if no targets are required and all valid targets are specified in the optional property.
#> 18       Array of horizon unique identifiers that must be present for submission to be valid. Can be null if no horizons are required and all valid horizons are specified in the optional property.
#> 19 Array of age group unique identifiers that must be present for submission to be valid. Can be null if no age groups are required and all valid age groups are specified in the optional property.
#>    parentSchema.type parentSchema.items.type parentSchema.items.format
#> 1        array, null                  string                      date
#> 2        null, array         integer, string                      <NA>
#> 3        array, null                  string                      <NA>
#> 4        array, null         integer, string                      <NA>
#> 5               NULL                    NULL                      <NA>
#> 6        array, null                    NULL                      <NA>
#> 7               NULL                    NULL                      <NA>
#> 8        array, null                  string                      date
#> 9        null, array         integer, string                      <NA>
#> 10       array, null                  string                      <NA>
#> 11       array, null         integer, string                      <NA>
#> 12   integer, string                    NULL                      <NA>
#> 13       array, null                    NULL                      <NA>
#> 14              NULL                    NULL                      <NA>
#> 15       array, null                  string                      date
#> 16       array, null         integer, string                      <NA>
#> 17       array, null                  string                      <NA>
#> 18       array, null         integer, string                      <NA>
#> 19       array, null                  string                      <NA>
#>                                parentSchema.items.oneOf
#> 1                                                  NULL
#> 2                                                  NULL
#> 3                                                  NULL
#> 4                                                  NULL
#> 5                                                  NULL
#> 6  number, string, 0, NA, NA, ^EW[0-9]{6}, NA, 8, NA, 8
#> 7                                                  NULL
#> 8                                                  NULL
#> 9                                                  NULL
#> 10                                                 NULL
#> 11                                                 NULL
#> 12                                                 NULL
#> 13 number, string, 0, NA, NA, ^EW[0-9]{6}, NA, 8, NA, 8
#> 14                                                 NULL
#> 15                                                 NULL
#> 16                                                 NULL
#> 17                                                 NULL
#> 18                                                 NULL
#> 19                                                 NULL
#>           parentSchema.enum parentSchema.maxItems parentSchema.example    data
#> 1                      NULL                    NA                 <NA>    NULL
#> 2                      NULL                    NA                 <NA>    NULL
#> 3                      NULL                    NA                 <NA>    NULL
#> 4                      NULL                    NA                 <NA>    NULL
#> 5                        NA                     1                 <NA>    NULL
#> 6                      NULL                    NA                 <NA>    NULL
#> 7  numeric, integer, double                    NA               double numeric
#> 8                      NULL                    NA                 <NA>    NULL
#> 9                      NULL                    NA                 <NA>    NULL
#> 10                     NULL                    NA                 <NA>    NULL
#> 11                     NULL                    NA                 <NA>    NULL
#> 12                     NULL                    NA                 <NA>    NULL
#> 13                     NULL                    NA                 <NA>    NULL
#> 14 numeric, integer, double                    NA               double numeric
#> 15                     NULL                    NA                 <NA>    NULL
#> 16                     NULL                    NA                 <NA>    NULL
#> 17                     NULL                    NA                 <NA>    NULL
#> 18                     NULL                    NA                 <NA>    NULL
#> 19                     NULL                    NA                 <NA>    NULL
#>                                                     dataPath
#> 1      /rounds/0/model_tasks/0/task_ids/origin_date/optional
#> 2      /rounds/0/model_tasks/0/task_ids/scenario_id/optional
#> 3           /rounds/0/model_tasks/0/task_ids/target/required
#> 4          /rounds/0/model_tasks/0/task_ids/horizon/required
#> 5        /rounds/0/model_tasks/0/output_types/mean/type_id/0
#> 6  /rounds/0/model_tasks/0/output_types/cdf/type_id/optional
#> 7        /rounds/0/model_tasks/0/output_types/cdf/value/type
#> 8      /rounds/0/model_tasks/1/task_ids/origin_date/optional
#> 9      /rounds/0/model_tasks/1/task_ids/scenario_id/optional
#> 10          /rounds/0/model_tasks/1/task_ids/target/required
#> 11         /rounds/0/model_tasks/1/task_ids/horizon/required
#> 12       /rounds/0/model_tasks/1/task_ids/horizon/optional/0
#> 13 /rounds/0/model_tasks/1/output_types/cdf/type_id/optional
#> 14       /rounds/0/model_tasks/1/output_types/cdf/value/type
#> 15     /rounds/1/model_tasks/0/task_ids/origin_date/optional
#> 16     /rounds/1/model_tasks/0/task_ids/scenario_id/required
#> 17          /rounds/1/model_tasks/0/task_ids/target/required
#> 18         /rounds/1/model_tasks/0/task_ids/horizon/required
#> 19       /rounds/1/model_tasks/0/task_ids/age_group/required

# Errors because NAs encoded as NULL whereas NULL encoded as empty.
jsonlite::prettify(json)
#> {
#>     "rounds": [
#>         {
#>             "round_id": "round-1",
#>             "model_tasks": [
#>                 {
#>                     "task_ids": {
#>                         "origin_date": {
#>                             "required": [
#>                                 "2022-09-03"
#>                             ],
#>                             "optional": {
#> 
#>                             }
#>                         },
#>                         "scenario_id": {
#>                             "required": [
#>                                 1
#>                             ],
#>                             "optional": {
#> 
#>                             }
#>                         },
#>                         "location": {
#>                             "required": [
#>                                 "01",
#>                                 "02",
#>                                 "04",
#>                                 "05",
#>                                 "06",
#>                                 "08",
#>                                 "09",
#>                                 "10",
#>                                 "11",
#>                                 "12",
#>                                 "13",
#>                                 "15",
#>                                 "16",
#>                                 "17",
#>                                 "18",
#>                                 "19",
#>                                 "20",
#>                                 "21",
#>                                 "22",
#>                                 "23",
#>                                 "24",
#>                                 "25",
#>                                 "26",
#>                                 "27",
#>                                 "28",
#>                                 "29",
#>                                 "30",
#>                                 "31",
#>                                 "32",
#>                                 "33",
#>                                 "34",
#>                                 "35",
#>                                 "36",
#>                                 "37",
#>                                 "38",
#>                                 "39",
#>                                 "40",
#>                                 "41",
#>                                 "42",
#>                                 "44",
#>                                 "45",
#>                                 "46",
#>                                 "47",
#>                                 "48",
#>                                 "49",
#>                                 "50",
#>                                 "51",
#>                                 "53",
#>                                 "54",
#>                                 "55",
#>                                 "56"
#>                             ],
#>                             "optional": [
#>                                 "US"
#>                             ]
#>                         },
#>                         "target": {
#>                             "required": {
#> 
#>                             },
#>                             "optional": [
#>                                 "weekly rate"
#>                             ]
#>                         },
#>                         "horizon": {
#>                             "required": {
#> 
#>                             },
#>                             "optional": [
#>                                 1,
#>                                 2
#>                             ]
#>                         }
#>                     },
#>                     "output_types": {
#>                         "mean": {
#>                             "type_id": [
#>                                 null
#>                             ],
#>                             "value": {
#>                                 "type": "integer",
#>                                 "minimum": 0
#>                             }
#>                         },
#>                         "quantile": {
#>                             "type_id": {
#>                                 "required": [
#>                                     0.25,
#>                                     0.5,
#>                                     0.75
#>                                 ],
#>                                 "optional": [
#>                                     0.1,
#>                                     0.2,
#>                                     0.3,
#>                                     0.4,
#>                                     0.6,
#>                                     0.7,
#>                                     0.8,
#>                                     0.9
#>                                 ]
#>                             },
#>                             "value": {
#>                                 "type": "numeric",
#>                                 "minimum": 0,
#>                                 "maximum": 1
#>                             }
#>                         },
#>                         "cdf": {
#>                             "type_id": {
#>                                 "required": [
#>                                     10,
#>                                     20
#>                                 ],
#>                                 "optional": {
#> 
#>                                 }
#>                             },
#>                             "value": {
#>                                 "type": [
#>                                     "numeric"
#>                                 ],
#>                                 "minimum": 0,
#>                                 "maximum": 1
#>                             }
#>                         }
#>                     }
#>                 },
#>                 {
#>                     "task_ids": {
#>                         "origin_date": {
#>                             "required": [
#>                                 "2022-09-03"
#>                             ],
#>                             "optional": {
#> 
#>                             }
#>                         },
#>                         "scenario_id": {
#>                             "required": [
#>                                 1
#>                             ],
#>                             "optional": {
#> 
#>                             }
#>                         },
#>                         "location": {
#>                             "required": [
#>                                 "01",
#>                                 "02",
#>                                 "04",
#>                                 "05",
#>                                 "06",
#>                                 "08",
#>                                 "09",
#>                                 "10",
#>                                 "11",
#>                                 "12",
#>                                 "13",
#>                                 "15",
#>                                 "16",
#>                                 "17",
#>                                 "18",
#>                                 "19",
#>                                 "20",
#>                                 "21",
#>                                 "22",
#>                                 "23",
#>                                 "24",
#>                                 "25",
#>                                 "26",
#>                                 "27",
#>                                 "28",
#>                                 "29",
#>                                 "30",
#>                                 "31",
#>                                 "32",
#>                                 "33",
#>                                 "34",
#>                                 "35",
#>                                 "36",
#>                                 "37",
#>                                 "38",
#>                                 "39",
#>                                 "40",
#>                                 "41",
#>                                 "42",
#>                                 "44",
#>                                 "45",
#>                                 "46",
#>                                 "47",
#>                                 "48",
#>                                 "49",
#>                                 "50",
#>                                 "51",
#>                                 "53",
#>                                 "54",
#>                                 "55",
#>                                 "56"
#>                             ],
#>                             "optional": [
#>                                 "US"
#>                             ]
#>                         },
#>                         "target": {
#>                             "required": {
#> 
#>                             },
#>                             "optional": [
#>                                 "peak week"
#>                             ]
#>                         },
#>                         "horizon": {
#>                             "required": {
#> 
#>                             },
#>                             "optional": [
#>                                 null
#>                             ]
#>                         }
#>                     },
#>                     "output_types": {
#>                         "cdf": {
#>                             "type_id": {
#>                                 "required": [
#>                                     "EW202240",
#>                                     "EW202241",
#>                                     "EW202242",
#>                                     "EW202243",
#>                                     "EW202244",
#>                                     "EW202245",
#>                                     "EW202246",
#>                                     "EW202247",
#>                                     "EW202248",
#>                                     "EW202249",
#>                                     "EW202250",
#>                                     "EW202251",
#>                                     "EW202252",
#>                                     "EW202301",
#>                                     "EW202302",
#>                                     "EW202303",
#>                                     "EW202304",
#>                                     "EW202305",
#>                                     "EW202306",
#>                                     "EW202307",
#>                                     "EW202308",
#>                                     "EW202309",
#>                                     "EW202310",
#>                                     "EW202311",
#>                                     "EW202312",
#>                                     "EW202313",
#>                                     "EW202314",
#>                                     "EW202315",
#>                                     "EW202316",
#>                                     "EW202317",
#>                                     "EW202318",
#>                                     "EW202319",
#>                                     "EW202320"
#>                                 ],
#>                                 "optional": {
#> 
#>                                 }
#>                             },
#>                             "value": {
#>                                 "type": [
#>                                     "numeric"
#>                                 ],
#>                                 "minimum": 0,
#>                                 "maximum": 1
#>                             }
#>                         }
#>                     }
#>                 }
#>             ],
#>             "submissions_due": {
#>                 "start": [
#>                     "2022-09-01"
#>                 ],
#>                 "end": [
#>                     "2022-09-05"
#>                 ]
#>             }
#>         },
#>         {
#>             "round_id": "round-2",
#>             "model_tasks": [
#>                 {
#>                     "task_ids": {
#>                         "origin_date": {
#>                             "required": [
#>                                 "2022-10-01"
#>                             ],
#>                             "optional": {
#> 
#>                             }
#>                         },
#>                         "scenario_id": {
#>                             "required": {
#> 
#>                             },
#>                             "optional": [
#>                                 2,
#>                                 3
#>                             ]
#>                         },
#>                         "location": {
#>                             "required": [
#>                                 "01",
#>                                 "02",
#>                                 "04",
#>                                 "05",
#>                                 "06",
#>                                 "08",
#>                                 "09",
#>                                 "10",
#>                                 "11",
#>                                 "12",
#>                                 "13",
#>                                 "15",
#>                                 "16",
#>                                 "17",
#>                                 "18",
#>                                 "19",
#>                                 "20",
#>                                 "21",
#>                                 "22",
#>                                 "23",
#>                                 "24",
#>                                 "25",
#>                                 "26",
#>                                 "27",
#>                                 "28",
#>                                 "29",
#>                                 "30",
#>                                 "31",
#>                                 "32",
#>                                 "33",
#>                                 "34",
#>                                 "35",
#>                                 "36",
#>                                 "37",
#>                                 "38",
#>                                 "39",
#>                                 "40",
#>                                 "41",
#>                                 "42",
#>                                 "44",
#>                                 "45",
#>                                 "46",
#>                                 "47",
#>                                 "48",
#>                                 "49",
#>                                 "50",
#>                                 "51",
#>                                 "53",
#>                                 "54",
#>                                 "55",
#>                                 "56"
#>                             ],
#>                             "optional": [
#>                                 "US"
#>                             ]
#>                         },
#>                         "target": {
#>                             "required": {
#> 
#>                             },
#>                             "optional": [
#>                                 "weekly rate"
#>                             ]
#>                         },
#>                         "age_group": {
#>                             "required": {
#> 
#>                             },
#>                             "optional": [
#>                                 "0-5",
#>                                 "6-18",
#>                                 "19-24",
#>                                 "25-64",
#>                                 "65+"
#>                             ]
#>                         },
#>                         "horizon": {
#>                             "required": {
#> 
#>                             },
#>                             "optional": [
#>                                 1,
#>                                 2
#>                             ]
#>                         }
#>                     },
#>                     "output_types": {
#>                         "quantile": {
#>                             "type_id": {
#>                                 "required": [
#>                                     0.25,
#>                                     0.5,
#>                                     0.75
#>                                 ],
#>                                 "optional": [
#>                                     0.1,
#>                                     0.2,
#>                                     0.3,
#>                                     0.4,
#>                                     0.6,
#>                                     0.7,
#>                                     0.8,
#>                                     0.9
#>                                 ]
#>                             },
#>                             "value": {
#>                                 "type": "integer",
#>                                 "minimum": 0,
#>                                 "maximum": 1
#>                             }
#>                         }
#>                     }
#>                 }
#>             ],
#>             "submissions_due": {
#>                 "start": [
#>                     "2022-09-28"
#>                 ],
#>                 "end": [
#>                     "2022-10-01"
#>                 ]
#>             },
#>             "last_data_date": [
#>                 "2022-09-30"
#>             ]
#>         }
#>     ],
#>     "$defs": {
#>         "task_ids": {
#>             "location": {
#>                 "us_states": [
#>                     "01",
#>                     "02",
#>                     "04",
#>                     "05",
#>                     "06",
#>                     "08",
#>                     "09",
#>                     "10",
#>                     "11",
#>                     "12",
#>                     "13",
#>                     "15",
#>                     "16",
#>                     "17",
#>                     "18",
#>                     "19",
#>                     "20",
#>                     "21",
#>                     "22",
#>                     "23",
#>                     "24",
#>                     "25",
#>                     "26",
#>                     "27",
#>                     "28",
#>                     "29",
#>                     "30",
#>                     "31",
#>                     "32",
#>                     "33",
#>                     "34",
#>                     "35",
#>                     "36",
#>                     "37",
#>                     "38",
#>                     "39",
#>                     "40",
#>                     "41",
#>                     "42",
#>                     "44",
#>                     "45",
#>                     "46",
#>                     "47",
#>                     "48",
#>                     "49",
#>                     "50",
#>                     "51",
#>                     "53",
#>                     "54",
#>                     "55",
#>                     "56"
#>                 ]
#>             }
#>         }
#>     }
#> }
#> 

Created on 2022-11-21 with reprex v2.0.2

Change behaviour of "greedy" to better support ajv

It needs to move into the constructor for the validator, not into the call. However, that performs poorly for the extant uses of the package - geojsonlint creates a validator object and exposes the greedy argument, for example. For the tableschema and dataschema packages it's not so bad as they always use greedy = TRUE

Failing Tests from JSON-Schema-Test

I code-generated a bunch of testthat tests that seem to be failing when I would expect them not to. First check is probably to be sure that the underlying is-my-json-valid passes these tests as well.

Test 1 - Pseudo-Array

context('an array of schemas for items')

schema <- '{
    "items": [
        {
            "type": "integer"
        },
        {
            "type": "string"
        }
    ]
}
'

test_that("JavaScript pseudo-array is valid",{ 

 json <- '{
    "0": "invalid",
    "1": "valid",
    "length": 2
}
'
 expect_true(json_validate(json,schema,verbose=TRUE)) 
})
##1. Failure: JavaScript pseudo-array is valid (@test-items.R#130) ------------------------
##json_validate(json, schema, verbose = TRUE) isn't true.

Test 2 - ref sibling keywords

context('ref overrides any sibling keywords')

schema <- '{
    "definitions": {
        "reffed": {
            "type": "array"
        }
    },
    "properties": {
        "foo": {
            "$ref": "#/definitions/reffed",
            "maxItems": 2
        }
    }
}
'


test_that("ref valid, maxItems ignored",{ 

 json <- '{
    "foo": [
        1,
        2,
        3
    ]
}
'
 expect_true(json_validate(json,schema,verbose=TRUE)) 
})
##2. Failure: ref valid, maxItems ignored (@test-ref.R#282) -------------------------------
##json_validate(json, schema, verbose = TRUE) isn't true.

Test 3 - ref inside a tree

context('Recursive references between schemas')

schema <- '{
    "id": "http://localhost:1234/tree",
    "description": "tree of nodes",
    "type": "object",
    "properties": {
        "meta": {
            "type": "string"
        },
        "nodes": {
            "type": "array",
            "items": {
                "$ref": "node"
            }
        }
    },
    "required": [
        "meta",
        "nodes"
    ],
    "definitions": {
        "node": {
            "id": "http://localhost:1234/node",
            "description": "node",
            "type": "object",
            "properties": {
                "value": {
                    "type": "number"
                },
                "subtree": {
                    "$ref": "tree"
                }
            },
            "required": [
                "value"
            ]
        }
    }
}
'

test_that("invalid tree",{ 

 json <- '{
    "meta": "root",
    "nodes": [
        {
            "value": 1,
            "subtree": {
                "meta": "child",
                "nodes": [
                    {
                        "value": "string is invalid"
                    },
                    {
                        "value": 1.2
                    }
                ]
            }
        },
        {
            "value": 2,
            "subtree": {
                "meta": "child",
                "nodes": [
                    {
                        "value": 2.1
                    },
                    {
                        "value": 2.2
                    }
                ]
            }
        }
    ]
}
'
 expect_false(json_validate(json,schema,verbose=TRUE)) 
})

##3. Failure: invalid tree (@test-ref.R#471) ----------------------------------------------
##json_validate(json, schema, verbose = TRUE) isn't false.

Support options for `ajv`

Hi,

I'm interested in using jsonvalidate for my projects, and one of the features that would be really helpful from ajv is being able to fill in default values from the schema.

Doing so would require being able to set the useDefaults option for ajv. Is there a way to pass options other than strict to json_schema$new() without using a modified version of in.js?

Serialization using `$ref` returns arrays

Hi,

I've been working with json_schema$serialise() recently, and I'm seeing that when I attempt to serialise an object using a composed/cobined schema (anything using $ref), the returned values are always arrays.

schema <- '{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "definitions": {
        "city": { "type": "string" }
    },
    "type": "object",
    "properties": {
        "city": { "$ref": "#/definitions/city" }
    }
}'
                        
validator <- jsonvalidate::json_schema$new(schema)

out <- validator$serialise(list(city = "Firenze"))

print(out)
#> {"city":["Firenze"]}

validator$validate(out, verbose = TRUE)
#> [1] FALSE
#> attr(,"errors")
#>   instancePath              schemaPath keyword   type        message schema
#> 1        /city #/definitions/city/type    type string must be string string
#>     type    data dataPath
#> 1 string Firenze    /city

Created on 2024-01-15 with reprex v2.0.2

Session info
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.3.2 (2023-10-31)
#>  os       macOS Sonoma 14.2
#>  system   aarch64, darwin23.0.0
#>  ui       unknown
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       Europe/Belgrade
#>  date     2024-01-15
#>  pandoc   3.1.7 @ /opt/homebrew/bin/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package      * version date (UTC) lib source
#>  cli            3.6.2   2023-12-11 [1] CRAN (R 4.3.1)
#>  curl           5.1.0   2023-10-02 [1] CRAN (R 4.3.1)
#>  digest         0.6.33  2023-07-07 [1] CRAN (R 4.3.1)
#>  evaluate       0.21    2023-05-05 [1] CRAN (R 4.3.1)
#>  fastmap        1.1.1   2023-02-24 [1] CRAN (R 4.3.1)
#>  fs             1.6.3   2023-07-20 [1] CRAN (R 4.3.1)
#>  glue           1.6.2   2022-02-24 [1] CRAN (R 4.3.1)
#>  htmltools      0.5.6   2023-08-10 [1] CRAN (R 4.3.1)
#>  jsonlite       1.8.7   2023-06-29 [1] CRAN (R 4.3.1)
#>  jsonvalidate   1.4.2   2023-11-07 [1] Github (ropensci/jsonvalidate@53600f3)
#>  knitr          1.43    2023-05-25 [1] CRAN (R 4.3.1)
#>  lifecycle      1.0.4   2023-11-07 [1] CRAN (R 4.3.1)
#>  magrittr       2.0.3   2022-03-30 [1] CRAN (R 4.3.1)
#>  purrr          1.0.2   2023-08-10 [1] CRAN (R 4.3.1)
#>  R.cache        0.16.0  2022-07-21 [1] CRAN (R 4.3.1)
#>  R.methodsS3    1.8.2   2022-06-13 [1] CRAN (R 4.3.1)
#>  R.oo           1.25.0  2022-06-12 [1] CRAN (R 4.3.1)
#>  R.utils        2.12.2  2022-11-11 [1] CRAN (R 4.3.1)
#>  R6             2.5.1   2021-08-19 [1] CRAN (R 4.3.1)
#>  Rcpp           1.0.11  2023-07-06 [1] CRAN (R 4.3.1)
#>  reprex         2.0.2   2022-08-17 [1] CRAN (R 4.3.1)
#>  rlang          1.1.2   2023-11-04 [1] CRAN (R 4.3.1)
#>  rmarkdown      2.25    2023-09-18 [1] CRAN (R 4.3.1)
#>  sessioninfo    1.2.2   2021-12-06 [1] CRAN (R 4.3.1)
#>  styler         1.10.2  2023-08-29 [1] CRAN (R 4.3.1)
#>  V8             4.4.0   2023-10-09 [1] CRAN (R 4.3.1)
#>  vctrs          0.6.5   2023-12-01 [1] CRAN (R 4.3.1)
#>  withr          2.5.2   2023-10-30 [1] CRAN (R 4.3.1)
#>  xfun           0.40    2023-08-09 [1] CRAN (R 4.3.1)
#>  yaml           2.3.7   2023-01-23 [1] CRAN (R 4.3.1)
#> 
#>  [1] /opt/homebrew/lib/R/4.3/site-library
#>  [2] /opt/homebrew/Cellar/r/4.3.2/lib/R/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

The example above is a reprex, but I see the same behavior for definitions defined in separate files ("$ref": "x.json") as well. at a first guess, this seems like the sub-schemas aren't propagating auto_unbox, but I'm out of my depth on JS/V8.

Can't resolve path to sub-schemas in sub-folders

Hello,
I am trying to validate a json file with nested references. Let's say a 'user' json file with its 'address' sub-schema. 'address' also has a sub-schema. The sub-schemas are all contained in a 'sub-schema/' folder.
The files are organised as following:

root/schemas/user
root/schemas/sub-schemas/address
root/schemas/sub-schemas/city

and they are referenced user -> sub-schemas/address -> sub-schemas/city

Here's the reproducible code in R which gives the error:

library(jsonvalidate)
library(testthat)

# Prepare paths and folder structure ----
root = file.path(tempdir(), "jsonvalidate")
dir = "schemas"
subdir = "sub-schemas"

dir.create(file.path(root, dir, subdir), recursive = TRUE, showWarnings = FALSE)

user_file = "user"
user_schema_path = file.path(root, dir, user_file)

address_file = "address"
address_rel_path = file.path(subdir, address_file)

city_file = "city"
city_rel_path = file.path(subdir, city_file)

# Json and schemas ----
json_to_validate = '
{
  "address":{
    "city":"Firenze"
  }
}
'
user_schema_ref = sprintf('
{
  "$schema": "http://json-schema.org/draft-07/schema",
  "$id": "file://%s",
  "type":"object",
  "required":["address"],
  "properties":{
    "address":{
      "$ref": "%s"
    }
  }
}
', user_schema_path, address_rel_path)

address_schema = sprintf('
{
  "$schema": "http://json-schema.org/draft-07/schema",
  "type":"object",
  "properties":{
    "city":{
      "$ref": "%s"
    }
  }
}
', city_rel_path)


city_schema = sprintf('
{
  "$schema": "http://json-schema.org/draft-07/schema",
  "type":"string",
  "enum":["Firenze"]
}
')

cat(user_schema_ref)
#> 
#> {
#>   "$schema": "http://json-schema.org/draft-07/schema",
#>   "$id": "file:///var/folders/yq/82fd9hrj1zs7kx2hqbqlg9280000gp/T//Rtmp0N5QNb/jsonvalidate/schemas/user",
#>   "type":"object",
#>   "required":["address"],
#>   "properties":{
#>     "address":{
#>       "$ref": "sub-schemas/address"
#>     }
#>   }
#> }
cat(address_schema)
#> 
#> {
#>   "$schema": "http://json-schema.org/draft-07/schema",
#>   "type":"object",
#>   "properties":{
#>     "city":{
#>       "$ref": "sub-schemas/city"
#>     }
#>   }
#> }
cat(city_schema)
#> 
#> {
#>   "$schema": "http://json-schema.org/draft-07/schema",
#>   "type":"string",
#>   "enum":["Firenze"]
#> }

# Write files ----
write(user_schema_ref, user_schema_path)
write(address_schema, file.path(root, dir, address_rel_path))
write(city_schema, file.path(root, dir, city_rel_path))

list.files(root, recursive = TRUE)
#> [1] "schemas/sub-schemas/address" "schemas/sub-schemas/city"   
#> [3] "schemas/user"

# Validate ----
json_validate(entry1, user_schema_path, engine = "ajv",
              verbose = TRUE, greedy = TRUE, 
              error = FALSE)
#> Error in context_eval(join(src), private$context, serialize, await): Error: can't resolve reference sub-schemas/city from id sub-schemas/address

Created on 2022-08-29 with reprex v2.0.2

I managed to make it work if I put the files all in the same folder:

root/schemas/user
root/schemas/address
root/schemas/city

And I reference as following:

{
  "$schema": "http://json-schema.org/draft-07/schema",
  "$id": "file:///var/folders/yq/82fd9hrj1zs7kx2hqbqlg9280000gp/T//Rtmpguupsl/jsonvalidate/schemas/user",
  "type":"object",
  "required":["address"],
  "properties":{
    "address":{
      "$ref": "address"
    }
  }
}
{
  "$schema": "http://json-schema.org/draft-07/schema",
  "type":"object",
  "properties":{
    "city":{
      "$ref": "city"
    }
  }
}

But, as soon as I move the sub-schemas in a sub-folder I can't make it work.

Thank you!

Support for validating sub-schemas?

I was wondering if this package might also support validating sub-schemas, e.g. giving a reference within a schema to validate, as ajv (which looks like it will be supported soon, thanks @karawoo , @ijlyttle, @richfitz ! ) supports that kind of validation. It can be useful to verify that a specific part of a json is valid, for example when building up a json bit by bit. (I'm not actually sure is-my-json-valid doesn't support it, but not obvious to me how to go about it using that package).

Here is a little example using ajv within R (using the use-ajv branch of jsonvalidate) to validate just the 'Axis' definition within the Vega-Lite schema:

library(jsonvalidate)

vl_schema <- readr::read_file(
  "https://raw.githubusercontent.com/vegawidget/vegawidget/8d95a9ac74ea0e590259a88b9d5fa8e127f54fb9/inst/schema/vega-lite/v3.3.0.json")

jsonvalidate:::env$ct$eval(
  sprintf("subTest = new Ajv({verbose: true}).addMetaSchema(AjvSchema6).addSchema(%s,'vega-lite');",
          vl_schema)
)

ref <- "#/definitions/Axis"

jsonvalidate:::env$ct$eval(sprintf("subVal = subTest.getSchema('%s');", ref))

test_json_positive = jsonlite::toJSON(
  list(domain = TRUE), 
  auto_unbox = TRUE, 
  null = "null")

res_yes <- jsonvalidate:::env$ct$call("subVal", V8::JS(test_json_positive))
res_yes #Should be TRUE

test_json_negative = jsonlite::toJSON(
  list(domain = 1), 
  auto_unbox = TRUE, 
  null = "null")

res_no <- jsonvalidate:::env$ct$call("subVal", V8::JS(test_json_negative))
res_no #should be FALSE

In terms of how this would fit into the current jsonvalidate API, I would propose that json_validator and json_validate can both get an optional reference parameter. If specified then a validator/validation for that reference is made rather than the full json schema.

If that addition (with either the API I am proposing or an alternate approach for incorporating) would be welcome in this package, I'd be happy to put together a PR! Just want to check first whether such an addition might fit into this package or not...

Restructure tests

As the package has grown, the flat test structure is increasingly annoying - it should be broken up a bit

Support for external references in json schema

What is the extent of the support for external references defined with $ref?

Can I referenece a file or url in a json schema?

Here is my reprex...

conceptSchemaFilename <- tempfile(fileext = ".json")
readr::write_file('
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://ohdsi.org/concept.schema.json",
  "title": "concept",
  "description": "An OMOP concept expression",
  "type": "object",
  "properties": {
    "concept": {
      "description": "An OMOP concept",
      "type": "object",
      "properties": {
        "CONCEPT_CLASS_ID": {
          "type" : "string"
        },
        "CONCEPT_CODE": {
          "type" : "string"
        },
        "CONCEPT_ID": {
          "type" : "integer",
          "minimum": 0
        },
        "CONCEPT_NAME": {
          "type" : "string"
        },
        "DOMAIN_ID": {
          "type" : "string"
        },
        "INVALID_REASON": {
          "type" : "string"
        },
        "INVALID_REASON_CAPTION": {
          "type": "string"
        },
        "STANDARD_CONCEPT": {
          "type": "string",
          "enum": ["S", "N", "C"]
        },
        "STANDARD_CONCEPT_CAPTION": {
          "type": "string"
        },
        "VOCABULARY_ID": {
          "type": "string"
        }
      },
      "required": ["CONCEPT_CLASS_ID", "CONCEPT_CODE", "CONCEPT_ID", "CONCEPT_NAME", "DOMAIN_ID", "INVALID_REASON", "INVALID_REASON_CAPTION", "STANDARD_CONCEPT", "STANDARD_CONCEPT_CAPTION", "VOCABULARY_ID"]
    },
    "isExcluded": {
      "description": "Should this concept be excluded from the concept set?",
      "type": "boolean"
    },
    "includeDescendants": {
      "description": "Should descendants be included/excluded? (true or false)",
      "type": "boolean"
    }
  },
  "required": [ "concept" ]
}
', conceptSchemaFilename) 


conceptSetJsonSchema <- paste('
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://ohdsi.org/conceptSet.schema.json",
  "title": "conceptSet",
  "description": "An OMOP concept set expression",
  "type": "object",
  "properties": {
    "items": {
      "desription": "An array of concept expressions",
      "type": "array",
      "items": {
        "$ref": "file:///', conceptSchemaFilename, '#/properties/concept"
      }
    }
  }
}
')

validator <- jsonvalidate::json_schema$new(conceptSetJsonSchema)
#> Error in read_schema_dependencies(string, children, c("(string)", parent), : Don't yet support protocol-based sub schemas

jsonToValidate <- '
{
  "items": [
    {
      "concept": {
        "CONCEPT_CLASS_ID": "Read",
        "CONCEPT_CODE": "E253.11",
        "CONCEPT_ID": 45486750,
        "CONCEPT_NAME": "\"Bad trips\"",
        "DOMAIN_ID": "Condition",
        "INVALID_REASON": "V",
        "INVALID_REASON_CAPTION": "Valid",
        "STANDARD_CONCEPT": "N",
        "STANDARD_CONCEPT_CAPTION": "Non-Standard",
        "VOCABULARY_ID": "Read"
      },
      "isExcluded": false,
      "includeDescendants": false,
      "includeMapped": false
    },
    {
      "concept": {
        "CONCEPT_CLASS_ID": "Clinical Finding",
        "CONCEPT_CODE": "242395002",
        "CONCEPT_ID": 4072955,
        "CONCEPT_NAME": "Fall due to trip on loose carpet",
        "DOMAIN_ID": "Observation",
        "INVALID_REASON": "V",
        "INVALID_REASON_CAPTION": "Valid",
        "STANDARD_CONCEPT": "S",
        "STANDARD_CONCEPT_CAPTION": "Standard",
        "VOCABULARY_ID": "SNOMED"
      },
      "isExcluded": false,
      "includeDescendants": false,
      "includeMapped": false
    }
  ]
}'

Created on 2022-06-19 by the reprex package (v2.0.1)

Regex flag when specifying a pattern

Hi,

When specifying a pattern using a regular expression, one can not use the case insensitive flag in the regex.

Reproducible example (using dplyr):

library(dplyr)
'
{
	"type": "object",
	"properties":{
	  "product_type": {
      "type": ["string"],
      "pattern": "^(?i)test$"
    }
	},
  "required": [
	  "product_type"
  ],
}
' %>% jsonvalidate::json_validator(engine = "ajv") -> validator

Latest CRAN headaches

Solaris has an ancient version of V8 (of course) so try forcing ES5 in the generated schemas.

The builder for r-devel-linux-x86_64-fedora-clang segfaults (!) which is surprising given we contain no compiled code. It's possible that the ES5 change will fix it (see trace below from r-hub) but we can't replicate the segfault anywhere. Most likely it is due to a certain professor creating an overly complicated installation of V8 (see https://www.stats.ox.ac.uk/pub/bdr/Rconfig/r-devel-linux-x86_64-fedora-clang which has a very low chance of being recreated anywhere outside that machine).

Details
> rhub::check("jsonvalidate_1.3.1.tar.gz",  platform = "fedora-clang-devel")
─  Uploading package
─  Preparing build, see status at
   https://builder.r-hub.io/status/jsonvalidate_1.3.1.tar.gz-3d8c3523dc6c4225a768e3eeab08ba48
-^C
─  Build started
─  Downloading and unpacking package file
─  Querying system requirements
─  Installing system requirements
─  Starting Docker container
─  Querying package dependencies
─  Installing package dependencies
─  Running R CMD check
   About to run xvfb-run R CMD check jsonvalidate_1.3.1.tar.gz
   Error : Bioconductor does not yet build and check packages for R version 4.2; see
     https://bioconductor.org/install
─  using log directory ‘/home/docker/jsonvalidate.Rcheck’
─  using R Under development (unstable) (2021-10-18 r81073)
─  using platform: x86_64-pc-linux-gnu (64-bit)
─  using session charset: UTF-8
✔  checking for file ‘jsonvalidate/DESCRIPTION’
─  this is package ‘jsonvalidate’ version ‘1.3.1’
─  package encoding: UTF-8
✔  checking package namespace information
✔  checking package dependencies
✔  checking if this is a source package
✔  checking if there is a namespace
✔  checking for executable files
✔  checking for hidden files and directories
✔  checking for portable file names
✔  checking for sufficient/correct file permissions
✔  checking whether package ‘jsonvalidate’ can be installed
✔  checking installed package size (9.3s)
✔  checking package directory
✔  checking ‘build’ directory
✔  checking DESCRIPTION meta-information
✔  checking top-level files
✔  checking for left-over files
✔  checking index information
✔  checking package subdirectories
✔  checking R files for non-ASCII characters
✔  checking R files for syntax errors
✔  checking whether the package can be loaded
✔  checking whether the package can be loaded with stated dependencies
✔  checking whether the package can be unloaded cleanly
✔  checking whether the namespace can be loaded with stated dependencies
✔  checking whether the namespace can be unloaded cleanly
✔  checking loading without being on the library search path
✔  checking dependencies in R code
✔  checking S3 generic/method consistency
✔  checking replacement functions
✔  checking foreign function calls
✔  checking R code for possible problems
✔  checking Rd files
✔  checking Rd metadata
✔  checking Rd cross-references
✔  checking for missing documentation entries
✔  checking for code/documentation mismatches
✔  checking Rd \usage sections
✔  checking Rd contents
✔  checking for unstated dependencies in examples
✔  checking installed files from ‘inst/doc’
✔  checking files in ‘vignettes’
E  checking examples (9.2s)
   Running examples in ‘jsonvalidate-Ex.R’ failed
   The error most likely occurred in:
   
   > ### Name: json_validate
   > ### Title: Validate a json file
   > ### Aliases: json_validate
   > 
   > ### ** Examples
   > 
   > # A simple schema example:
   > schema <- '{
   +     "$schema": "http://json-schema.org/draft-04/schema#",
   +     "title": "Product",
   +     "description": "A product from Acme\'s catalog",
   +     "type": "object",
   +     "properties": {
   +         "id": {
   +             "description": "The unique identifier for a product",
   +             "type": "integer"
   +         },
   +         "name": {
   +             "description": "Name of the product",
   +             "type": "string"
   +         },
   +         "price": {
   +             "type": "number",
   +             "minimum": 0,
   +             "exclusiveMinimum": true
   +         },
   +         "tags": {
   +             "type": "array",
   +             "items": {
   +                 "type": "string"
   +             },
   +             "minItems": 1,
   +             "uniqueItems": true
   +         }
   +     },
   +     "required": ["id", "name", "price"]
   + }'
   > 
   > # Test if some (invalid) json conforms to the schema
   > jsonvalidate::json_validate("{}", schema, verbose = TRUE)
   Error in context_eval(join(src), private$context, serialize) : 
     SyntaxError: Use of const in strict mode.
   Calls: <Anonymous> ... <Anonymous> -> evaluate_js -> get_str_output -> context_eval
   Execution halted
✔  checking for unstated dependencies in ‘tests’
─  checking tests
E  Running ‘testthat.R’
   Running the tests in ‘tests/testthat.R’ failed.
   Last 13 lines of output:
       2. │ └─testthat:::expect_condition_matching(...)
       3. │   └─testthat:::quasi_capture(...)
       4. │     ├─testthat:::.capture(...)
       5. │     │ └─base::withCallingHandlers(...)
       6. │     └─rlang::eval_bare(quo_get_expr(.quo), quo_get_env(.quo))
       7. └─jsonvalidate::json_validator(schema, engine = "ajv")
       8.   └─jsonvalidate:::jsonvalidate_js()
       9.     └─ct$source(system.file("bundle.js", package = "jsonvalidate"))
      10.       └─V8:::evaluate_js(readLines(file, encoding = "UTF-8", warn = FALSE))
      11.         ├─V8:::get_str_output(...)
      12.         └─V8:::context_eval(join(src), private$context, serialize)
     
     [ FAIL 32 | WARN 0 | SKIP 0 | PASS 30 ]
     Error: Test failures
     Execution halted
✔  checking for unstated dependencies in vignettes
✔  checking package vignettes in ‘inst/doc’
─  checking running R code from vignettes
     ‘jsonvalidate.Rmd’ using ‘UTF-8’... OK
    NONE
W  checking re-building of vignette outputs
   Error(s) in re-building vignettes:
     ...
   --- re-building ‘jsonvalidate.Rmd’ using rmarkdown
   Quitting from lines 121-122 (jsonvalidate.Rmd) 
   Error: processing vignette 'jsonvalidate.Rmd' failed with diagnostics:
   SyntaxError: Use of const in strict mode.
   --- failed re-building ‘jsonvalidate.Rmd’
   
   SUMMARY: processing the following file failed:
     ‘jsonvalidate.Rmd’
   
   Error: Vignette re-building failed.
   Execution halted
   
✔  checking PDF version of manual
   
─  Done with R CMD check
─  Saving artifacts
    

Dev version fails with biocompute

* checking examples with --run-donttest ... ERROR
Running examples in ‘biocompute-Ex.R’ failed
The error most likely occurred in:

> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
> ### Name: validate_schema_v1.3.0
> ### Title: BioCompute Objects schema validator (v1.3.0)
> ### Aliases: validate_schema_v1.3.0 validate_schema
> 
> ### ** Examples
> 
> bco <- tempfile(fileext = ".json")
> generate_example("HCV1a") %>%
+   convert_json() %>%
+   export_json(bco)
> bco %>% validate_schema()
── 0: Validating BioCompute Object ─────────────────────────────────────────────
Error: While reading 'biocomputeobject.json' > 'provenance_domain.json' > 'biocomputeobject.json#/definitions/contributor'
Did not find schema file 'biocomputeobject.json#/definitions/contributor'
Execution halted

Regression: cannot create validator for vega-lite schema, problem with finding dependencies

A schema that is readable by ajv fails to be read in correctly with latest development version of jsonvalidate, choking on the new logic to try to find dependencies.

Example code to read in:

library(jsonvalidate)
sc <- readr::read_file("https://raw.githubusercontent.com/vegawidget/vegawidget/master/inst/schema/vega-lite/v3.3.0.json")
val <- jsonvalidate::json_validator(sc, engine = "ajv")

The error message is:
Error in context_eval(join(src), private$context) :
TypeError: Cannot use 'in' operator to search for '$ref' in null

And the traceback shows it is coming from a call to v8$call("find_reference", V8::JS(schema))

The issue is with the the code for finding the dependencies, but I haven't tracked down the exact cause yet. This schema works just fine with ajv if passing it in as is without doing the dependency hunting.

JSON schema referencing local file validates all input as valid

Using a $ref with URI of relative path to another schema marks all input as valid

> schema <- '{
+     "id": "Response",
+     "type": "object",
+     "properties": {
+       "example": { "$ref": "Example.schema.json" }
+     },
+     "additionalProperties": false,
+     "required": [ "example" ]
+   }'
> jsonvalidate::json_validate('{"example": "test"}', schema)
[1] TRUE
> jsonvalidate::json_validate('{"example": false}', schema)
[1] TRUE
> jsonvalidate::json_validate('{"example": 123}', schema)
[1] TRUE

As an aside also noticed that README references validate_json where exported function is json_validate.

validating unique key values in array of objects

Firstly I'd like to thank you for this incredibly useful wrapper!

I apologise if the answer to my question should be clear, and whether my use of terminology is mispalced. I am very new to JSON schema and validation of json structures within R, and despite attempting to find the answer within the documentation of jsonvalidate, ajv, and JSON Schema, it's still not completely clear if what I am looking to do is possible with this libary.

I am wondering if there is a way to validate that the values of keys for an array of objects are unique.

This problem has been well-discussed previously here: Key-based item uniqueness

For example, given the schema:

{
  "type" : "array",
  "items" : {
    "type" : "object",
    "properties" : {
      "foo" : { "type" : "integer" },
      "bar" : { "type" : "string" }
    }
  },
  "uniqueItems" : true
}

The instance:

[ { "foo" : 1, "bar" : "value1" }, { "foo" : 1, "bar" : "value2" } ]

Currently passes as there is no vocabulary(?) for validating that a value is repeated in the given data.

In my efforts to find a way to do this, I came across two possible options generally speaking:

Firstly, by defining a custom keyword for ajv, as described here: https://ajv.js.org/packages/ajv-keywords.html#uniqueitemproperties

Is it possible to define keywords to be used by the ajv engine when using jsonvalidate?

Another solution I found was resulting from the Key-based item uniqueness discussion I mentioned earlier. Using a meta-schema "vocabulary" extending the validation of arrays.

Is it possible to extend the vocabulary of the 'legal' draft schema within jsonvalidate?

Again I apologise if this should be clear that it is beyond the intention of this library, but I wanted to clarify these questions before resolving this situation using other means!

Thank you in advance

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.