easonfg/cali_tutorial

cali_tutorial's Issues

Multiplication by 2 to obtain the p-value of a two-tailed Spiegelhalter's z-test

In Line 11 in Spiegelhalter_z.R:
cat('p value: ', 1-pnorm(abs(z_score)), '\n')
the p-value should be multiplied by 2 to obtain the p-value of a two-tailed Spiegelhalter's z-test. As a result, the line could be:
cat('p value: ', 2 * (1-pnorm(abs(z_score))), '\n')
or
cat('p value: ', 2 * pnorm(- abs(z_score)), '\n')

Small error in the H-L-C calculation

Thanks for your paper and providing this code, very useful!

I just quickly wanted to flag that there is a little error in the H-L-C calculation when splitting the dataset into equal chunks. If the dataset size isn't a multiple of g, the nrow(data) % g rows with the highest predicted scores are grouped with the lowest decile group. The issue is in the following line:

cali_tutorial/hosmer_lemeshow.R

Line 10 in cb854b3

split_mtx = split(mtx, rep(1:ceiling(nr/n), each=n, length.out=nr))

Minimal example

g = 3
y = rep(c(0, 1), each = 5)
prob = seq(0, 1, length.out = 10)

mtx = cbind(y, y_not = 1- y, prob, prob_not = 1-prob)
mtx = as.data.frame(mtx)
mtx = mtx[order(mtx$prob),]
n <- length(prob)/g
nr <- nrow(mtx)

split(mtx, rep(1:ceiling(nr/n), each=n, length.out=nr))
#> $`1`
#>    y y_not      prob  prob_not
#> 1  0     1 0.0000000 1.0000000
#> 2  0     1 0.1111111 0.8888889
#> 3  0     1 0.2222222 0.7777778
#> 10 1     0 1.0000000 0.0000000    <-- wrongly put in lowest class
#> 
#> $`2`
#>   y y_not      prob  prob_not
#> 4 0     1 0.3333333 0.6666667
#> 5 0     1 0.4444444 0.5555556
#> 6 1     0 0.5555556 0.4444444
#> 
#> $`3`
#>   y y_not      prob  prob_not
#> 7 1     0 0.6666667 0.3333333
#> 8 1     0 0.7777778 0.2222222
#> 9 1     0 0.8888889 0.1111111

Easy fix

An easy fix could be to use dplyr::ntile instead or rep.