Apply honestdid on did2s results

[asheshrambachan]

Apply honestdid on did2s results

[jonathandroth]

Thanks for your Q! Is it possible to give a minimal working example that throws the error? (E.g. using an example public dataset or a deidentified version of your data that you can share publicly)

…

On Mon, Mar 13, 2023 at 2:24 PM Randol340 ***@***.***> wrote: Dear authors, I'm trying to apply the output of did2s to honestdid. But there are errors so I wonder if you have any ideas about it? Thank you very much!! method <- NULL Mvec <- NULL Mbarvec <- NULL monotonicityDirection <- NULL biasDirection <- NULL alpha <- 0.05 parallel <- FALSE robust_ci <- createSensitivityResults( betahat = beta, sigma = V, numPrePeriods = npre, numPostPeriods = npost, method = method, l_vec = baseVec1, monotonicityDirection = monotonicityDirection, biasDirection = biasDirection, alpha = alpha, parallel = parallel ) I got an error message Error in { : task 1 failed - "attempt to apply non-function". I wonder if you have any ideas? Thank you very much! [image: image] <https://user-images.githubusercontent.com/78134871/224826691-b32f3a16-3a53-4287-a636-638e28c07b51.png> — Reply to this email directly, view it on GitHub <#25>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6EXFBCDFY7UNEIDVAFN7LW36GARANCNFSM6AAAAAAVZTWZ4M> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Randol340]

Thank you very much for your reply! I've prepared the data and code and sent them to you via email. Thank you again for your help!

[mcaceresb]

@Randol340 I didn't get any errors when I ran your code. How are you running it? Maybe the output of

library(here)
library(dplyr)
library(did)
library(haven)
library(ggplot2)
library(fixest)
library(HonestDiD)
library(data.table)
library(did2s)
sessionInfo()

will be informative. Incidentally, you can post a MWE here since you only need to define betahat and V from data (but you don't need the underlying data to post betahat and V).

[Randol340]

Thank you very much!! Here is the output of running the codes above:

R version 4.2.2 (2022-10-31)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Ventura 13.0.1

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] did2s_0.7.0       data.table_1.14.6 HonestDiD_0.2.2   fixest_0.11.1     ggplot2_3.4.0    
[6] haven_2.5.1       did_2.1.2         dplyr_1.1.0       here_1.0.1       

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.10         lattice_0.20-45     tidyr_1.2.1         zoo_1.8-11         
 [5] rprojroot_2.0.3     digest_0.6.31       foreach_1.5.2       utf8_1.2.3         
 [9] gmp_0.7-1           slam_0.1-50         R6_2.5.1            backports_1.4.1    
[13] evaluate_0.19       pillar_1.8.1        rlang_1.0.6         rstudioapi_0.14    
[17] car_3.1-1           Matrix_1.5-1        rmarkdown_2.19      osqp_0.6.0.8       
[21] readr_2.1.3         stringr_1.5.0       bit_4.0.5           munsell_0.5.0      
[25] broom_1.0.2         compiler_4.2.2      numDeriv_2016.8-1.1 xfun_0.36          
[29] pkgconfig_2.0.3     BMisc_1.4.5         CVXR_1.0-11         Rglpk_0.6-4        
[33] htmltools_0.5.4     tidyselect_1.2.0    tibble_3.1.8        codetools_0.2-18   
[37] fansi_1.0.4         tzdb_0.3.0          withr_2.5.0         ggpubr_0.5.0       
[41] grid_4.2.2          nlme_3.1-160        gtable_0.3.1        lifecycle_1.0.3    
[45] magrittr_2.0.3      scales_1.2.1        cli_3.6.0           stringi_1.7.12     
[49] carData_3.0-5       dreamerr_1.2.3      ggsignif_0.6.4      Rmpfr_0.9-1        
[53] ellipsis_0.3.2      generics_0.1.3      vctrs_0.5.2         sandwich_3.0-2     
[57] Formula_1.2-4       iterators_1.0.14    tools_4.2.2         forcats_0.5.2      
[61] bit64_4.0.5         glue_1.6.2          purrr_1.0.1         hms_1.1.2          
[65] abind_1.4-5         fastmap_1.1.0       yaml_2.3.6          colorspace_2.0-3   
[69] rstatix_0.7.1       ECOSolveR_0.5.4     knitr_1.41      

What do you mean "didn't get any errors"? Does the commend createSensitivityResults give the expected plot? Thank you very much!

[mcaceresb]

@Randol340 Mmm... nothing odd at a glance.

And I meant that I don't see the error you report here. Here's a MWE I can run without issue (unzip the archive for the files I use):

betahat_V.zip

library(HonestDiD)

.loadmat <- function(inf) {
    con <- file(inf, 'rb')
    on.exit(close(con))
    nr <- readBin(con, "int", size=4)
    nc <- readBin(con, "int", size=4)
    mat <- matrix(readBin(con, "numeric", nr * nc, size=8), ncol=nc, nrow=nr, byrow=TRUE)
    flush(con)
    return(mat)
}

betahat               <- .loadmat("betahat.bin")
V                     <- .loadmat("V.bin")
npre                  <- 14
npost                 <- 29
method                <- NULL
baseVec1              <- basisVector(index = 1, size = npost)
monotonicityDirection <- NULL
biasDirection         <- NULL
alpha                 <- 0.05
parallel              <- FALSE

robust_ci <- createSensitivityResults(
  betahat               = betahat,
  sigma                 = V,
  numPrePeriods         = npre,
  numPostPeriods        = npost,
  method                = method,
  l_vec                 = baseVec1,
  monotonicityDirection = monotonicityDirection,
  biasDirection         = biasDirection,
  alpha                 = alpha,
  parallel              = parallel
)

[Randol340]

Sorry for the late reply and thank you very much! I successfully get the following results!!

Sensitivity analysis using relative magnitudes restrictions:

Sensitivity Analysis Using Smoothness Restrictions:

Following the documentation, the results I get mean that 1) a significant result is robust to allowing for violations of parallel trends up to 0.5 times as big as the max violation in the pre-treatment period, and 2) we can reject a null effect unless we are willing to allow for the linear extrapolation across consecutive periods to be off by more than 0.08 percentage points.

Could you help me in understanding the interpretations better: what are "max violation in the pre-treatment period" and "linear extrapolation across consecutive periods"? Especially, how do you think that help speak to the parallel pretend assumption in our setting?

Also attach the Gardner estimates event study for your reference:

Thank you so much for your help!!!

[jonathandroth]

Hi there, I think what you wrote about the interpretation is correct. We describe the formal meaning of these restrictions in Section 2.4 of the paper <https://www.jonathandroth.com/assets/files/HonestParallelTrends_Main.pdf>. You might also check out slides 24-29 here <https://github.com/Mixtape-Sessions/Advanced-DID/blob/main/Slides/03-violations.pdf> for a short summary. The applications in the paper also discuss the interpretations in applied contexts. Best, Jon

…

On Thu, Mar 30, 2023 at 7:30 PM Randol340 ***@***.***> wrote: Sorry for the late reply and thank you very much! I successfully get the following results!! Sensitivity analysis using relative magnitudes restrictions: [image: image] <https://user-images.githubusercontent.com/78134871/228986913-6b10efc0-1f41-40f3-9268-736ec767505c.png> Sensitivity Analysis Using Smoothness Restrictions: [image: image] <https://user-images.githubusercontent.com/78134871/228984844-96b71b81-f7e4-46e5-a30d-88b4b55fa94c.png> Following the documentation, the results I get mean that 1) a significant result is robust to allowing for violations of parallel trends up to 0.5 times as big as the max violation in the pre-treatment period, and 2) we can reject a null effect unless we are willing to allow for the linear extrapolation across consecutive periods to be off by more than 0.08 percentage points. Could you help me in understanding the interpretations better: what are "max violation in the pre-treatment period" and "linear extrapolation across consecutive periods"? Especially, how do you think that help speak to the parallel pretend assumption in our setting? Also attach the Gardner estimates event study for your reference: [image: image] <https://user-images.githubusercontent.com/78134871/228985952-dd80ec03-a4b6-4ca3-92d4-067488e0c6cd.png> Thank you so much for your help!!! — Reply to this email directly, view it on GitHub <#25 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6EXFASTFPTB33G6EBNSYDW6YJRFANCNFSM6AAAAAAVZTWZ4M> . You are receiving this because you commented.Message ID: ***@***.***>

[Randol340]

Hi Jon,

Thank you so much for your and Mauricio's continuous support! We really appreciate it!

Best regards,
Randol

[jonathandroth]

My pleasure - thanks for using our tools :)

…

On Mon, Apr 3, 2023 at 3:40 PM Randol340 ***@***.***> wrote: Hi Jon, Thank you so much for your and Mauricio's continuous support! We really appreciate it! Best regards, Randol — Reply to this email directly, view it on GitHub <#25 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6EXFECELG4UMGIE4HSSR3W7MRTFANCNFSM6AAAAAAVZTWZ4M> . You are receiving this because you commented.Message ID: ***@***.***>