Error when running sensitivity analysis with honest did

[asheshrambachan]

Error when running sensitivity analysis with honest did

[jonathandroth]

Can you please provide a minimal working example that we can try to use to reproduce the error, either by sharing the data that you used or a synthetic dataset with similar structure that reproduces the error?

[Hanh7]

Hello,
Sorry for my late reply. I put my dataset and description in this folder: https://drive.google.com/drive/folders/19tmsg6HR2k6xC7omqJkTV7lQ48PuBTwC?usp=drive_link

Could you kindly send me a request so that I can share to only you?

[jonathandroth]

@mcaceresb when you have RA time, can you look into this please?

[mcaceresb]

@jonathandroth @Hanh7 I can't download the example. I might not need it

• It seems the error is from this line: basisVector(index=(e+1),size=npost)
• e <- 0,npost <- nperiods - npre, nperiods <- nrow(V) and npre <- sum(1*(es$egt < -1))
• es <- cs_es_nevertreat
• V is based on es$inf.function$dynamic.inf.func.e and referencePeriodIndex <- which(es$egt == -1)

So it seems there's two objects to analyze. What is

es <- cs_es_nevertreat
dim(crossprod(es$inf.function$dynamic.inf.func.e))
es$egt

? My guess is that there's an issue with the egt vector.

[Hanh7]

Dear @mcaceresb,

I am still working on this.
I extracted egt, and this is it:

Is it what it should look like?
Or should I fix the bug in the code of the function honest_did? I am using the code in your example, because my dataset seems to have the same structure with your example. Perhaps I incorrectly transform my dataset; it is not the same as yours?!

By the way, I have made my dataset opened access, if you have time, could you kindly take a look?
https://drive.google.com/drive/folders/19tmsg6HR2k6xC7omqJkTV7lQ48PuBTwC

[mcaceresb]

@Hanh7 I don't think it's necessary. The issue appears to be that the reference period is hard-coded to -1 in honest_did.

You can change it to -2 (I presume) for your use case, divide egt by 2, or copy the updated function from here and pass on the right reference period.

If there's still an issue I'll have a look at the data. LMK.

[Hanh7]

Many thanks for your quick reply. I'll try again.

[Hanh7]

Hello,

May I ask about the meaning of Mbar mentioned in your Honest approach paper?

I understand that Mbar as "The horizontal axis M shows our restriction on violations, i.e. the fraction of the maximum violations in post-treatment periods and the maximum violations in pre-treatment periods".

Please correct me if I am wrong.

[jonathandroth]

I would have said something like "the restriction that the magnitude of the post-treament violations of parallel trends is no more than M times the largest pre-treatment violation" or something like that.

…

On Fri, Sep 29, 2023 at 12:11 PM Hanh Pham Nguyen ***@***.***> wrote: Hello, May I ask about the meaning of Mbar mentioned in your Honest approach paper? I understand that Mbar as "*The horizontal axis M shows our restriction on violations, i.e. the fraction of the maximum violations in post-treatment periods and the maximum violations in pre-treatment periods* ". Please correct me if I am wrong. — Reply to this email directly, view it on GitHub <#45 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6EXFGPBAY2Q2ZJZ7U2QA3X43XLXANCNFSM6AAAAAA4ZGIY5A> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Hanh7]

Yes. I get the idea. Thank you. On Sat, Sep 30, 2023 at 03:38 Jonathan Roth ***@***.***> wrote:

…

I would have said something like "the restriction that the magnitude of the post-treament violations of parallel trends is no more than M times the largest pre-treatment violation" or something like that. On Fri, Sep 29, 2023 at 12:11 PM Hanh Pham Nguyen ***@***.***> wrote: > Hello, > > May I ask about the meaning of Mbar mentioned in your Honest approach > paper? > > I understand that Mbar as "*The horizontal axis M shows our restriction > on violations, i.e. the fraction of the maximum violations in > post-treatment periods and the maximum violations in pre-treatment periods* > ". > > Please correct me if I am wrong. > > — > Reply to this email directly, view it on GitHub > < #45 (comment)>, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AE6EXFGPBAY2Q2ZJZ7U2QA3X43XLXANCNFSM6AAAAAA4ZGIY5A> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#45 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQIZP3CXYUNE654VY4BBKXLX44WU5ANCNFSM6AAAAAA4ZGIY5A> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[mcaceresb]

@Hanh7 We decided to throw an error if the time vector is not coded to be consecutive and have -1 as the reference period. So in this case you'd have to re-code it yourself before using HonestDiD. Cheers.