conjugateprior/cbn

An R package to replicate Caliskan, Bryson and Narayanan 2017, add new comparison functions, and use your own word vectors.

cbn's Issues

Make travis and R believe in C++14

My basket lists are all different lengths

 res <- weat_boot(alldone, wordvecs,
+                  x_name = "DescriptivePos", y_name = "DescriptiveNeg",
+                  a_name = "NormPos", b_name = "NormNeg",
+                  se.calc = "quantile")
Error in vectors[y_words, ] : subscript out of bounds

> table(alldone$Condition)

       NormPos        NormNeg DescriptivePos DescriptiveNeg 
            59             53             69             50 

brokenbaskets.csv.zip

When would "Analyze Your Vectors (To Be Written)" section be finished?

Hello there :)

Thanks for the nice instructions on how to replicate your study. It's really helpful and saves me a lot of time. I was wondering when I can expect the Analyze Your Vectors section to be complete? I'm looking forward to comparing my method with the original creator's 👍

p-value

Hi,

While reproducing some of the results, I have come across some incongruity in the p-values.

In the paper, it was reported that WEAT1 has a p-value of 10^−7. However, the provided code was showing up as p = 0. So, I looked at the weat_perm and noticed that p_val <- sum(reps > S_xyab)/length(reps)

So, I've added this code on the original weat_perm function:

  reps[i] <- sum(S_xab) - sum(S_yab)
  # reps' object of i is the subtraction of those two

  }
  print(reps[i])
  print(S_xyab)
  print(sum(reps > S_xyab))
  print(length(reps))
  
  p_val <- sum(reps > S_xyab)/length(reps) 

which returned

[1] -0.4718922
[1] 2.238165
[1] 0
[1] 1000
    S_xyab        d p_value
1 2.238165 1.504315       0

I take it sum(reps > S_xyab) most likely returns 0, thus the 0 p-value. But, how was the p-value obtained in the original study? Did you use a different method for the paper publication?

Many thanks.

WEFAT 1 tba

Hello!

I played around with the examples and successfully replicated WEAT using my own target words.

I want to learn and acquire more insight about WEFAT from your website, but I noticed that WEFAT1 shows "tba" When would be the appropriate time for me to revisit this page for updated content?

Many thanks,

Jim

Error with weat_perms when using own items and vectors?

Thank you for making your package for WEAT. This is very helpful.

I just started using cbn to compute WEAT scores for own set of targets and attributes. Unfortunately, the effect size and p-value returned are way different from what I obtain for the same word vectors using Python's WEFE package : https://wefe.readthedocs.io/en/latest/api.html#weat

Here's the R code I have that uses cbn package:

library(cbn)
library(stringr)
cbn_set_vectorfile_location("/home/user1/my_corpus_vectors.txt")

its= cbn_make_items("OWN_1", 
                    words,
                    conditions,
                    roles)
summary(its)

Condition      Role N
    Female attribute 5
      Male attribute 5
      Arts    target 8
   Science    target 8

vecs <- cbn_extract_word_vectors(words, verbose= TRUE)
weat_perm(its, vecs, x_name= "Science", y_name= "Arts", 
          a_name= "Male", b_name= "Female", 1000)

S_xyab	d	p_value
<dbl>	<dbl>	<dbl>
-0.1394635	-0.1211806	0.58

This is significantly different than effect size of 1.674 obtained using the Python package.

Not sure if my implementation based on your package is incorrect in some way. Could you kindly help?