carloscinelli / benford.analysis Goto Github PK
View Code? Open in Web Editor NEWTools that make it easier to use Benford’s law for data validation and forensic analytics.
Tools that make it easier to use Benford’s law for data validation and forensic analytics.
Maybe we should also allow the analysis of the second digit only. (or an arbitrary digit only).
For instance, if one plots only a chi squared plot, no legend is needed.
Hi! First of all, thank you for this great work. I am using on Power Bi Desktop and works great. However, for some reason, it's timing out in Power Bi Service (cloud). The error message is that:
Execution Timeout
The script execution timed out, please try again later
Could you give me some guidance for this issue?
Thank you!
Current devel data.table gets own precise method for all.equal, which testthat seems to use behind the scene.
It appears that unit test named Negative numbers, simulated log-normal *(-1) has expected results defined as test
data.table which has named vector inside as a column. This does not match to output from your mantissa
function.
This example creates warning messages:
test <- benford(1000)
Verify what is going on.
In print.Benford the labels for the summary stats of the mantissa are missing. They were accidentally excluded.
When the figure is exactly 1e11 (ie 100,000,000,000), the extract.digit() output is 0 instead of 1. Might have some issues about the trunc part. Thanks
Carlos, are you still maintaining this package? I saw that there was an update in 2017 but not much activity otherwise.
I recently was asked to perform a Benford Analysis in my professional work on financial data and your package was very helpful. I am going to write a general paper that might be turned into a vignette and would like to reach out to you if I have specific questions.
Thanks for building this.
Kier
Hello Carlos,
When you are testing whether or not the data supplied to extract.digits()
is numeric data, I think you should be using is.numeric()
. As it stands now, the use of class(data) != "numeric"
produces unexpected errors when the class is "integer"
. Consider the following:
library(benford.analysis)
dat <- data.frame(v1 = 1:5, v2 = c(1, 2, 3, 4, 5))
benford(dat$v1) # produces error
This error is returned because while dat$v1
is clearly numeric, the call class(dat$v1)
will return "integer"
.
On a different note, thank you for a wonderful R package!
-Paul
Hi Carlos,
thanks a lot for this package, it's really useful! I am asking about the meaning of this option, since I am not sure what exactly it means. I am analysing interest rate data and thus I am not sure what exactly it means, that in the documentation it states that "number.of.digits" defines how many first digits to analyse.
Does it care about the decimal separator of a numeric value or does it just take the number in as it is, like 035 for an interest rate of 0.35%?
Thanks a lot for some clarification
I would like to ask if it would be possible to add an option so as to make it possible to not only analyze subsequent digits together with the first digits, but alone.
I know that this is not really in the spirit of the original benford analysis, but it would be interesting to check the distribution of the digits alone and it probably would be pretty easy to implement, or? I will try to come up with something on my own, so that it becomes a bit clearer, what I mean with this request.
All the best,
Johannes
the argument grid=F is not working for the summation graph
remove rootogram from the default plots, 8 plots as default is too much.
Change title: "Barchart of digits" --> "Digits distribution\nBarchart". Where "\n" is a line break.
Change title: "Rootogram of digits" --? "Digits distribution\nRootogram".
Do the same for sencond order but now "Digits distribution\nSecond Order Test - Barchart", "Digits distribution\nSecond Order Test - Rootogram".
Can we remove the extra white-space in the Chi-squared and summation difference plots?
data(corporate.payment)
cp <- corporate.payment$Amount
plot(cp, select = "digits", except = "digits")
plot(cp, select = "none", except = "all")
It might not be clear to users to what the chi-squared difference refers to. Maybe put both next to each other, and improve the description of the plot.
Also, think about a better default legend placement.
Hi Carlos,
since I stumbled across the problem that the Chi-Square tests suffer from excess power, when using a lot of observations. I've implemented a rather simple function to implement the check based on the MAD as proposed by Nigrini (2012, Chapter 7, page 160).
I basically just check the MAD value based on the digits analyzed and then just print out the decision based on Nigrinis intervals. Don't know if that's a valuable contribution, but at least it's an additional point to add, since the Chi-Square test often leads to wrong conclusions.
Cheers,
Johannes
201803_MAD_Conformity.txt
Nigrini, M.J. (2012). Benford's law. Applications for forensic accounting, auditing, and fraud detection (Hoboken, NJ: Wiley).
It was recently pointed out to us that some README.html files (generated
from the corresponding README.md ones) on the CRAN package web pages are
incomplete, missing 'local' images not available from the web page and
in most cases actually not even shipped with the package. This clearly
should be changed, so we will move to using '--self-contained' for the
pandoc conversion to ensure that the README.html files are "complete".
Of course, this implies that all 'local' images used in README.md are
needed in the package sources.
If the images are also used for vignettes or Rd files, you can put them
in the 'vignettes' or 'man/figures' directories. Otherwise, please put
them in the top-level 'tools' directory, or a subdirectory of it.
The CRAN incoming checks in r-devel were changed to perform the pandoc
conversion checks with '--self-contained', and hence will warn about
missing images.
Pls ensure completeness in the next regular update of your package.
For your information, I attach a list of images used in README.md but
missing from the package sources.
Best
-k
@rafaelslins the current critical value for the KS-test seems to be hard coded for the significance level of 5%.
For now, this should be informed in the output of the test, since the significance level is hard coded.
But as an improvement, we should either provide other significance levels as an option, or, instead, provide the p-value of the KS-test (we can use a ks critical value table to get the approximate p-value).
Another alternative is using the base R ks.test function, but we need to double check whether it works well.
Write package vignette.
Although I drop all other options, the function plot.Benford
I use also gives me two plots. Here is an example in reprex
# Load package benford.analysis
library(benford.analysis)
data(census.2009)
# Check conformity
bfd.cen <- benford(census.2009$pop.2009, number.of.digits = 1)
plot(bfd.cen, except = c("second order", "summation", "mantissa", "chi squared","abs diff", "ex summation", "Legend"), multiple = F)
I think a better way is to avoid the second plot.
Here is my session infomation.
"click the button"
#> [1] "click the button"
Created on 2018-11-24 by the reprex package (v0.2.1)
devtools::session_info()
#> ─ Session info ──────────────────────────────────────────────────────────
#> setting value
#> version R version 3.5.1 (2018-07-02)
#> os macOS 10.14
#> system x86_64, darwin15.6.0
#> ui X11
#> language (EN)
#> collate zh_CN.UTF-8
#> ctype zh_CN.UTF-8
#> tz Asia/Shanghai
#> date 2018-11-24
#>
#> ─ Packages ──────────────────────────────────────────────────────────────
#> package * version date lib source
#> assertthat 0.2.0 2017-04-11 [1] CRAN (R 3.5.0)
#> backports 1.1.2 2017-12-13 [1] CRAN (R 3.5.0)
#> base64enc 0.1-3 2015-07-28 [1] CRAN (R 3.5.0)
#> callr 3.0.0 2018-08-24 [1] CRAN (R 3.5.0)
#> cli 1.0.1 2018-09-25 [1] CRAN (R 3.5.0)
#> crayon 1.3.4 2017-09-16 [1] CRAN (R 3.5.0)
#> desc 1.2.0 2018-05-01 [1] CRAN (R 3.5.0)
#> devtools 2.0.1 2018-10-26 [1] CRAN (R 3.5.1)
#> digest 0.6.18 2018-10-10 [1] CRAN (R 3.5.0)
#> evaluate 0.12 2018-10-09 [1] CRAN (R 3.5.0)
#> fs 1.2.6 2018-08-23 [1] CRAN (R 3.5.0)
#> glue 1.3.0 2018-07-17 [1] CRAN (R 3.5.0)
#> htmltools 0.3.6 2017-04-28 [1] CRAN (R 3.5.0)
#> knitr 1.20 2018-02-20 [1] CRAN (R 3.5.0)
#> magrittr 1.5 2014-11-22 [1] CRAN (R 3.5.0)
#> memoise 1.1.0 2017-04-21 [1] CRAN (R 3.5.0)
#> pkgbuild 1.0.2 2018-10-16 [1] CRAN (R 3.5.0)
#> pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.5.0)
#> prettyunits 1.0.2 2015-07-13 [1] CRAN (R 3.5.0)
#> processx 3.2.0 2018-08-16 [1] CRAN (R 3.5.0)
#> ps 1.2.1 2018-11-06 [1] CRAN (R 3.5.0)
#> R6 2.3.0 2018-10-04 [1] CRAN (R 3.5.0)
#> Rcpp 1.0.0 2018-11-07 [1] CRAN (R 3.5.0)
#> remotes 2.0.2 2018-10-30 [1] CRAN (R 3.5.0)
#> rlang 0.3.0.1 2018-10-25 [1] CRAN (R 3.5.0)
#> rmarkdown 1.10 2018-06-11 [1] CRAN (R 3.5.0)
#> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.5.0)
#> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.5.0)
#> stringi 1.2.4 2018-07-20 [1] CRAN (R 3.5.0)
#> stringr 1.3.1 2018-05-10 [1] CRAN (R 3.5.0)
#> testthat 2.0.0 2017-12-13 [1] CRAN (R 3.5.0)
#> usethis 1.4.0 2018-08-14 [1] CRAN (R 3.5.0)
#> withr 2.1.2 2018-03-15 [1] CRAN (R 3.5.0)
#> yaml 2.2.0 2018-07-25 [1] CRAN (R 3.5.0)
#>
#> [1] /Library/Frameworks/R.framework/Versions/3.5/Resources/library
The error bounds must be computed before setting up the plot limits, so we are sure the bounds are shown in the plot correctly.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.