stanjulia/stan.jl

Stan.jl illustrates the usage of the 'single method' packages, e.g. StanSample, StanOptimize, etc.

stan.jl's Issues

update_R_file (in stancode.jl) fails when handling large arrays

I'm a new user to Stan on julia, although I'm a long-time cmdstan user. I'm working on a problem where arrays of over a million elements need to be written to the .R file. I noticed that the stan() function never returned, seeming to get stuck just after opening the .R file but not writing to it. Upon investigation I found that update_R_file() puts each stan variable into a string before writing it, meaning that a large array generates a huge string. It's my guess that julia balks at this huge string. I modified update_R_file() to write arrays using writecsv() instead, and this solved the problem. My updated code is attached. I'm new to github, so pardon me if there is another way to upload proposed revisions. You'll note that the revised code is shorter as well.

Michael

function update_R_file{T<:Any}(file::String, dct::Dict{String, T}; replaceNaNs::Bool=true)
isfile(file) && rm(file)
strmout = open(file, "w")

str = ""
for entry in dct
str = """entry[1]"""" <- "
val = entry[2]
if length(val)==1 && length(size(val))==0
# Scalar
str = str"$(val)\n"
elseif length(val)==1 && length(size(val))==1
# Single element vector
str = str_"$(val[1])\n"
elseif length(val)>1 && length(size(val))==1
# Vector
str = str_"structure(c("

= Replace this code

  for i in 1:length(val)
    str = str*"$(val[i])"
    if i < length(val)
      str = str*", "
    end
  end

=## with this code
write(strmout, str)
str = ""
writecsv(strmout, val');

  str = str*"), .Dim=c($(length(val))))\n"
elseif length(val)>1 && length(size(val))>1
  # Array
  str = str*"structure(c("

= Replace this code

  for i in 1:length(val)
    str = str*"$(val[i])"
    if i < length(val)
      str = str*", "
    end
  end

=## with this code
write(strmout, str)
str = ""
writecsv(strmout, val[:]');

  dimstr = "c"*string(size(val))
  str = str*"), .Dim=$(dimstr))\n"
end
write(strmout, str)

end
close(strmout)
end

Stan.jl warning

From Eric Chameroy on Discourse:

Hi everyone,

When I load Stan.jl, I receive the following message:
WARNING: using Stan.CMDSTAN_HOME in module Main conflicts with an existing identifier.

I had created a .juliarc.jl file containing the statement:
CMDSTAN_HOME = “C:\Users\USERNAME\CmdStan” (I’m using Windows 10)
as stated in the Stan.jl documentation for setting up CmdStan.

What does this warning mean and is there a way to fix it? Considering it’s a warning and not an error, do I need to be concerned with it at all?

Using "save_warmup = 1" returns only one sample

When sampling a model with:

model = Stanmodel(Sample(save_warmup=1),...)
sim = Stan.stan(model,data,...)

Only the one sample is returned in the Mamba.Chains object sim.

However, the model_samples.csv file has the expected number of samples (num_warmup+num_samples) in it.

I should also note that the Stan executable returns the message:
Warning: non-fatal error reading adapation data.

update_R_file exports single element vector as scalar, causing error

Data vectors containing a single element get written to the R input data file like a scalar, according to https://github.com/goedman/Stan.jl/blob/master/src/stancode.jl#L182.
This causes a dimensionality error if I have just one data point:

std::exception::what: mismatch in number dimensions declared and found in context; processing stage=data initialization; variable name=y; dims declared=(1); dims found=()

Simpy removing the special treatment of single element vectors would solve this. Is there a reason for not doing this? Thanks!

CMDSTAN_HOME or JULIA_CMDSTAN_HOME issues.

Please open an issue here or send email to [email protected].

Can't remove tmp directory on Windows?

Attempting to delete the tmp directory in Pkg.test("Stan") returns wit a EPERM rejection.

Failed Process

I've been trying to determine the cause of an error message:
ErrorException("failed processes:\n Process(./odePlatelets sample num_samples=1000 num_warmup=1000 save_warmup=0 thin=1 adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=hmc engine=nuts max_depth=10 metric=diag_e stepsize=1.0 stepsize_jitter=1.0 random seed=-1 init=2 id=1 data file=odePlatelets_1.data.R output file=odePlatelets_samples_1.csv diagnostic_file=odePlatelets_diagnostics_1.csv refresh=100, ProcessExited(70)) [70]\

What does the 70 represent?

mambaThinning vs. CmdStan thinning

Hi @RalphAS,

Thanks for the patch. Are you using Stan.jl v2.0.0 or master (after Pkg.checkout("Stan")?

In the master version, the call to stanmodel() has an additional keyword argument mambaThinning which is set to 1 by default. This means by default thinning occurs in CmdStan, but optionally you can not use CmdStan and postpone thinning until later. One place where that might be useful is the CmdStan problem/issue that thinning and attempting to save warmups is not working correctly. See also http://goedman.github.io/Stan.jl/latest/VERSIONS.html#Version-2.0.0-1 .

I apologies for not having updated METADATA yet, but this is because I am at the tail end of a rather complicated move ( #46 ). I will spend some time on cleaning this up over the next 2 weeks.

Hope it is ok I respond in an issue, not directly on the PR.

Best,
Rob

Difficulty getting CMDSTAN_HOME to be recognized

I have tried defining CMDSTAN_HOME as a Julia variable in my ~/.juliarc.jl file and as an environment variable before I start jupyter-notebook. Neither of these seems to do the trick. I always end up in my IJulia session with errors like

sim1 = stan(ratstan);
LoadError: UndefVarError: CMDSTAN_HOME not defined
while loading In[15], in expression starting on line 1

 in stan at /home/bates/.julia/v0.4/Stan/src/stancode.jl:13

I notice that the checks for CMDSTAN_HOME and JULIA_SVG_BROWSER are part of the src/Stan.jl file but not wrapped in an __init__ function. It is probably better to assign these values as global const in an __init__ function, which will be called when the package is loaded. Also a more elegant approach than

try
    CMDSTAN_HOME = ENV["CMDSTAN_HOME"]
catch e
    ...
end

is

if haskey(ENV, "CMDSTAN_HOME")
   global const CMDSTAN_HOME = ENV["CMDSTAN_HOME"]
else
   error(...)
end

I could create a pull request with such changes if you wish. You could take a look at the RCall package sources to see some of this in action.

Tag Stan v5.0.2

@JuliaRegistrator register()

CmdStan does not install properly with Homebrew

I installed cmdstan using:

brew install cmdstan

I then added the path in .juliarc.jl

CMDSTAN_HOME = "/usr/local/Cellar/cmdstan/2.7.0/"

I believe this is the correct directory, it has bin/, include/, and share/ directories within it. I've confirmed that the variable CMDSTAN_HOME is set appropriately when I launch julia.

When I either try to reproduce the example given in the README or run Pkg.test() I get errors like:

ErrorException("failed process: Process(`make /Users/alex/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli`, ProcessExited(2)) [2]")

Pkg.test() also outputs this line:

CMDSTAN_HOME found, but CmdStan not installed properly.

Is there a way I can test that homebrew installed cmdstan correctly? Any other thoughts on what the problem could be? Thanks in advance. -- Alex

Slow writing of <model>_<i>.data.R

First of all, thanks for putting this up. It is greatly appreciated.

This might be more a question than an issue, but in any case. When using the stan.jl module on input data consisting of ~200K floating point values per thread, I observe a slow down due to the writing of the data into the <model>_<i>.data.R files. When I manually monitor the tmp/ directory I estimate that julia spends 1-2 minutes on writing each of these files (the files are approx 1.5 MB each and I use a 1 year old macbook pro). Is this expected behaviour? This seems really slow to me, but there might be something else going on?

Documentation update

Needs a complete rewrite to make it easy to find the right example.

No Homebrew on Ubuntu

I'm running julia 0.5 on Ubuntu 16.04. Installing CmdStan was no problem but I got this error with Stan.jl

julia> Pkg.add("Stan")
INFO: Cloning cache of Stan from https://github.com/goedman/Stan.jl.git
INFO: Installing Stan v1.0.2
INFO: Building Rmath
INFO: Building Cairo
INFO: Building Stan
==========================================================[ ERROR: Stan ]==========================================================

LoadError: ArgumentError: Module Homebrew not found in current path.
Run `Pkg.add("Homebrew")` to install the Homebrew package.
while loading /home/beaujean/.julia/v0.5/Stan/deps/build.jl, in expression starting on line 1

===================================================================================================================================

=========================================================[ BUILD ERRORS ]==========================================================

WARNING: Stan had build errors.

 - packages with build errors remain installed in /home/beaujean/.julia/v0.5
 - build the package(s) and all dependencies with `Pkg.build("Stan")`
 - build a single package by running its `deps/build.jl` script

===================================================================================================================================
INFO: Package database updated

Now I read the Readme that says Stan.jl is mostly tested on Mac but why would you require Homebrew for any platform? I guess it only makes sense to have on Mac OS.

Since your build.jl should do nothing if not on Mac OS, I guess you just have to avoid the first line

using BinDeps, Homebrew

that requires homebrew regardless of the operating system.

Test Fails

Since updating to Julia .4, I've encountered numerous problems. I suspect that the problems might be related to the failed package test, which gives the following error message:

INFO: Testing Stan
Running tests for Stan-j0.3-v0.2.1:

• test_utilities.jl *
  ERROR: LoadError: could not open file /Users/chrisfisher/.julia/v0.4/Cairo/src/../deps/deps.jl
  in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  [inlined code] from none:2
  in anonymous at no file:0
  in process_options at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in _start at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  while loading /Users/chrisfisher/.julia/v0.4/Cairo/src/Cairo.jl, in expression starting on line 7
  ERROR: LoadError: LoadError: Failed to precompile Cairo to /Users/chrisfisher/.julia/lib/v0.4/Cairo.ji
  in error at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in compilecache at loading.jl:383
  in require at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  [inlined code] from none:2
  in anonymous at no file:0
  in process_options at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in _start at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  while loading /Users/chrisfisher/.julia/v0.4/Compose/src/cairo_backends.jl, in expression starting on line 4
  while loading /Users/chrisfisher/.julia/v0.4/Compose/src/Compose.jl, in expression starting on line 141
  ERROR: LoadError: Failed to precompile Compose to /Users/chrisfisher/.julia/lib/v0.4/Compose.ji
  in error at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in compilecache at loading.jl:383
  in require at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  [inlined code] from none:2
  in anonymous at no file:0
  in process_options at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in _start at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  while loading /Users/chrisfisher/.julia/v0.4/Gadfly/src/Gadfly.jl, in expression starting on line 9
  ERROR: LoadError: LoadError: LoadError: LoadError: Failed to precompile Gadfly to /Users/chrisfisher/.julia/lib/v0.4/Gadfly.ji
  in error at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in compilecache at loading.jl:383
  in require at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in require at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in require at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  [inlined code] from /Users/chrisfisher/.julia/v0.4/Stan/test/runtests.jl:22
  in anonymous at no file:21
  in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in process_options at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in _start at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  while loading /Users/chrisfisher/.julia/v0.4/Mamba/src/Mamba.jl, in expression starting on line 6
  while loading /Users/chrisfisher/.julia/v0.4/Stan/src/Stan.jl, in expression starting on line 5
  while loading /Users/chrisfisher/.julia/v0.4/Stan/test/test_utilities.jl, in expression starting on line 1
  while loading /Users/chrisfisher/.julia/v0.4/Stan/test/runtests.jl, in expression starting on line 20
  LoadError: Stan had test errors
  while loading In[9], in expression starting on line 1

  in error at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  in test at pkg/entry.jl:753
  in anonymous at pkg/dir.jl:31
  in cd at file.jl:22
  in cd at pkg/dir.jl:31
  in test at pkg.jl:71

================================[ ERROR: Stan ]=================================

failed process: Process(/Applications/Julia-0.4.0.app/Contents/Resources/julia/bin/julia --check-bounds=yes --code-coverage=none --color=no /Users/chrisfisher/.julia/v0.4/Stan/test/runtests.jl, ProcessExited(1)) [1]

https://github.com/avehtari/PSIS

Hello:
I'm porting, mostly done, the matlab code of PSIS, PSIS-LOO which implement Pareto smoothed importance sampling (PSIS) and PSIS leave-one-out cross-validation. I wonder if you are interested in merge it here or maybe go and try to merge in Mamba.jl.

Regards.

On Windows passing in .juliarc.jl and/or environment variables seems to work differently

Not sure yet how to test and fix this.

`Pkg.test("Stan")` fails Julia-0.4.0-dev+2207 on 32-bit Linux

All of the cmdstan tests passed (e.g., make src/test/interface).

$ ./julia -E 'versioninfo();println(Pkg.installed("Stan"));CMDSTAN_HOME="/usr/local/src/cmdstan";Pkg.test("Stan")'
Julia Version 0.4.0-dev+2177
Commit a2ff1ea* (2014-12-17 01:05 UTC)
Platform Info:
  System: Linux (i686-redhat-linux)
  CPU: Genuine Intel(R) CPU           T2250  @ 1.73GHz
  WORD_SIZE: 32
  BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Banias)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.3
0.2.0
INFO: Testing Stan
Running tests for Stan-j0.3-v0.2.0:

  * test_utilities.jl *

[Lots & lots of ---------> ]WARNING: deprecated syntax "{}" 

Environment variable JULIA_SVG_BROWSER not found.
`echo 2`
`echo 3 4`
`echo 5`

1
2
3
4
5

hello
hello
hello
hello
hello
hello

  * test_cmdtype.jl *

WARNING: deprecated syntax "(ASCIIString=>Any)[a=>b, ...]" at /home/rick/.julia/v0.4/Stan/test/test_cmdtype.jl:23.
Use "Dict{ASCIIString,Any}(a=>b, ...)" instead.

WARNING: deprecated syntax "(ASCIIString=>Any)[a=>b, ...]" at /home/rick/.julia/v0.4/Stan/test/test_cmdtype.jl:24.
Use "Dict{ASCIIString,Any}(a=>b, ...)" instead.

WARNING: deprecated syntax "(ASCIIString=>Any)[a=>b, ...]" at /home/rick/.julia/v0.4/Stan/test/test_cmdtype.jl:25.
Use "Dict{ASCIIString,Any}(a=>b, ...)" instead.

WARNING: deprecated syntax "(ASCIIString=>Any)[a=>b, ...]" at /home/rick/.julia/v0.4/Stan/test/test_cmdtype.jl:26.
Use "Dict{ASCIIString,Any}(a=>b, ...)" instead.

File /home/rick/.julia/v0.4/Stan/test/tmp/bernoulli.stan will be updated.


  name =                    "bernoulli"
  nchains =                 4
  update =                   1000
  adapt =                    1000
  thin =                     1
  monitors =                ASCIIString[]
  model_file =              "bernoulli.stan"
  data_file =                ""
  output =                  Output()
    file =                    ""
    diagnostics_file =        ""
    refresh =                 100
Optimize(Stan.Lbfgs(0.001,1.0e-8,1.0e-8,1.0e-8,5), 2000, false)

  name =                    "bernoulli"
  nchains =                 4
  update =                   1000
  adapt =                    1000
  thin =                     1
  monitors =                ASCIIString[]
  model_file =              "bernoulli.stan"
  data_file =                ""
  output =                  Output()
    file =                    ""
    diagnostics_file =        ""
    refresh =                 100
  method =                  Optimize()
    algorithm =               Lbfgs()
      init_alpha =              0.001
      tol_obj =                 1.0e-8
      tol_grad =                1.0e-8
      tol_param =               1.0e-8
      history_size =            5
    iterations =              2000
    save_iterations =         false

Sample(1000, 1000, false, 1, Stan.Adapt(true,0.05,0.8,0.75,10.0,75,50,25), Stan.Hmc(Stan.Nuts(10),Stan.diag_e(),1.0,1.0))

  method =                  Sample()
    num_samples =             1000
    num_warmup =              1000
    save_warmup =             false
    thin =                    1
    algorithm =               HMC()
      engine =                  NUTS()
        max_depth =               10
      metric =                  Stan.diag_e
      stepsize =                1.0
      stepsize_jitter =         1.0
    adapt =                   Adapt()
      gamma =                   0.05
      delta =                   0.8
      kappa =                   0.75
      t0 =                      10.0
      init_buffer =             75
      term_buffer =             50
      window =                  25

Optimize(Stan.Bfgs(0.001,1.0e-9,1.0e-8,1.0e-8), 2000, true)

  method =                  Optimize()
    algorithm =               Bfgs()
      init_alpha =              0.001
      tol_obj =                 1.0e-9
      tol_grad =                1.0e-8
      tol_param =               1.0e-8
    iterations =              2000
    save_iterations =         true

Diagnose(Stan.Gradient(1.0e-6,1.0e-7))

  method =                  Diagnose()
    diagnostic =              Gradient()
      epsilon =                 1.0e-6
      error =                   1.0e-7

CMDSTAN_HOME found! Try to run bernoulli.

  * test_bernoulli.jl *
Moving to directory: /home/rick/.julia/v0.4/Stan/Examples/Bernoulli

File /home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli.stan will be updated.


Stanmodel that will be used:
  name =                    "bernoulli"
  nchains =                 4
  update =                   1200
  adapt =                    1000
  thin =                     2
  monitors =                ASCIIString[]
  model_file =              "bernoulli.stan"
  data_file =                ""
  output =                  Output()
    file =                    ""
    diagnostics_file =        ""
    refresh =                 100
  method =                  Sample()
    num_samples =             1200
    num_warmup =              1000
    save_warmup =             false
    thin =                    1
    algorithm =               HMC()
      engine =                  NUTS()
        max_depth =               10
      metric =                  Stan.diag_e
      stepsize =                1.0
      stepsize_jitter =         1.0
    adapt =                   Adapt()
      gamma =                   0.05
      delta =                   0.8
      kappa =                   0.75
      t0 =                      10.0
      init_buffer =             75
      term_buffer =             50
      window =                  25
Input observed data dictionary:
4-element Array{Dict{ASCIIString,Any},1}:
 Dict{ASCIIString,Any}("N"=>10,"y"=>[0,1,0,1,0,0,0,0,0,1])
 Dict{ASCIIString,Any}("N"=>10,"y"=>[0,1,0,0,0,0,1,0,0,1])
 Dict{ASCIIString,Any}("N"=>10,"y"=>[0,0,0,0,0,0,1,0,1,1])
 Dict{ASCIIString,Any}("N"=>10,"y"=>[0,0,0,1,0,0,0,1,0,1])


--- Translating Stan model to C++ code ---
bin/stanc /home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli.stan --o=/home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli.cpp --no_main
Model name=bernoulli_model
Input file=/home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli.stan
Output file=/home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli.cpp

--- Linking C++ model ---
g++ -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -I src -I stan/src -isystem stan/lib/eigen_3.2.0 -isystem stan/lib/boost_1.54.0 -Wall -pipe -DEIGEN_NO_DEBUG -Wno-unused-local-typedefs  -lpthread  -O3 -o /home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli src/cmdstan/main.cpp -include /home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli.cpp -Lbin -lstan

ErrorException("failed processes:\n  Process(`./bernoulli stan.sample num_samples=1200 num_warmup=1000 save_warmup=0 thin=1 stan.adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=stan.hmc engine=stan.nuts max_depth=10 metric=Stan.diag_e stepsize=1.0 stepsize_jitter=1.0 stan.random seed=-1 init=2 id=1 data file=bernoulli_1.data.R output file=bernoulli_samples_1.csv refresh=100`, ProcessExited(64)) [64]\n  Process(`./bernoulli stan.sample num_samples=1200 num_warmup=1000 save_warmup=0 thin=1 stan.adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=stan.hmc engine=stan.nuts max_depth=10 metric=Stan.diag_e stepsize=1.0 stepsize_jitter=1.0 stan.random seed=-1 init=2 id=2 data file=bernoulli_2.data.R output file=bernoulli_samples_2.csv refresh=100`, ProcessExited(64)) [64]\n  Process(`./bernoulli stan.sample num_samples=1200 num_warmup=1000 save_warmup=0 thin=1 stan.adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=stan.hmc engine=stan.nuts max_depth=10 metric=Stan.diag_e stepsize=1.0 stepsize_jitter=1.0 stan.random seed=-1 init=2 id=3 data file=bernoulli_3.data.R output file=bernoulli_samples_3.csv refresh=100`, ProcessExited(64)) [64]\n  Process(`./bernoulli stan.sample num_samples=1200 num_warmup=1000 save_warmup=0 thin=1 stan.adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=stan.hmc engine=stan.nuts max_depth=10 metric=Stan.diag_e stepsize=1.0 stepsize_jitter=1.0 stan.random seed=-1 init=2 id=4 data file=bernoulli_4.data.R output file=bernoulli_samples_4.csv refresh=100`, ProcessExited(64)) [64]")
CMDSTAN_HOME found, but CmdStan not installed properly.
LoadError("/home/rick/.julia/v0.4/Stan/test/test_bernoulli.jl",10,LoadError("/home/rick/.julia/v0.4/Stan/Examples/Bernoulli/bernoulli.jl",44,MethodError(size,(nothing,1))))
No simulation runs have been performed.


INFO: Stan tests passed
INFO: No packages to install, update or remove
nothing

Interestingly, with a few commandline param changes ...

$ pwd
/home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp
$ ./bernoulli sample num_samples=1200 num_warmup=1000 save_warmup=0 thin=1 adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=hmc engine=nuts max_depth=10 metric=diag_e stepsize=1.0 stepsize_jitter=1.0 random seed=-1 init=2 id=1 data file=bernoulli_1.data.R output file=bernoulli_samples_1.csv refresh=100
 method = sample (Default)
   sample
     num_samples = 1200
     num_warmup = 1000 (Default)
     save_warmup = 0 (Default)
     thin = 1 (Default)
     adapt
       engaged = 1 (Default)
       gamma = 0.050000000000000003 (Default)
       delta = 0.80000000000000004 (Default)
       kappa = 0.75 (Default)
       t0 = 10 (Default)
       init_buffer = 75 (Default)
       term_buffer = 50 (Default)
       window = 25 (Default)
     algorithm = hmc (Default)
       hmc
         engine = nuts (Default)
           nuts
             max_depth = 10 (Default)
         metric = diag_e (Default)
         stepsize = 1 (Default)
         stepsize_jitter = 1
 id = 1
 data
   file = bernoulli_1.data.R
 init = 2 (Default)
 random
   seed = 1566406245
 output
   file = bernoulli_samples_1.csv
   diagnostic_file =  (Default)
   refresh = 100 (Default)


Gradient evaluation took 0 seconds
1000 transitions using 10 leapfrog steps per transition would take 0 seconds.
Adjust your expectations accordingly!


Iteration:    1 / 2200 [  0%]  (Warmup)
Iteration:  100 / 2200 [  4%]  (Warmup)
Iteration:  200 / 2200 [  9%]  (Warmup)
Iteration:  300 / 2200 [ 13%]  (Warmup)
Iteration:  400 / 2200 [ 18%]  (Warmup)
Iteration:  500 / 2200 [ 22%]  (Warmup)
Iteration:  600 / 2200 [ 27%]  (Warmup)
Iteration:  700 / 2200 [ 31%]  (Warmup)
Iteration:  800 / 2200 [ 36%]  (Warmup)
Iteration:  900 / 2200 [ 40%]  (Warmup)
Iteration: 1000 / 2200 [ 45%]  (Warmup)
Iteration: 1001 / 2200 [ 45%]  (Sampling)
Iteration: 1100 / 2200 [ 50%]  (Sampling)
Iteration: 1200 / 2200 [ 54%]  (Sampling)
Iteration: 1300 / 2200 [ 59%]  (Sampling)
Iteration: 1400 / 2200 [ 63%]  (Sampling)
Iteration: 1500 / 2200 [ 68%]  (Sampling)
Iteration: 1600 / 2200 [ 72%]  (Sampling)
Iteration: 1700 / 2200 [ 77%]  (Sampling)
Iteration: 1800 / 2200 [ 81%]  (Sampling)
Iteration: 1900 / 2200 [ 86%]  (Sampling)
Iteration: 2000 / 2200 [ 90%]  (Sampling)
Iteration: 2100 / 2200 [ 95%]  (Sampling)
Iteration: 2200 / 2200 [100%]  (Sampling)

#  Elapsed Time: 0.02 seconds (Warm-up)
#                0.05 seconds (Sampling)
#                0.07 seconds (Total)

Excessive print of Stanmodel information on Mac

Hi,

This is more of an annoyance than a bug. I am using stan.jl with stan 2.20.0 in Atom. After I run Stanmodel, it prints that information of the summary everytime I use the repl for something.

Eg.:

bernoullistanmodel = "
data { 
  int<lower=0> N; 
  int<lower=0,upper=1> y[N];
} 
parameters {
  real<lower=0,upper=1> theta;
} 
model {
  theta ~ beta(1,1);
    y ~ bernoulli(theta);
}
"

stanmodel = Stanmodel(name="bernoulli", model=bernoullistanmodel);

And it will print a bunch of information about the model. Now do something else, like 2+2and it will print the info all over again.

This is a Mac OS related bug, it does not happens on Windows. I am using Mojave.

Thanks in advance

read_stan_files.jl does not skip comment symbol "#"

Description

I am now using optimize method to do stan. But function read_stanfit in read_stan_files.jl does not parse the output csv properly.

Code

Code File: unfair_dice_newton.jl

using Printf
using Stan, Mamba

model_script = "
data {
    int<lower=0> N;
    real y[N];
}

parameters{
    real<lower=0, upper=1> p;
}

model{
    target += log(1); // assume flat prior

    for(i in 1:N) {
        if(y[i] == 1 || y[i] == 6) {
            target += log(p);
        } else {
            target += log((1.0 - 2.0 * p) / 4.0);
        }
    }
}
"
##### Making data begin #####
N = 1000
p = 0.2
q = (1.0 - 2p) / 4.0

x = rand(N)

p_hist = [p, q, q, q, q, p]
for i = 2:length(p_hist)
    p_hist[i] += p_hist[i-1]
end
p_hist[end] = 1.0

function rnd2pt(x)
    for i = 1:length(p_hist)
        if x < p_hist[i]
            return i
        end
    end
end

y = rnd2pt.(x)
##### Making data end #####

data = Dict(
    "N" => N,
    "y" => y
)

@printf("Now we are going to build stan model...\n")
stanmodel = Stanmodel(
    Stan.Optimize();
    name="Dice",
    nchains=4,
    model=model_script,
    useMamba=true
)

stanmodel |> display

rc, sim = stan(
    stanmodel,
    [data],
    "tmp"
    ;
    CmdStanDir=ENV["CMDSTAN_HOME"],
    init=Dict(
        "p" => 0.1
    )
)

if rc == 0
    println(sim[1]["p"])
else
    println("There are errors!!")
end

Output

ERROR: LoadError: ArgumentError: cannot parse "#" as Float64
Stacktrace:
 [1] _parse_failure(::Type, ::SubString{String}, ::Int64, ::Int64) at ./parse.jl:357 (repeats 2 times)
 [2] tryparse_internal at ./parse.jl:353 [inlined]
 [3] parse at ./parse.jl:363 [inlined]
 [4] _broadcast_getindex_evalf at ./broadcast.jl:574 [inlined]
 [5] _broadcast_getindex at ./broadcast.jl:557 [inlined]
 [6] getindex at ./broadcast.jl:507 [inlined]
 [7] macro expansion at ./broadcast.jl:838 [inlined]
 [8] macro expansion at ./simdloop.jl:73 [inlined]
 [9] copyto! at ./broadcast.jl:837 [inlined]
 [10] copyto! at ./broadcast.jl:792 [inlined]
 [11] copy at ./broadcast.jl:768 [inlined]
 [12] materialize at ./broadcast.jl:748 [inlined]
 [13] read_stanfit(::Stanmodel) at /home/tienyiah/.julia/packages/Stan/krRNa/src/utilities/read_stan_files.jl:109
 [14] #stan#4(::Dict{String,Float64}, ::Bool, ::Bool, ::String, ::Function, ::Stanmodel, ::Array{Dict{String,Any},1}, ::String) at /home/tienyiah/.julia/packages/Stan/krRNa/src/main/stancode.jl:205
 [15] (::getfield(Stan, Symbol("#kw##stan")))(::NamedTuple{(:CmdStanDir, :init),Tuple{String,Dict{String,Float64}}}, ::typeof(stan), ::Stanmodel, ::Array{Dict{String,Any},1}, ::String) at ./none:0
 [16] top-level scope at none:0
 [17] include at ./boot.jl:317 [inlined]
 [18] include_relative(::Module, ::String) at ./loading.jl:1041
 [19] include(::Module, ::String) at ./sysimg.jl:29
 [20] exec_options(::Base.JLOptions) at ./client.jl:229
 [21] _start() at ./client.jl:421
in expression starting at /home/tienyiah/projects/stan_practice/unfair_dice_newton.jl:66

Bug?

After look into the code, I figured out that the computational part works fine. But when reading the output csv data file tmp/Dice_samples_1.csv, it tries to read the last few lines which are line comments:

File: tmp/Dice_samples_1.csv

# stan_version_major = 2
# stan_version_minor = 18
# stan_version_patch = 0
# model = Dice_model
...
...
   refresh = 100 (Default)
lp__,accept_stat__,stepsize__,treedepth__,n_leapfrog__,divergent__,energy__,p
# Adaptation terminated
# Step size = 0.981302
# Diagonal elements of inverse mass matrix:
# 0.00280974
-1788.04,0.579651,0.981302,1,3,0,1789.78,0.195009
-1788.38,0.748201,0.981302,2,3,0,1789.73,0.185326
...
...
-1787.98,0.635984,0.981302,2,3,0,1790.76,0.19273
-1787.98,0.877663,0.981302,1,3,0,1788.42,0.19273
-1788.3,0.922961,0.981302,2,3,0,1788.49,0.19844
-1787.98,1,0.981302,1,3,0,1788.24,0.192878
# 
#  Elapsed Time: 0.25 seconds (Warm-up)
#                0.27 seconds (Sampling)
#                0.52 seconds (Total)
# 

I successfully avoid the problem by uncommenting 99 line of the file utilities/read_stan_files.jl. But I am not sure if this is the correct way to fix it.

 97           while true
 98             j += 1
 99             #skipchars(isspace, instream, linecomment='#')  # This line should not be commented
100             line = Unicode.normalize(readline(instream), newline2lf=true)
101             flds = Float64[]
102             if eof(instream) && length(line) < 2
103               #println("EOF detected")
104               close(instream)
105               #return(tdict)
106               break
107             else
108               #println(split(strip(line)))
109               flds = parse.(Float64, (split(strip(line), ",")))  # This line throws the error

Julia info

I am currently using Stan v4.0.0. Below is my julia info

Julia Version 1.0.1
Commit 0d713926f8 (2018-09-29 19:05 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, haswell)

Future of Stan.jl

Hi,

In issue #39 I started a discussion on the future of Stan.jl by asking several questions. Hiding the discussion inside another issue is not right, so I'm moving it to its own issue.

Before tagging Stan.jl v"1.1.0" I have 3 items on my list of things to do:

1. Julia v"0.6" (also depends on Mamba and Gadfly)
2. Running Stan models from anywhere (I'm not sure why occasionally I see problems with that)
3. Fixing the initialization issues (e.g. items #34, #38 and now #39)

With respect to 3 I'm leaning towards adding a new (multiple dispatch or keyword?) argument to stan() to make this work similar to data, e.g. sim1 = stan(stanmodel, bernoullidata, bernoulliinit) in the BernoulliInitTheta example.

Other questions (and also based on Michael's response in #39 for item 5):

4. Has anyone used item #36 (PSIS/LOO)?
5. Separating out running CmdStan and creating Mamba chains
6. Using DataTables to store and query the sampling results
7. Switch/add an option for plots using Plots.jl
8. Tracking of R functionality

I asked folks for feedback on these and other topics such as "do folks like the coupling with Mamba diagnostics and Mamba graphics?" and "how important is access to Stan functions on C++ level?".

Over the last year I have certainly seen more R focus in the Stan team, how important is that? Hence item 8 and possibly item 4.

After 2 years is it time to do major surgery on Stan.jl? I'm also pretty busy right now, so no promises, even if I could tackle some of the larger topics.

Regards,
Rob

confusing docstrings

The docstring for stan() says that the data and ProjDir arguments are required, but in the source they're defined as optional positional arguments. Also, the way they're described is confusing since you don't call the method with the argument name (while you do for the keyword arguments, which are described similarly).

I think it would help a lot to have example syntax for all of them. If I get time I'll open a PR but wanted to point this out here since it came up on slack.

Long lists of generated parameters don't work in DataFrames

Sebastian reported this on the Stan mailing list as slow. I've seen stack overflows in R and index errors in Julia.

Does stansummary still exist?

The Julia function stan_summary tries to call an executable $(CmdStanDir)/bin/stansummary but that executable was not built in my copy of cmdstan-2.6.0. Am I missing something?

Error tagging new release

The URL of this package does not match that stored in METADATA.jl.
cc: @goedman

could not build on Linux

julia> isfile(joinpath(ENV["CMDSTAN_HOME"], "bin/stanc"))
true

julia> VERSION
v"0.6.0-pre.beta.460"

julia> Pkg.build("Stan")
INFO: Building Rmath
INFO: Building Cairo
INFO: Building Stan
============================================================================[ ERROR: Stan ]============================================================================

LoadError: ArgumentError: Module Homebrew not found in current path.
Run `Pkg.add("Homebrew")` to install the Homebrew package.
while loading /home/tamas/.julia/v0.6/Stan/deps/build.jl, in expression starting on line 1

=======================================================================================================================================================================

===========================================================================[ BUILD ERRORS ]============================================================================

WARNING: Stan had build errors.

 - packages with build errors remain installed in /home/tamas/.julia/v0.6
 - build the package(s) and all dependencies with `Pkg.build("Stan")`
 - build a single package by running its `deps/build.jl` script

=======================================================================================================================================================================

Same happens on v0.5. Homebrew.jl does not seem to support Linux. Can Stan.jl be used on Linux?

Tag Stan v5.0.1

@JuliaRegistrator register()

read_stanfit_samples gives InexactError when step neq 1 in Sample

Hi Rob, I believe I've found a bug in read_stanfit_samples.

Minimal example:

const simplecode = "
data {real sigma;}
parameters {real y;}
model {y ~ normal(0,sigma);}
"
stanmodel = Stanmodel(Sample(thin=5), name="simple", model=simplecode);
sim = stan(stanmodel, [Dict("sigma" => 1.)], CmdStanDir=CMDSTAN_HOME);

gives

InexactError()

 in peekchar(::IOStream) at ./iostream.jl:269
 in #skipchars#287(::Char, ::Function, ::IOStream, ::Base.UTF8proc.#isspace) at ./iostream.jl:286
 in (::Base.#kw##skipchars)(::Array{Any,1}, ::Base.#skipchars, ::IOStream, ::Function) at ./<missing>:0
 in read_stanfit_samples(::Stan.Stanmodel, ::Bool, ::Bool) at /Users/jonalm/.julia/v0.5/Stan/src/stancode.jl:382
 in #stan#16(::Bool, ::Bool, ::String, ::Function, ::Stan.Stanmodel, ::Array{Dict{String,Float64},1}, ::String) at /Users/jonalm/.julia/v0.5/Stan/src/stancode.jl:147
 in (::Stan.#kw##stan)(::Array{Any,1}, ::Stan.#stan, ::Stan.Stanmodel, ::Array{Dict{String,Float64},1}, ::String) at ./<missing>:0 (repeats 2 times)

I've tried to understand the read_stanfit_samples function, but I get lost after the sample file is opened on line 362 in stancode.jl.

Both Stan and Optim export Newton

I was comparing the results of a Bayesian SLOPE regression model with one that uses Optim for minimizing the loss function in SLOPE but ran into problems since Newton is exported by both packages. Not sure what is the best way to avoid these things but reporting it anyway since this type of comparison may not be super-uncommon... ;)

WARNING: both Optim and Stan export "Newton"; uses of it in module Main must be qualified

walk-through error

if rc == 0
# the following line is not needed because it is already defined above. It also won't work, because stan 
# returns a tuple and we need to bind the second value
# sim1 = stan(stanmodel, [bernoullidata], ProjDir, CmdStanDir=CMDSTAN_HOME)
  println("Subset Sampler Output")
  sim = sim1[1:1000, ["lp__", "theta", "accept_stat__"], :]
  describe(sim)
end

algorithm = "Fixed_param"

I'm trying to use Stan in generative model to simulate stochastic processes and I get a Return code -5
When I check the run.log file I find the following error:

Must use algorithm=fixed_param for model that has no parameters
If I try to add the fixed param
stancode.jl 76 got unsupported keyword arguments "algorithm"

Can the algorithm keyword option be added to the Julia code? This option is available to PyStan and RStan users.

I would like to do it myself, but I'm just learning Julia and don't feel competent enough to give it a go.
Thanks.

Cannot find CMDSTAN_HOME

Hi Rob-

I encountered a strange situation in which CMDSTAN_HOME cannot be found on my system. I am using Ubuntu 16.04 and Julia .7. (@info Stan does not provide the version of Stan). First, I tried adding the path to my bashcr file to no avail. Next, I tried to hard code it into Stan.jl also to no avail. I also tried using the set function, but it did not work either.

I believe it is pointing to the correct location:

dfish@dfish-MS-7850:~/cmdstan-2.17.1$ ls
examples     LICENSE  makefile  README.md           src   test-all.sh
Jenkinsfile  make     Projects  runCmdStanTests.py  stan
dfish@dfish-MS-7850:~/cmdstan-2.17.1$ pwd
/home/dfish/cmdstan-2.17.1

Error:

julia> using Pkg

julia> using Stan
[ Info: Recompiling stale cache file /home/dfish/.julia/compiled/v0.7/Stan/zPI1R.ji for Stan [682df890-35be-576f-97d0-3d8c8b33a550]
WARNING: eval from module Base to Stan:    
Expr(:->, :x, Expr(:block, #= Symbol("/home/dfish/.julia/packages/Documenter/sN5Qn/src/DocChecks.jl"):650 =#, :y = :have_color, #= Symbol("/home/dfish/.julia/packages/Documenter/sN5Qn/src/DocChecks.jl"):650 =#, Expr(:global, :have_color = :x), #= Symbol("/home/dfish/.julia/packages/Documenter/sN5Qn/src/DocChecks.jl"):650 =#, :y))
  ** incremental compilation may be broken for this module **

┌ Warning: `warn()` is deprecated, use `@warn` instead.
│   caller = #warn#781(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String) at deprecated.jl:1032
└ @ Base ./deprecated.jl:1032
WARNING: Environment variable CMDSTAN_HOME not found. Use set_cmdstan_home!.

julia> CMDSTAN_HOME
""

julia> set_cmdstan_home!("/home/dfish/cmdstan-2.17.1")
"/home/dfish/cmdstan-2.17.1"

julia> CMDSTAN_HOME
"/home/dfish/cmdstan-2.17.1"

julia> using Stan

julia> Pkg.test("Stan")
   Testing Stan
    Status `/tmp/tmppcKWvH/Manifest.toml`
  [9e28174c] BinDeps v0.8.9
  [34da2185] Compat v1.0.1
  [ffbed154] DocStringExtensions v0.4.5
  [e30172f5] Documenter v0.19.3
  [d9be37ee] Homebrew v0.6.4
  [682c06a0] JSON v0.19.0
  [682df890] Stan v3.0.1
  [30578b45] URIParser v0.4.0
  [2a0f44e3] Base64  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Base64`]
  [ade2ca70] Dates  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Dates`]
  [8bb1440f] DelimitedFiles  [`~/julia.7/bin/../share/julia/stdlib/v0.7/DelimitedFiles`]
  [8ba89e20] Distributed  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Distributed`]
  [b77e0a4c] InteractiveUtils  [`~/julia.7/bin/../share/julia/stdlib/v0.7/InteractiveUtils`]
  [76f85450] LibGit2  [`~/julia.7/bin/../share/julia/stdlib/v0.7/LibGit2`]
  [8f399da3] Libdl  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Libdl`]
  [37e2e46d] LinearAlgebra  [`~/julia.7/bin/../share/julia/stdlib/v0.7/LinearAlgebra`]
  [56ddb016] Logging  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Logging`]
  [d6f4376e] Markdown  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Markdown`]
  [a63ad114] Mmap  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Mmap`]
  [44cfe95a] Pkg  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Pkg`]
  [de0858da] Printf  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Printf`]
  [3fa0cd96] REPL  [`~/julia.7/bin/../share/julia/stdlib/v0.7/REPL`]
  [9a3f8284] Random  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Random`]
  [ea8e919c] SHA  [`~/julia.7/bin/../share/julia/stdlib/v0.7/SHA`]
  [9e88b42a] Serialization  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Serialization`]
  [1a1011a3] SharedArrays  [`~/julia.7/bin/../share/julia/stdlib/v0.7/SharedArrays`]
  [6462fe0b] Sockets  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Sockets`]
  [2f01184e] SparseArrays  [`~/julia.7/bin/../share/julia/stdlib/v0.7/SparseArrays`]
  [10745b16] Statistics  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Statistics`]
  [8dfed614] Test  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Test`]
  [cf7118a7] UUIDs  [`~/julia.7/bin/../share/julia/stdlib/v0.7/UUIDs`]
  [4ec0a83e] Unicode  [`~/julia.7/bin/../share/julia/stdlib/v0.7/Unicode`]
┌ Warning: `warn()` is deprecated, use `@warn` instead.
│   caller = #warn#781(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String) at deprecated.jl:1032
└ @ Base ./deprecated.jl:1032
WARNING: Environment variable CMDSTAN_HOME not found. Use set_cmdstan_home!.
Running tests for Stan-j0.7-v3.0.0:


CMDSTAN_HOME not set or found.
Skipping all tests that depend on CmdStan!



   Testing Stan tests passed 

julia> CMDSTAN_HOME
"/home/dfish/cmdstan-2.17.1"

Tag Stan 5.2.1

@JuliaRegistrator register()

ADVI variational inference support?

Hi,

thanks for Stan.jl, it is a great package!

As of version 2.7.0 Stan supports variational inference as well. At the moment, is there any way to use the ADVI variational method from Stan.jl?

Thanks,
Sebastian

Stan 5.0.2 precompile warnings on Julia 1.3.

I'm not sure if this is a problem, but when I did a package update this morning I saw that Stan was updated to 5.0.2. Then a precompile resulted in the following warnings. Just thought I'd mention it.

Michael

(v1.3) pkg> up
Updating registry at ~/.julia/registries/General
Updating git-repo https://github.com/JuliaRegistries/General.git
Resolving package versions...
Installed OpenBLAS_jll ───── v0.3.7+1
Installed CSTParser ──────── v2.0.0
Installed Arpack ─────────── v0.4.0
Installed StatsFuns ──────── v0.9.2
Installed JuliaInterpreter ─ v0.7.5
Installed Stan ───────────── v5.0.2
Installed Arpack_jll ─────── v3.5.0+2
Installed Optim ──────────── v0.19.6
Updating ~/.julia/environments/v1.3/Project.toml
[682df890] ↑ Stan v5.0.1 ⇒ v5.0.2
Updating ~/.julia/environments/v1.3/Manifest.toml
[7d9fca2a] ↑ Arpack v0.3.1 ⇒ v0.4.0
[68821587] + Arpack_jll v3.5.0+2
[00ebfdb7] ↑ CSTParser v1.1.0 ⇒ v2.0.0
[aa1ae85d] ↑ JuliaInterpreter v0.7.4 ⇒ v0.7.5
[4536629a] + OpenBLAS_jll v0.3.7+1
[429524aa] ↑ Optim v0.19.5 ⇒ v0.19.6
[682df890] ↑ Stan v5.0.1 ⇒ v5.0.2
[4c63d2b9] ↑ StatsFuns v0.8.0 ⇒ v0.9.2

(v1.3) pkg> precompile
Precompiling project...
Precompiling Stan
[ Info: Precompiling Stan [682df890-35be-576f-97d0-3d8c8b33a550]
WARNING: Method definition findall(Union{Base.Regex, AbstractString}, AbstractString) in module Base at regex.jl:345 overwritten in module StanBase at /Users/prange/.julia/packages/StanBase/uoK8W/src/utils/findall.jl:20.
** incremental compilation may be fatally broken for this module **

WARNING: Method definition #findall(Any, typeof(Base.findall), Union{Base.Regex, AbstractString}, AbstractString) in module Base overwritten in module StanBase.
** incremental compilation may be fatally broken for this module **

WARNING: Method definition findall(Union{Base.Regex, AbstractString}, AbstractString) in module Base at regex.jl:345 overwritten in module StanBase at /Users/prange/.julia/packages/StanBase/uoK8W/src/utils/findall.jl:20.
** incremental compilation may be fatally broken for this module **

WARNING: Method definition #findall(Any, typeof(Base.findall), Union{Base.Regex, AbstractString}, AbstractString) in module Base overwritten in module StanBase.
** incremental compilation may be fatally broken for this module **

WARNING: Method definition findall(Union{Base.Regex, AbstractString}, AbstractString) in module Base at regex.jl:345 overwritten in module StanBase at /Users/prange/.julia/packages/StanBase/uoK8W/src/utils/findall.jl:20.
** incremental compilation may be fatally broken for this module **

WARNING: Method definition #findall(Any, typeof(Base.findall), Union{Base.Regex, AbstractString}, AbstractString) in module Base overwritten in module StanBase.
** incremental compilation may be fatally broken for this module **

WARNING: Method definition findall(Union{Base.Regex, AbstractString}, AbstractString) in module Base at regex.jl:345 overwritten in module StanBase at /Users/prange/.julia/packages/StanBase/uoK8W/src/utils/findall.jl:20.
** incremental compilation may be fatally broken for this module **

WARNING: Method definition #findall(Any, typeof(Base.findall), Union{Base.Regex, AbstractString}, AbstractString) in module Base overwritten in module StanBase.
** incremental compilation may be fatally broken for this module **

WARNING: Method definition findall(Union{Base.Regex, AbstractString}, AbstractString) in module Base at regex.jl:345 overwritten in module StanBase at /Users/prange/.julia/packages/StanBase/uoK8W/src/utils/findall.jl:20.
** incremental compilation may be fatally broken for this module **

WARNING: Method definition #findall(Any, typeof(Base.findall), Union{Base.Regex, AbstractString}, AbstractString) in module Base overwritten in module StanBase.
** incremental compilation may be fatally broken for this module **

WARNING: Method definition findall(Union{Base.Regex, AbstractString}, AbstractString) in module Base at regex.jl:345 overwritten in module StanBase at /Users/prange/.julia/packages/StanBase/uoK8W/src/utils/findall.jl:20.
** incremental compilation may be fatally broken for this module **

WARNING: Method definition #findall(Any, typeof(Base.findall), Union{Base.Regex, AbstractString}, AbstractString) in module Base overwritten in module StanBase.
** incremental compilation may be fatally broken for this module **

(v1.3) pkg>

A few requests for features

My apologies in advance if these features currently exist. I was wondering if it would be possible to add the ability to modify default settings, such as number of samples per chain and warm up samples. Currently, my workaround is to modify line 58 in sampletype.jl and change the size of the output (e.g. sim1[1:N] where N is the number of samples per chain)

Also, is there a simple method of allocating the chains to each processor? This might be useful for complex models.

Thank you in advance

Tag Stan.jl v5.2.0

@JuliaRegistrator register()

Tag Stan v5.1.0

@JuliaRegistrator register()

Install ok but had to manually install cmdstan and define CMDSTAN_HOME

I can install the Julia Stan package without problems now but when I try the Bernoulli example I get:

julia> sim1 = stan(stanmodel, bernoullidata, ProjDir, diagnostics=false, CmdStanDir=CMDSTAN_HOME);
chdir /Users/feldt/usr/local/cmdstan-2.6.0: no such file or directory (ENOENT)

and there is indeed no such directory. To solve this I downloaded latest cmdstan from here:

https://github.com/stan-dev/cmdstan/releases

unpacked it, ran

make build

and then in julia set the CMDSTAN_HOME variable to where I had unpacked cmdstan. After this I could successfully run the bernoulli.jl example from Stan.jl.

Maybe updating the README to make these steps clear? Or did I miss something?

[PkgEval] Stan may have a testing issue on Julia 0.3 (2014-08-16)

PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.3) and the nightly build of the unstable version (0.4). The results of this script are used to generate a package listing enhanced with testing results.

On Julia 0.3

• On 2014-08-15 the testing status was Tests pass.
• On 2014-08-16 the testing status changed to Package doesn't load.

Tests pass. means that PackageEvaluator found the tests for your package, executed them, and they all passed.

Package doesn't load. means that PackageEvaluator did not find tests for your package. Additionally, trying to load your package with using failed.

This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.

Test log:

>>> 'Pkg.add("Stan")' log
INFO: Cloning cache of Stan from git://github.com/goedman/Stan.jl.git
INFO: Installing Stan v0.0.2
INFO: Package database updated
INFO: METADATA is out-of-date a you may not have the latest version of Stan
INFO: Use `Pkg.update()` to get the latest versions of your packages

>>> 'using Stan' log
ERROR: DataFrames not found
 in require at loading.jl:47
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:51
 in include at ./boot.jl:245
 in include_from_node1 at loading.jl:128
 in process_options at ./client.jl:285
 in _start at ./client.jl:354
 in _start_3B_1699 at /home/idunning/julia03/usr/bin/../lib/julia/sys.so
while loading /home/idunning/pkgtest/.julia/v0.3/Stan/src/Stan.jl, in expression starting on line 4
while loading /home/idunning/pkgtest/.julia/v0.3/Stan/testusing.jl, in expression starting on line 1


>>> test log
no tests to run
>>> end of log

Tag Stan v5.0.0

@JuliaRegistrator register()

Walk through example doesn't work

When running the example, stan complains

sim1 = Stan.stan(m, data)
...
Exception: variable does not exist; processing stage=data initialization; variable name=N; base type=int
Diagnostic information: 
Dynamic exception type: std::runtime_error
std::exception::what: variable does not exist; processing stage=data initialization; variable name=N; base type=int

ErrorException("failed processes:\n  Process(`./lin.reg sample num_samples=1000 num_warmup=1000 save_warmup=0 thin=1 adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=hmc engine=nuts max_depth=10 metric=diag_e stepsize=1.0 stepsize_jitter=1.0 random seed=-1 init=2 output file=lin.reg_samples_1.csv refresh=100`, ProcessExited(255)) [255]\n  Process(`./lin.reg sample num_samples=1000 num_warmup=1000 save_warmup=0 thin=1 adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=hmc engine=nuts max_depth=10 metric=diag_e stepsize=1.0 stepsize_jitter=1.0 random seed=-1 init=2 output file=lin.reg_samples_2.csv refresh=100`, ProcessExited(255)) [255]\n  Process(`./lin.reg sample num_samples=1000 num_warmup=1000 save_warmup=0 thin=1 adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=hmc engine=nuts max_depth=10 metric=diag_e stepsize=1.0 stepsize_jitter=1.0 random seed=-1 init=2 output file=lin.reg_samples_3.csv refresh=100`, ProcessExited(255)) [255]\n  Process(`./lin.reg sample num_samples=1000 num_warmup=1000 save_warmup=0 thin=1 adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=hmc engine=nuts max_depth=10 metric=diag_e stepsize=1.0 stepsize_jitter=1.0 random seed=-1 init=2 output file=lin.reg_samples_4.csv refresh=100`, ProcessExited(255)) [255]")

I hit this problem with my own code and the same problem. It is due to the fact that the data are just one dictionary and not a vector of dictionaries. It works if I do Stan.stan(m, [data,data, data,data]) for the default 4 chains. That shouldn't be necessary and there is code in src/stancode.jlto deal with it but obviously that fails. There is nodata file=XXX.R` in the call to cmdstan. I know I updated the readme and it worked then but it doesn't anymore. Would be good to check this behavior in the tests. The bernoulli example now has different data input for every chain.

TypeError for user defined starting points

I am trying to initialize the starting values for my model. As an example, I tried to run BernoulliInitTheta. However, I received the following error:

TypeError: typeassert: expected Number, got Array{Dict{String,Any},1}
in (::Core.#kw#Type)(::Array{Any,1}, ::Type{Stan.Init}) at :0
in include_string(::String, ::String) at loading.jl:441
in include_string(::String, ::String) at sys.dylib:?
in eval(::Module, ::Any) at boot.jl:234
in eval(::Module, ::Any) at sys.dylib:?
in (::Atom.##65#68)() at eval.jl:40
in withpath(::Atom.##65#68, ::Void) at utils.jl:30
in withpath(::Function, ::Void) at eval.jl:46
in macro expansion at eval.jl:109 [inlined]
in (::Atom.##64#67{Dict{String,Any}})() at task.jl:60

Any help would be greatly appreciated.

plot example does not work

In the README,

p = plot(sim, [:trace, :mean, :density, :autocor], legend=true);
draw(p, ncol=4, filename="summaryplot", fmt=:svg)

but the second line gives me an error:

julia> draw(p, ncol=4, filename="summaryplot", fmt=:svg)
ERROR: MethodError: `isless` has no method matching isless(::Float64, ::Void)
Closest candidates are:
  isless(::Float64, !Matched::Float64)
  isless(::AbstractFloat, !Matched::AbstractFloat)
  isless(::Real, !Matched::AbstractFloat)
  ...
 in max at operators.jl:57
 in apply_coordinate at /home/tamas/.julia/v0.4/Gadfly/src/coord.jl:232
 in render_prepared at /home/tamas/.julia/v0.4/Gadfly/src/Gadfly.jl:776
 in render at /home/tamas/.julia/v0.4/Gadfly/src/Gadfly.jl:722
 in draw at /home/tamas/.julia/v0.4/Mamba/src/output/plot.jl:45

Using 4.1, but get the same with latest nightly.
Workaround: dropping the [:trace, :mean, :density, :autocor] argument.

Feature Enhancement: auto save posterior summaries

Hi Rob-
Perhaps there is already a solution, but I could not figure out how to save either the Stan or Mamba posterior summaries. In either case, I was thinking it might be convenient to automatically save that along with the data and posterior sample files.

Problem setting initial values

I can't find a way to set initial values for my sampler.

Stanmodel.jl Stanmodel type has init which I believe should act as the container for initial values

type Stanmodel
  name::String
  nchains::Int
  ...
  init::Init
  output::Output
  tmpdir::String
end

which is defined to be of type Init

type Init
  init::Int64
end
Init(;init::Number=2) = Init(init)

So it only accepts one integer.

In pystan one can give a dictionary in following manner:

init = {"theta1": 0.5, "theta2": 0.9}

I tried giving a dictionary, self-made Init type with the attributes corresponding to the parameter values I'd like to initialize but could not work around it.

[PkgEval] Stan may have a testing issue on Julia 0.4 (2014-08-16)

PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.3) and the nightly build of the unstable version (0.4). The results of this script are used to generate a package listing enhanced with testing results.

On Julia 0.4

• On 2014-08-15 the testing status was Tests pass.
• On 2014-08-16 the testing status changed to Package doesn't load.

Tests pass. means that PackageEvaluator found the tests for your package, executed them, and they all passed.

Package doesn't load. means that PackageEvaluator did not find tests for your package. Additionally, trying to load your package with using failed.

This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.

Test log:

>>> 'Pkg.add("Stan")' log
INFO: Installing Stan v0.0.2
INFO: Package database updated
INFO: METADATA is out-of-date a you may not have the latest version of Stan
INFO: Use `Pkg.update()` to get the latest versions of your packages

>>> 'using Stan' log

ERROR: DataFrames not found
 in require at loading.jl:47
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:51
 in include at ./boot.jl:245
 in include_from_node1 at loading.jl:128
 in process_options at ./client.jl:285
 in _start at ./client.jl:354
 in _start_3B_3500 at /home/idunning/julia04/usr/bin/../lib/julia/sys.so
while loading /home/idunning/pkgtest/.julia/v0.4/Stan/src/Stan.jl, in expression starting on line 4
while loading /home/idunning/pkgtest/.julia/v0.4/Stan/testusing.jl, in expression starting on line 1

>>> test log
no tests to run
>>> end of log

change names(sid1) to fieldnames(sid1) in README

Just tried the example in the README and things work but I get this warning:

julia> names(sim1) |> display
WARNING: names(v) is deprecated, use fieldnames(v) instead.

so maybe update the README?

Fix link in readme.md

Most of the links in the readme.md file do not work.
It would be nice if this could be fixed.

Possible problem with initializing chain

Hi Rob-
I think there is an issue initializing chains. When I print theta in your simple example, it prints the correct value. However, in my hierarchical example, print(omega); should yield 3, but produces a random value. I could not find an error with my code unfortunately.

Best regards,

Chris