Giter Site home page Giter Site logo

xrjulia's Introduction

XRJulia - An Interface from R to Julia

This package provides an interface from R to Julia, based on the XR structure, as implemented in the XR package, in this repository.

The interface is designed as a basis for computations in R that use functions, objects and classes in Julia. In particular, the design caters to programmers developing application packages. The XR structure encourages definition of proxy functions and classes in \R{}, which users of the package can treat essentially as they would in R, without special programming imposed by the interface.

The interface structure is described in the book Extending R (John M. Chambers, 2016, Chapman & Hall). A pdf version of the XRJulia chapter from the book is included with the documentation of this package. To open the pdf file from R:

RShowDoc("Chapter_XRJulia", package = "XRJulia")

Version 0.76 of the package has been updated to deal with back-incompatible changes in Julia (up to version 0.6).

xrjulia's People

Contributors

johnmchambers avatar vh-d avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

vh-d takewiki

xrjulia's Issues

Error when creating a new evaluator with specified port

Hi,

I am trying to create a new Julia evaluator on a specified port number myportid using RJulia(port = as.integer(myportid), .makeNew = T) or XRJulia::JuliaInterface(port = as.integer(myportid)), but every time this throws the following error:

Error in .Object$initialize(...) : 
  Unable to start Julia connection on port 1319 : all connections are in use

even if I choose a port number equal to 1 + the port number of the previous evaluator (what is done by default) and even though I don't have any connection on this port. Is there any way to specify manually the port number for an evaluator?

The idea would be to run a code in parallel in R using mclapply, and for each iteration create a new Julia evaluator on which I can run my computations. I cannot simply use RJulia(.makeNew = T) otherwise the different R threads try to open a connection on the same port (as the evalutor creation runs in parallel).

Thanks for your help!

Suggest mentioning how to install Julia in the setup instructions

This looks like a great package and I am excited to try it.

I would like to suggest in your install instructions to mention that you need to install Julia and set the Path variable (I am on Windows 10). I realise this is blindingly obvious to you but data scientists like myself are not always aware of the behind-the-scenes stuff, and it is a barrier to entry.

Running long code on Julia returns error in R

Hi,

I am trying to run long simulations in Julia (using the module BioSimulator.jl), calling the appropriate functions from R, via the function juliaCall. In some cases the simulation is really long (i.e. more than 10 minutes). When trying to run it, after approx. 10 minutes the R interface throws the following error:

Error: lexical error: invalid char in json text.
                                       NA
                     (right here) ------^

But the Julia server continues to run the computation.

This is due to the fact that the JuliaInterface method ServerTask is constructed such that it tries for ten minutes (10 trials with a sleep time of 1 sec between them) to listen to the connection. After these 10 trials if the method wasn't able to read anything from the connection it simply return the value value = NA, which when read by the function XR::valueFromServer(value, key, get, .self) from the JuliaInterface method ServerEval throws the error.
I was wondering if there is any way to wait for the Julia server to finish the computation before trying to read something from the server.
I am sorry if my question has a really obvious answer, I am not yet familiar with the XR and XRJulia package.

Thanks for your help!

Calling Julia functions in R very slow due to overhead of juliaGet()

I am testing calibration in R of a model in Julia. It works, but having to use juliaGet() for each model call makes it extremely slow. Is there a faster way to do it, where the Julia functions act like native R functions without the overhead?

My Julia module is

# linmod_module.jl
module linmod_module

export linmod, linmodvec, loglikelihood

function linmod(x, a, b)
  y = a + b * x
  return y
end

function linmodvec(datax, a, b)
  y = linmod(datax, a, b)
  return y
end

function loglikelihood(datay, datax, a, b, error)
  datan = size(datax, 1)
  y = linmod(datax, a, b)
  z = (datay - y) / error
  ll = - 0.5 * datan * log(2 * pi * error * error) - 0.5 * sum(z .* z);
  return ll
end

end

My R Script is

# try BayesianTools with Julia model

library(XRJulia)
library(BayesianTools)
# library(coda)

# make some data
x_data <- seq(0, 1, length.out=1000)
y_data <- 0.5+0.5*x_data+rnorm(length(x_data),0,0.1)
model_se <- 0.1

# find Julia functions
findJulia(test=TRUE)
ev <- RJulia() # create Julia evaluator
ev$AddToPath(getwd()) # find module linmod

# test Julia model
linmod <- JuliaFunction("linmod", module="linmod_module")
test <- linmod(0.5, 0.5, 0.5)

# test vectorised Julia model
linmodvec <- JuliaFunction("linmodvec", module="linmod_module")
test <- linmodvec(x_data, 0.5, 0.5)
juliaGet(test) # but this is very slow

# test Julia loglikelihood
loglikelihood <- JuliaFunction("loglikelihood", module="linmod_module")
test <- loglikelihood(y_data, x_data, 0.5, 0.5, model_se)

# BayesianTools setup
bt_names <- c("a", "b")
bt_prior <- createUniformPrior(c(0,0), c(1,1))
bt_likelihood <- function(params){
  loglikelihood(y_data, x_data, params[1], params[2], model_se)
}
bt_parallel <- FALSE
bt_setup <- createBayesianSetup(likelihood=bt_likelihood,
                                prior=bt_prior,
                                parallel=bt_parallel,
                                names=bt_names)
nBurnin <- 900
nSample <- 900
nTotal <-  nSample + nBurnin
nChains <- 3
nInternal <- 3
bt_settings <- list(startValue=nInternal, # DREAMzs internal chains
                    iterations=nTotal/nChains, # iterations per external chain
                    nrChains=nChains, 
                    burnin=nBurnin/nChains+1, # discard these ones
                    parallel=bt_parallel,
                    message=TRUE) 
bt_out <- runMCMC(bayesianSetup=bt_setup, 
                  sampler = "DREAMzs", 
                  settings=bt_settings)

suppressWarnings(summary(bt_out))
tracePlot(bt_out)
correlationPlot(bt_out)

bt_predict <- function(params){
  juliaGet(linmodvec(x_data, params[1], params[2]))
}

bt_error <- function(mean, params){
  return(rnorm(length(mean), mean=mean, sd=model_se))
}

bt_samples <- getSample(bt_out, start=2)
summary(bt_samples)

# this throws an error
suppressMessages({
  plotTimeSeriesResults(sampler=bt_samples,
                      model=bt_predict,
                      observed=y_data,
                      error=bt_error,
                      plotResiduals=TRUE,
                      main="Residual Analysis")
})

Error: Julia expression is too large

Hi, I found that once the matrix size is even moderately large, the XRJulia will give error:

library("XRJulia")
ev <- RJulia()
juliaCommand("using Base.LinAlg")
n = 500; p = 700; r = 3
U = matrix(rnorm(n*r), n, r)
V = matrix(rnorm(p*r), p, r)
X = U %*% t(V)
Xm = juliaSend(X)
juliaCall("svdfact", Xm)

Julia error: syntax: expression too large
Error: C stack usage  12608550 is too close to the limit

It seems that this can be got around by evaluating everything in Julia environment:

ev$Command("using Base.LinAlg")
ev$Command("n = 500; p = 700; r = 3")
ev$Command("U = randn(n, r); V = randn(p, r); X = U*V'")
ev$Eval("svdfact(X)")

But is this the only way to resolve the issue? I guess this would be very inconvenient if the large-scale input for functions in julia is computed from some other R functions. Thanks for your help!

XRJulia's `juliaEVal` doesn't seem to work on Windows

library(XRJulia)
findJulia(test = TRUE)

The above works fine and returns the path to Julia.

When I run the below it just hangs and doesn't return.

regjl <- juliaEval("
  function reg(x,y)
    n=size(x,1)
    xreg=hcat(ones(size(x)[1],1),x)
    k=size(xreg,2)
    p1=((xreg'xreg)^(-1))
    b=p1*xreg'y
    r=y-xreg*b
    sig=(r'r)/(n-k)
    vmat=sig[1]*p1
    sigb=sqrt(diag(vmat))
    t=b./sigb
 
    return (b,t)
  end
")

Below is my session info and I am using JuliaPro 0.6.0.1

R version 3.4.1 (2017-06-30)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

Matrix products: default

locale:
[1] LC_COLLATE=English_Australia.1252 
[2] LC_CTYPE=English_Australia.1252   
[3] LC_MONETARY=English_Australia.1252
[4] LC_NUMERIC=C                      
[5] LC_TIME=English_Australia.1252    

attached base packages:
[1] stats     graphics 
[3] grDevices utils    
[5] datasets  methods  
[7] base     

other attached packages:
[1] XRJulia_0.7         
[2] RevoUtilsMath_10.0.0
[3] RevoUtils_10.0.5    
[4] RevoMods_11.0.0     
[5] MicrosoftML_1.5.0   
[6] mrsdeploy_1.1.2     
[7] RevoScaleR_9.2.1    
[8] lattice_0.20-35     
[9] rpart_4.1-11        

loaded via a namespace (and not attached):
 [1] codetools_0.2-15      
 [2] CompatibilityAPI_1.1.0
 [3] foreach_1.4.4         
 [4] grid_3.4.1            
 [5] R6_2.2.0              
 [6] jsonlite_1.4          
 [7] curl_2.6              
 [8] iterators_1.0.8       
 [9] tools_3.4.1           
[10] yaml_2.1.14           
[11] compiler_3.4.1        
[12] XR_0.7                
[13] mrupdate_1.0.1       

Last digit of floating point numbers truncated in R -> Julia connection

Hi,

This seems to round the inputs.

library(XRJulia)
Sys.setenv(JULIA_BIN = "/home/celrod/Documents/julia-static-llvm/usr/bin/julia",
           LD_LIBRARY_PATH="/usr/local/lib64");
findJulia()

id <- JuliaFunction("identity")

id(pi) - pi
# -3.108624e-15

id(2*pi - id(pi)) - pi # was hoping to compensate
# 7.105427e-15

id(exp(1)) - exp(1)
# 4.884981e-15

id(1/3) - 1/3
# -3.330669e-16

nf <- JuliaFunction("nextfloat")
nf(pi) - pi
# -2.664535e-15
nf(nf(pi)) - pi
# -2.664535e-15
nf(nf(nf(pi))) - pi
# -2.664535e-15

I got this on two computers, both running Ubuntu 16.04 and Julia 0.6.2.

EDIT:
To say something positive, I noticed this issue because of this test problem:

assess_approx <- function (ints, target){
  (ints[1] / ints[2]) %>% (function(approx) c(approx, approx-target))
}
juliaEval("function check_approx!(best_inds, i, j, target, best)
            diff = abs(i / j - target)
            @inbounds if diff < best
              best_inds[1] = Int(i)
              best_inds[2] = j
            return diff
            end
            best
          end")
juliaEval("function slow_approx(n::Int, target=pi)
            best = Inf
            best_inds = [0,0]
            for j in 1:n
              i_mid = j*target
              best = check_approx!(best_inds, floor(i_mid), j, target, best)
              best = check_approx!(best_inds, ceil(i_mid), j, target, best)
            end
            best_inds
          end")

sa_jl <- JuliaFunction("slow_approx")
slow_approx_jl <- function(x) juliaGet(sa_jl(as.integer(x)))
slow_approx_jl(1e3)
system.time(exactjl <- slow_approx_jl(1e9))
#   user  system elapsed 
#  0.006   0.000   7.239
assess_approx(exactjl,pi)
# 3.141593 0.000000
slow_approx_jl2 <- function(x, y = pi) juliaGet(sa_jl(as.integer(x),y))
slow_approx_jl2(1e3)
system.time(closejl <- slow_approx_jl2(1e9))
#   user  system elapsed 
#  0.002   0.004   7.363
assess_approx(closejl,pi)
# 3.141593e+00 -3.108624e-15
assess_approx(closejl,id(pi))
# 3.141593 0.000000

versus

library(Rcpp); library(magrittr)
"#include <Rcpp.h>
#include <math.h>
#include <limits>
#include <tuple>
using namespace Rcpp;
typedef std::tuple<double, double, int> best_tup;
  best_tup check_approx(double i, int j, double target, best_tup best){
  double diff = std::abs(i / j - target);
  return diff < std::get<0>(best) ? std::make_tuple(diff, i, j) : best;
}
// [[Rcpp::export]]
IntegerVector brutish_tup(int n, double target) {
  best_tup best = std::make_tuple(std::numeric_limits<double>::infinity(),0.0,0);
  for (int j=1; j <= n; j++ ){
    double i_mid = j*target;
    best = check_approx( std::floor(i_mid), j, target, best);
    best = check_approx( std::ceil(i_mid), j, target, best);
  }
  return IntegerVector::create((int) std::get<1>(best), std::get<2>(best));
}" %>% sourceCpp(code = .) #requires c++11
brutish_tup(500, pi)
## [1] 355 113
system.time(exact <- brutish_tup(1e9, pi))
#   user  system elapsed 
# 14.904   0.011  14.916
assess_approx(exact, pi)
# 3.141593 0.000000

On my other computer the best (for C++) relative time I got was about 17 vs 5 seconds.
I don't know C++ well, so perhaps I'm doing something very wrong that is killing performance -- when I tried a version that looks more like the Julia code, C++ took closer to 50 seconds. Theoretically, C++ should not be slower, so I'll look into that more.
I just bring this up to say, XRJulia seems to be working well -- in this case it's looking much better than Rcpp.

EDIT:
#12 (comment)
"have to format and then parse data every time."
It looks like the problem here is that it truncates the last digit.

julia> pi |> Float64
3.141592653589793

julia> 3.14159265358979 - pi
-3.1086244689504383e-15

julia>|> Float64
2.718281828459045

julia> 2.71828182845904 --4.884981308350689e-15

julia> exp(1)
2.718281828459045

julia> 1/3
0.3333333333333333

julia> 0.333333333333333 - 1/3
-3.3306690738754696e-16

Change ASCIIString to String

Using ASCIIString doesn't seem to work any more. I simply changed it to String in my package and it works fine.

Long arrays wrongly converted from Julia to R

Hi,

I noticed a strange problem when playing with long arrays and passing them from R to Julia. When creating arrays of length <1000, no problem:

m = juliaEval("[i for i in 1:999]")
juliaGet(m)

returns:

  [1]   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33
 [34]  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60  61  62  63  64  65  66
... 
[991] 991 992 993 994 995 996 997 998 999

However, if the array length exceeds 1001, the value returned to R is as follow:

m = juliaEval("[i for i in 1:1001]")
juliaGet(m)
  [1]   1   0   2   0   3   0   4   0   5   0   6   0   7   0   8   0   9   0  10   0  11   0  12   0  13   0  14   0  15   0  16   0  17
  [34]   0  18   0  19   0  20   0  21   0  22   0  23   0  24   0  25   0  26   0  27   0  28   0  29   0  30   0  31   0  32   0  33   0
  ...
 [991] 496   0 497   0 498   0 499   0 500   0 501

Zeros are inserted between the values of the array that Julia returns. Is there any way to correct that?

Thank you very much!

Olivia

Update CRAN version of XRJulia (v 0.9.0)?

Hi,

I don't know if this really counts as an issue: I would like to submit my package (sismonr) to CRAN. However it depends on version 0.9.0 of XRJulia, which is currently only available on Github and not CRAN.
Are you planning to update XRJulia in the CRAN?

Thank you!

Problem using XRJulia on a sun grid engine (UGER)

Hello - I'm having a problem where my jobs using XRJulia crash when I submit them to my compute farm (even though they execute fine interactively).

Log from 1 example R job:


Loading required package: XR

Loading required package: XRJulia

Loading MSACS

Loading MSACS v0 [ALPHA]

Error in .setupMethodsTables(fdef, initialize = TRUE) : 

  trying to get slot "group" from an object of a basic class ("NULL") with no slots

Calls: <Anonymous> ... <Anonymous> -> as -> .getMethodsTable -> .setupMethodsTables

Execution halted

Does this error msg mean anything to you? Since I can't reproduce this in an interactive shell, I have no way to debug w. my limited skills.

Also - is it possible that this is due to my hacky implementation? I have not bothered set things up as a package, I'm just sourcing julia files on startup as follows:


XRJulia::findJulia(test = TRUE)
XRJulia::juliaUsing("Distributions")

XRJulia::juliaSource( paste0(pkg_dir, "/msacs/julia/onesegmulti-model.jl") )
XRJulia::juliaSource( paste0(pkg_dir, "/msacs/julia/MS_HMM_BM_call.jl") )
MS_HMM_BM_call = XRJulia::JuliaFunction(name="MS_HMM_BM_call" )
get_f_1d_grid_approx = XRJulia::JuliaFunction(name="get_f_1d_grid_approx" )
res = try(XRJulia::juliaImport("OneSegMulti"))
if( class(res)=="try-error") XRJulia::juliaImport("OneSegMulti")  ## trying this again works!!! don't ask why.

I never understood why the last import needs to be tried twice.

Thanks for your help!

Scott

Julia v0.7

Hi,

I was wondering if you are planning at some point to allow using the package with Julia v0.7?

Thank you!

asRObject() cannot erase temp files on Windows

Hi,

I have been using XRJulia on Windows. If I run:

bigvector = juliaGet(juliaEval("collect(1:100000)"))

I get the following warning:

/Rtools/bin/rm: cannot remove '[TMPDIR]\Rtmp8uV2Db\Julia25502a4466b5_212': Device or resource busy

I tried to track down the cause of the warning, and it turns out the asRObject method defined for vector_R_direct tries to erase the file containing the value (l.803 of RJuliaConnect.R):

on.exit(base::system(paste("rm ",object@file)))

Which is impossible because the file is used by the Julia process. It happened on 2 different Windows computers, but not in Linux.
Is there a way to bypass the warning and force the rm?

Thanks!

juliaCMD doesn't quote testFile path on Windows (with fix)

XRJulia 0.7.7, on Windows 10, with successful findJulia, with a user account name that has a space:

juliaEval("1+1")
ERROR: could not open file C:\Users\Alan
Stacktrace:
[1] include_from_node1(::String) at .\loading.jl:576
[2] include(::String) at .\sysimg.jl:14
[3] process_options(::Base.JLOptions) at .\client.jl:305
[4] _start() at .\client.jl:371
Error in .Object$initialize(...) :
No JSON module in Julia and unable to add: try in julia

This fixes the problem by adding quotes around the testFile, just as they are added around
julia_bin in juliaCMD:

assignInNamespace("juliaCMD",
function (julia_bin, testFile) {
if (.Platform$OS.type == "windows")
paste0(""", julia_bin, "" ", paste0(""",testFile,"""))
else paste(julia_bin, "<", testFile)
},
"XRJulia")

Doesn't seem to work with Julia v1.0

Hello, I assume the package is not adjusted yet to the new Julia release?
Tried on Mac:
`> library(XRJulia)

findJulia()
[1] "/usr/local/bin/julia"
findJulia(test = TRUE)
ERROR: MethodError: no method matching parse(::String)
Closest candidates are:
parse(!Matched::Type{T<:Integer}, !Matched::AbstractChar; base) where T<:Integer at parse.jl:38
parse(!Matched::Type{LibGit2.GitCredential}, !Matched::AbstractString) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/LibGit2/src/gitcredential.jl:71
parse(!Matched::Type{LibGit2.GitCredentialHelper}, !Matched::AbstractString) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/LibGit2/src/gitcredential.jl:158
...
Stacktrace:
[1] top-level scope at none:0
ERROR: UndefVarError: STDOUT not defined
Stacktrace:
[1] top-level scope at none:8
[1] FALSE`
It worked with 0.6 version.

Parallelizing julia calls

As expected, calling a julia function within a parallel loop (using e.g. foreach and %dopar%) results in errors as the threads collide.

Intuitively, it seems like there should be a way to startup a julia process for each thread and then pass the connection as an arg to the parallel function.

If someone could help walk me through this I would be very grateful.

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.