wildart/manifoldlearning.jl

A Julia package for manifold learning and nonlinear dimensionality reduction

ManifoldLearning

A Julia package for manifold learning and nonlinear dimensionality reduction.

Documentation	Build Status

Methods

• Isomap
• Diffusion maps
• Locally Linear Embedding (LLE)
• Hessian Eigenmaps (HLLE)
• Laplacian Eigenmaps (LEM)
• Local tangent space alignment (LTSA)
• t-Distributed Stochastic Neighborhood Embedding (t-SNE)

Installation

The package can be installed with the Julia package manager. From the Julia REPL, type ] to enter the Pkg REPL mode and run:

pkg> add ManifoldLearning

Examples

A simple example of using the Isomap reduction method.

julia> X, _ = ManifoldLearning.swiss_roll();

julia> X
3×1000 Array{Float64,2}:
  -3.19512  3.51939   -0.0390153  …  -9.46166   3.44159
  29.1222   9.99283    2.25296       25.1417   28.8007
 -10.1861   6.59074  -11.037         -1.04484  13.4034

julia> M = fit(Isomap, X)
Isomap(outdim = 2, neighbors = 12)

julia> Y = transform(M)
2×1000 Array{Float64,2}:
 11.0033  -13.069   16.7116  …  -3.26095   25.7771
 18.4133   -6.2693  10.6698     20.0646   -24.8973

Performance

Most of the methods use k-nearest neighbors method for constructing local subspace representation. By default, neighbors are computed from a distance matrix of a dataset. This is not an efficient method, especially, for large datasets.

Consider using a custom k-nearest neighbors function, e.g. from NearestNeighbors.jl or FLANN.jl.

See example of custom knn function here.

manifoldlearning.jl's Issues

DiffMap-error : k not defined

Hello - Thanks for the package.
Following Problem:
The Diffusionmap example with its 3 parameters from the documentation.
Some X Matrix{Float64} (if it matters in this case 200x16) with
Y = transform(DiffMap, X; d=2, t=1, ɛ=1.0)
yields

`
UndefVarError: k not defined

Stacktrace:
[1] top-level scope
@ In[190]:5
[2] eval
@ ./boot.jl:360 [inlined]
[3] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
@ Base ./loading.jl:1094
`

Even defining some global k does not ged rid of the error.

Tag a new version?

`rw` normalization non-negotiable for `DiffusionMap`?

for the diffusion map, by default α::Real=0.0 and this code here to normalize the kernel matrix is not called.

I'm pretty sure the normalize!(L, α=α, norm=:rw) line is non-negotiable. otherwise, the kernel matrix will not have columns sum to one, breaking the theory behind the diffusion map. pretty sure this needs to happen to turn the matrix into a Markov transition matrix. see Sec. 2.1 here. agree? disagree?

Unexpected size of output of Isomap transform

Another behavior I can't explain on data from example.txt.

Load it with

import DelimitedFiles: readdlm
using ManifoldLearning

example = readdlm("example.txt");

But when I run transform on different subsets, output size doesn't always match to size of input:

• size(transform(Isomap, example[:, 1:500], d=2).proj, 2) returns 500
• size(transform(Isomap, example[:, 1:1000], d=2).proj, 2) returns 999 (why not 1000?)
• size(transform(Isomap, example[:, 1:1141], d=2).proj, 2) returns 1141 (again as expected)

MethodError for `connected_components`

Trying the "Getting started" example from the docs:

julia> using ManifoldLearning

julia> X, _ = ManifoldLearning.swiss_roll();

julia> M = fit(Isomap, X)
ERROR: MethodError: no method matching connected_components(::SimpleWeightedGraphs.SimpleWeightedGraph{Int64, Float64})
Closest candidates are:
  connected_components(::LightGraphs.AbstractGraph{T}) where T at C:\Users\sternlab\.julia\packages\LightGraphs\IgJif\src\connectivity.jl:100
Stacktrace:
 [1] largest_component(G::SimpleWeightedGraphs.SimpleWeightedGraph{Int64, Float64})
   @ ManifoldLearning C:\Users\sternlab\.julia\packages\ManifoldLearning\ocJhX\src\utils.jl:52
 [2] fit(::Type{Isomap}, X::Matrix{Float64}; k::Int64, maxoutdim::Int64, nntype::Type)
   @ ManifoldLearning C:\Users\sternlab\.julia\packages\ManifoldLearning\ocJhX\src\isomap.jl:54
 [3] fit(::Type{Isomap}, X::Matrix{Float64})
   @ ManifoldLearning C:\Users\sternlab\.julia\packages\ManifoldLearning\ocJhX\src\isomap.jl:52
 [4] top-level scope
   @ REPL[43]:1

This happens for all methods except DiffMap. Tested in julia 1.6.2 in a new temp environment.

(jl_wfx66s) pkg> status
      Status `C:\Users\sternlab\AppData\Local\Temp\jl_wfx66s\Project.toml`
  [06eb3307] ManifoldLearning v0.6.2

(jl_wfx66s) pkg> status -m
      Status `C:\Users\sternlab\AppData\Local\Temp\jl_wfx66s\Manifest.toml`
  [ec485272] ArnoldiMethod v0.1.0
  [7d9fca2a] Arpack v0.4.0
  [d360d2e6] ChainRulesCore v1.11.1
  [861a8166] Combinatorics v1.0.2
  [34da2185] Compat v3.40.0
  [9a962f9c] DataAPI v1.9.0
  [864edb3b] DataStructures v0.18.10
  [ffbed154] DocStringExtensions v0.8.6
  [86223c79] Graphs v1.4.1
  [d25df0c9] Inflate v0.1.2
  [3587e190] InverseFunctions v0.1.1
  [92d709cd] IrrationalConstants v0.1.1
  [093fc24a] LightGraphs v1.3.5
  [2ab3a3ac] LogExpFunctions v0.3.4
  [1914dd2f] MacroTools v0.5.9
  [06eb3307] ManifoldLearning v0.6.2
  [e1d29d7a] Missings v1.0.2
  [6f286f6a] MultivariateStats v0.8.0
  [bac558e1] OrderedCollections v1.4.1
  [699a6c99] SimpleTraits v0.9.4
  [47aef6b3] SimpleWeightedGraphs v1.2.0
  [a2af1166] SortingAlgorithms v1.0.1
  [90137ffa] StaticArrays v1.2.13
  [82ae8749] StatsAPI v1.0.0
  [2913bbd2] StatsBase v0.33.12
  [68821587] Arpack_jll v3.5.0+3
  [0dad84c5] ArgTools
  [56f22d72] Artifacts
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8bb1440f] DelimitedFiles
  [8ba89e20] Distributed
  [f43a241f] Downloads
  [b77e0a4c] InteractiveUtils
  [b27032c2] LibCURL
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [ca575930] NetworkOptions
  [44cfe95a] Pkg
  [de0858da] Printf
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA
  [9e88b42a] Serialization
  [1a1011a3] SharedArrays
  [6462fe0b] Sockets
  [2f01184e] SparseArrays
  [10745b16] Statistics
  [fa267f1f] TOML
  [a4e569a6] Tar
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
  [e66e0078] CompilerSupportLibraries_jll
  [deac9b47] LibCURL_jll
  [29816b5a] LibSSH2_jll
  [c8ffd9c3] MbedTLS_jll
  [14a3606d] MozillaCACerts_jll
  [4536629a] OpenBLAS_jll
  [83775a58] Zlib_jll
  [8e850ede] nghttp2_jll
  [3f19e933] p7zip_jll

Isomap transform fails with "Empty collection" error

Hi,

I run Isomap on the data from example.txt. Here is the result

import DelimitedFiles: readdlm
using ManifoldLearning

example = readdlm("example.txt");
transformed = transform(Isomap, example, d=2)

ArgumentError: collection must be non-empty

Stacktrace:
 [1] _extrema_itr at ./operators.jl:447 [inlined]
 [2] _extrema_dims at ./multidimensional.jl:1455 [inlined]
 [3] #extrema#405 at ./multidimensional.jl:1453 [inlined]
 [4] extrema at ./multidimensional.jl:1453 [inlined]
 [5] #classical_mds#23(::Bool, ::Function, ::Array{Float64,2}, ::Int64) at /home/vp/.julia/packages/MultivariateStats/56OfE/src/cmds.jl:76
 [6] classical_mds(::Array{Float64,2}, ::Int64) at /home/vp/.julia/packages/MultivariateStats/56OfE/src/cmds.jl:63
 [7] top-level scope at In[84]:7

The reason is in 1142'nd element, which is kind of outlier, as far as I understood. Looks like something is wrong with distance matrix on line 47. But as this graph has single connected component, I can't understand why do we have infinite distances in the matrix.

---

BTW, why do you use your own functions for work with graphs and nearest neighbor search? For instance, LightGraphs is optimized and well-tested. And NearestNeighbors package has amazing implementation on KD trees, which is really easy to use and works way faster than estimation of the whole distance matrix.

Isomap nnType: Method error

Hey, thanks for this awesome package.

I'm trying to fit Isomap using nntype=KDTree from NearestNeighbors but I get a method error

The error line being NN = fit(nntype, X) here.

Any advice on how to get around this? Using brute force tree works but I'd like to find an easier solution.
Also, is there a way to pass an already fitted tree? I'd like to re-use the same tree across different analysis steps.

Thank you

Update examples?

The docs say to use transform(DiffMap, X; d=2, t=1, ɛ=1.0) but I get the error

UndefVarError: k not defined

Stacktrace:
 [1] (::getfield(MultivariateStats, Symbol("#kw##transform")))(::NamedTuple{(:d, :t, :ε),Tuple{Int64,Int64,Float64}}, ::typeof(transform), ::Type, ::Array{Float64,2}) at ./none:0
 [2] top-level scope at In[66]:4

Should the docs/readme be updated or is the api still in flux? I got this example working if you want to use it. Not sure how to get diffmap to return a circle like wikipedia claims (any ideas?). Isomap looks good though :)

# Create winding curve
t = collect(1:500) / 500 * 2π
X = zeros(3,500)
X[1,:] = cos.(t).*(.5cos.(6t).+1)
X[2,:] = sin.(t).*(.4cos.(6t).+1)
X[3,:] = 0.4sin.(6t)

# Isomap
Y = transform(fit(Isomap, X))
scatter(Y[1,:], Y[2,:], zcolor=t, legend=nothing, title="isomap")

# Diffmap
M = fit(DiffMap, X, ε=0.3 , t=8, maxoutdim=2)
Y = transform(M)
scatter(Y[1,:], Y[2,:], zcolor=t, legend=nothing, title="diffusion map")

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

MultivariateStats.jl and ManifoldLearning.jl

First off, thank you for the package 💯

Am I correct to affirm that ManifoldLearning.jl is not part of MultivariateStats.jl due to additional dependencies such as LightGraphs.jl?

Problem with running lle.jl

Running the tests, I get this error:

LoadError: LoadError: DimensionMismatch("dimensions must match")
while loading C:\Users\Denis\Documents\Finmod\ManifoldLearning.jl-master\test\lle.jl, in expression starting on line 8
while loading In[1], in expression starting on line 12

in promote_shape(::Tuple{Base.OneTo{Int64},Base.OneTo{Int64}}, ::Tuple{Base.OneTo{Int64}}) at .\operators.jl:406
in promote_shape(::Tuple{Base.OneTo{Int64}}, ::Tuple{Base.OneTo{Int64},Base.OneTo{Int64}}) at .\operators.jl:402
in promote_shape(::SparseVector{Float64,Int64}, ::Array{Float64,2}) at .\operators.jl:397
in _elementwise(::Base.#-, ::Type{Float64}, ::SparseVector{Float64,Int64}, ::Array{Float64,2}) at .\arraymath.jl:57
in #transform#9(::Int64, ::Int64, ::Function, ::Type{ManifoldLearning.LLE}, ::Array{Float64,2}) at C:\Users\Denis.julia\v0.5\ManifoldLearning\src\lle.jl:82
in (::MultivariateStats.#kw##transform)(::Array{Any,1}, ::MultivariateStats.#transform, ::Type{ManifoldLearning.LLE}, ::Array{Float64,2}) at .:0
in include_from_node1(::String) at .\loading.jl:488
in macro expansion; at .\In[1]:14 [inlined]
in anonymous at .:?

Isomap not working in master

julia> m = transform(Isomap, randn(3,100))
WARNING: Dict{K,V}(ks::AbstractArray{K},vs::AbstractArray{V}) is deprecated, use Dict{K,V}(zip(ks,vs)) instead.
 in depwarn at deprecated.jl:73
 in call at deprecated.jl:50
 in transform at /local/home/fredrikb/.julia/v0.4/ManifoldLearning/src/isomap.jl:59
while loading no file, in expression starting on line 0
ERROR: DimensionMismatch("tried to assign 13-element array to 12x1 destination")
 in throw_setindex_mismatch at operators.jl:246
 in setindex_shape_check at operators.jl:298
 in _unsafe_batchsetindex! at multidimensional.jl:322
 in _unsafe_setindex! at multidimensional.jl:294
 in _setindex! at multidimensional.jl:291
 in transform at /local/home/fredrikb/.julia/v0.4/ManifoldLearning/src/isomap.jl:62

Stackoverflow Question about this package

Just saw this: http://stackoverflow.com/questions/33741396/non-empty-collection-error-in-manifoldlearning-jl

[PkgEval] ManifoldLearning may have a testing issue on Julia 0.4 (2014-08-23)

PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.3) and the nightly build of the unstable version (0.4). The results of this script are used to generate a package listing enhanced with testing results.

On Julia 0.4

• On 2014-08-22 the testing status was Tests pass.
• On 2014-08-23 the testing status changed to Tests fail, but package loads.

Tests pass. means that PackageEvaluator found the tests for your package, executed them, and they all passed.

Tests fail, but package loads. means that PackageEvaluator found the tests for your package, executed them, and they didn't pass. However, trying to load your package with using worked.

This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.

Test log:

>>> 'Pkg.add("ManifoldLearning")' log
INFO: Installing ArrayViews v0.4.6
INFO: Installing ManifoldLearning v0.1.0
INFO: Installing MultivariateStats v0.1.1
INFO: Installing StatsBase v0.6.3
INFO: Package database updated

>>> 'using ManifoldLearning' log
Julia Version 0.4.0-dev+311
Commit b850d0a (2014-08-23 04:20 UTC)
Platform Info:
  System: Linux (x86_64-unknown-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

>>> test log
ERROR: test failed: (996 == 1000)
 in expression: size(projection(Y),2) == size(X,2)
 in error at error.jl:21
 in default_handler at test.jl:25
 in do_test at test.jl:50
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in anonymous at no file:13
 in include at ./boot.jl:245
 in include_from_node1 at loading.jl:128
 in process_options at ./client.jl:285
 in _start at ./client.jl:354
 in _start_3B_3595 at /home/idunning/julia04/usr/bin/../lib/julia/sys.so
while loading /home/idunning/pkgtest/.julia/v0.4/ManifoldLearning/test/isomap.jl, in expression starting on line 11
while loading /home/idunning/pkgtest/.julia/v0.4/ManifoldLearning/test/runtests.jl, in expression starting on line 11
Running tests:
 * utils.jl
 * isomap.jl


>>> end of log

Add documentation address

Add the website link https://wildart.github.io/ManifoldLearning.jl/dev/ to the "About" of the package, like this:

[PkgEval] ManifoldLearning may have a testing issue on Julia 0.4 (2015-02-13)

PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.3) and the nightly build of the unstable version (0.4). The results of this script are used to generate a package listing enhanced with testing results.

On Julia 0.4

• On 2015-02-10 the testing status was Tests pass.
• On 2015-02-13 the testing status changed to Tests fail, but package loads.

Tests pass. means that PackageEvaluator found the tests for your package, executed them, and they all passed.

Tests fail, but package loads. means that PackageEvaluator found the tests for your package, executed them, and they didn't pass. However, trying to load your package with using worked.

This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.

Test log:

>>> 'Pkg.add("ManifoldLearning")' log
INFO: Installing ManifoldLearning v0.1.0
INFO: Installing MultivariateStats v0.1.3
INFO: Package database updated
INFO: METADATA is out-of-date a you may not have the latest version of ManifoldLearning
INFO: Use `Pkg.update()` to get the latest versions of your packages

>>> 'using ManifoldLearning' log
Julia Version 0.4.0-dev+3316
Commit c0f2818 (2015-02-12 07:46 UTC)
Platform Info:
  System: Linux (x86_64-unknown-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

>>> test log
WARNING: K>D: regularization will be used
ERROR: LoadError: LoadError: UndefVarError: AbstractSparseArray not defined
 in decompose at /home/idunning/pkgtest/.julia/v0.4/ManifoldLearning/src/utils.jl:155
 in transform at /home/idunning/pkgtest/.julia/v0.4/ManifoldLearning/src/lle.jl:87
 in include at ./boot.jl:249
 in include_from_node1 at ./loading.jl:128
 in anonymous at no file:13
 in include at ./boot.jl:249
 in include_from_node1 at loading.jl:128
 in process_options at ./client.jl:319
 in _start at ./client.jl:403
while loading /home/idunning/pkgtest/.julia/v0.4/ManifoldLearning/test/lle.jl, in expression starting on line 8
while loading /home/idunning/pkgtest/.julia/v0.4/ManifoldLearning/test/runtests.jl, in expression starting on line 11
Running tests:
 * utils.jl
 * isomap.jl
 * lle.jl


INFO: Testing ManifoldLearning
==========================[ ERROR: ManifoldLearning ]===========================

failed process: Process(`/home/idunning/julia04/usr/bin/julia --check-bounds=yes --code-coverage=none --color=no /home/idunning/pkgtest/.julia/v0.4/ManifoldLearning/test/runtests.jl`, ProcessExited(1)) [1]

================================================================================
INFO: No packages to install, update or remove
ERROR: ManifoldLearning had test errors
 in error at error.jl:19
 in test at pkg/entry.jl:717
 in anonymous at pkg/dir.jl:28
 in cd at file.jl:20
 in cd at pkg/dir.jl:28
 in test at pkg.jl:69
 in process_options at ./client.jl:241
 in _start at ./client.jl:403

>>> end of log

OutOfMemoryError()

Hi!

I am trying to run Isomap on a dataset composed of about 2 million, 24-D datapoints, and I get an OutOfMemoryError(). Would you have some tricks/advice about how I can deal with this? I'd really like to use as many of these points as possible for manifold-estimation.

Thank you!

Is diffusion map working?

When I try diffusion map, I encounter this error.
I also found that document needs update.

model = fit(DiffMap, X; dims=2, t=1, ɛ=1.0)

MethodError: no method matching fit(::Type{DiffMap}, ::Array{Float64,2}; dims=2, t=1, ε=1.0)
Closest candidates are:
  fit(::Type{DiffMap}, ::AbstractArray{T<:Real,2}; maxoutdim, t, ε) where T<:Real at /home/pika/.julia/packages/ManifoldLearning/jTQMq/src/diffmaps.jl:38 got unsupported keyword argument "dims"
  fit(!Matched::Type{StatsBase.Histogram}, ::Any...; kwargs...) at /home/pika/.julia/packages/StatsBase/Zfd1X/src/hist.jl:383
  fit(!Matched::StatsBase.StatisticalModel, ::Any...) at /home/pika/.julia/packages/StatsBase/Zfd1X/src/statmodels.jl:152 got unsupported keyword arguments "dims", "t", "ε"
  ...

Stacktrace:
 [1] kwerr(::NamedTuple{(:dims, :t, :ε),Tuple{Int64,Int64,Float64}}, ::Function, ::Type, ::Array{Float64,2}) at ./error.jl:125
 [2] (::getfield(StatsBase, Symbol("#kw##fit")))(::NamedTuple{(:dims, :t, :ε),Tuple{Int64,Int64,Float64}}, ::typeof(fit), ::Type{DiffMap}, ::Array{Float64,2}) at ./none:0
 [3] top-level scope at In[7]:1

If there is anything I can help, I can fix it myself and pull request.
I just need more information about this error...

allow diffusion map to take as input a Gram matrix?

hi. wanting to learn a diffusion map with a custom kernel.
I have my own Gram matrix. (can't explicitly build the feature vectors...)
are you able / can I make a PR to allow this to take as input a Gram matrix whose element (i,j) is k(x_i, x_j) with k the kernel?

https://github.com/wildart/ManifoldLearning.jl/blob/master/src/diffmaps.jl#L66

something like the below?

function fit(::Type{DiffMap}, X::AbstractMatrix{T}; maxoutdim::Int=2, t::Int=1, α::Real=0.0, ɛ::Real=1.0, custom_gram::Bool=false) where {T<:Real}
    if ! custom_gram
      sumX = sum(X.^ 2, dims=1)
      K = exp.(-( transpose(sumX) .+ sumX .- 2*transpose(X) * X ) 
   else
      @assert size(K)[1] == size(K)[2] "Gram matrix must be square!"
      @assert all(K .>= 0.0)
      @assert K == K'
   end
   L = K ./ convert(T, ɛ))

https://inside.mines.edu/~whereman/talks/delaPorte-Herbst-Hereman-vanderWalt-DiffusionMaps-PRASA2008.pdf
going off of the above reference.

edit

I now understand that

    D = Diagonal(vec(sum(L, dims=1)))
    M = inv(D)*L

will normalize the rows :)

thx!

Fit / transform interface for embeddings

Hi,

Thank you for this package! Though it would be great to be able to split fit and transform steps, where it's possible from the algorithm side (like in sklearn implementation of isomap). Great benefit of this algorithm is slow fit and fast transform steps. So, on really large data, you can learn embedding only on small subset and than transform the whole dataset.

Just for reference, here is sklearn code with comments and clear instructions on how to do that.

Connected components

Hi,

I am working with some test data which gives me a Found 2 connected components. Largest component is selected. warning. The data has about 2000 vectors. However, when I run predict on the fitted transform, only 1000 vectors are returned. I am guessing this is because the largest component is selected. I am not very familiar with the algorithm, and was wondering if you could point me towards the paper/chapter that describes it. Furthermore, I would like to use both the components - is that possible ?

Sorry, I would have liked to email you directly, but I can't find your contact information anywhere! :)

Cheers

[PkgEval] ManifoldLearning may have a testing issue on Julia 0.3 (2014-09-14)

PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.3) and the nightly build of the unstable version (0.4). The results of this script are used to generate a package listing enhanced with testing results.

On Julia 0.3

• On 2014-09-13 the testing status was Tests pass.
• On 2014-09-14 the testing status changed to Tests fail, but package loads.

Tests pass. means that PackageEvaluator found the tests for your package, executed them, and they all passed.

Tests fail, but package loads. means that PackageEvaluator found the tests for your package, executed them, and they didn't pass. However, trying to load your package with using worked.

This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.

Test log:

>>> 'Pkg.add("ManifoldLearning")' log
INFO: Cloning cache of ManifoldLearning from git://github.com/wildart/ManifoldLearning.jl.git
INFO: Cloning cache of MultivariateStats from git://github.com/JuliaStats/MultivariateStats.jl.git
INFO: Installing ArrayViews v0.4.6
INFO: Installing ManifoldLearning v0.1.0
INFO: Installing MultivariateStats v0.1.1
INFO: Installing StatsBase v0.6.4
INFO: Package database updated
INFO: METADATA is out-of-date a you may not have the latest version of ManifoldLearning
INFO: Use `Pkg.update()` to get the latest versions of your packages

>>> 'using ManifoldLearning' log
Julia Version 0.3.1-pre+26
Commit 91ba8a9 (2014-08-26 16:55 UTC)
Platform Info:
  System: Linux (x86_64-unknown-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

>>> test log
Running tests:
 * utils.jl
 * isomap.jl

ERROR: test failed: size(projection(Y),2) == size(X,2)
 in error at error.jl:21
 in default_handler at test.jl:19
 in do_test at test.jl:39
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in anonymous at no file:13
 in include at ./boot.jl:245
 in include_from_node1 at loading.jl:128
 in process_options at ./client.jl:285
 in _start at ./client.jl:354
 in _start_3B_1717 at /home/idunning/julia03/usr/bin/../lib/julia/sys.so
while loading /home/idunning/pkgtest/.julia/v0.3/ManifoldLearning/test/isomap.jl, in expression starting on line 11
while loading /home/idunning/pkgtest/.julia/v0.3/ManifoldLearning/test/runtests.jl, in expression starting on line 11
INFO: Testing ManifoldLearning
==========================[ ERROR: ManifoldLearning ]===========================

failed process: Process(`/home/idunning/julia03/usr/bin/julia /home/idunning/pkgtest/.julia/v0.3/ManifoldLearning/test/runtests.jl`, ProcessExited(1)) [1]

================================================================================
INFO: No packages to install, update or remove
ERROR: ManifoldLearning had test errors
 in error at error.jl:21
 in test at pkg/entry.jl:711
 in anonymous at pkg/dir.jl:28
 in cd at ./file.jl:20
 in cd at pkg/dir.jl:28
 in test at pkg.jl:67
 in process_options at ./client.jl:213
 in _start at ./client.jl:354
 in _start_3B_1717 at /home/idunning/julia03/usr/bin/../lib/julia/sys.so


>>> end of log

Custom knn function not working?

I tried the custom knn function (NearestNeighbors.jl) and it seems that passing a custom function doesn't work, as shown here: https://github.com/wildart/ManifoldLearning.jl/blob/master/misc/nearestneighbors.jl

Error:

julia> # Use NearestNeighbors package to find nearest neighbors
       M2 = fit(Isomap, X, knn=knn_nearestneighbors)
ERROR: MethodError: no method matching fit(::Type{Isomap}, ::Array{Float64,2}; knn=knn_nearestneighbors)
Closest candidates are:
  fit(::Type{Isomap}, ::AbstractArray{T,2}; k, maxoutdim, nntype) where T<:Real at /home/user/.julia/packages/ManifoldLearning/vRIIU/src/isomap.jl:49 got unsupported keyword argument "knn"
  fit(::Type{StatsBase.Histogram}, ::Any...; kwargs...) at /home/user/.julia/packages/StatsBase/EA8Mh/src/hist.jl:383
  fit(::StatsBase.StatisticalModel, ::Any...) at /home/user/.julia/packages/StatsBase/EA8Mh/src/statmodels.jl:155 got unsupported keyword argument "knn"
  ...
Stacktrace:
 [1] kwerr(::NamedTuple{(:knn,),Tuple{typeof(knn_nearestneighbors)}}, ::Function, ::Type{T} where T, ::Array{Float64,2}) at ./error.jl:157
 [2] top-level scope at REPL[9]:2

What's the correct way to pass a custom function? If this is not an error on my side the tutorial is a bit out of date.