Comments (5)
After #124:
julia> using JET
julia> foo() = fetch(Threads.@spawn 1 + "foo")
foo (generic function with 1 method)
julia> @report_call foo()
═════ 1 possible error found ═════
┌ @ threadingconstructs.jl:174 Base.Threads.Task(#3)
│┌ @ task.jl:5 #self#(f, 0)
││┌ @ threadingconstructs.jl:170 Main.+(1, "foo")
│││ no matching method found for call signature: Main.+(1, "foo")
││└──────────────────────────────
Any
From the technical reason, there is a limitation that JET currently runs the additional analysis pass on Task
construction, regardless of whether it's really schedule
d or not.
I'd like to run the analysis pass only when we encounter schedule(::Task)
call, but I'm not sure if there is any good solution for that, so I'd let this issue to be closed as is for now.
from jet.jl.
I think a challenging point here is that fetch(@async f())
is effectively Base.invokelatest(f)
:
julia> f() = 1
f (generic function with 1 method)
julia> t = Task(f)
Task (runnable) @0x00007f5c77464c40
julia> f() = 2
f (generic function with 1 method)
julia> schedule(t)
Task (done) @0x00007f5c77464c40
julia> fetch(t)
2
If we can change how Task
interacts with the world age, I think it's actually fixable without bringing the Big Gun (JuliaLang/julia#39773).
You can find related discussions here:
- what world should Tasks run in? · Issue #35690 · JuliaLang/julia
- Can we have inferable fetch(task)? - Internals & Design - JuliaLang
- A PoC inferable task: https://github.com/tkf/InferableTasks.jl
from jet.jl.
Currently Julia compiler doesn't run type inference on Threads.@spawn
ed code block, as shown below:
julia> code_typed(; optimize = false) do
fetch(Threads.@spawn 1 + 2)
end |> first
CodeInfo(
1 ─ (#34 = %new(Main.:(var"#34#36")))::Core.Const(var"#34#36"())
│ %2 = #34::Core.Const(var"#34#36"())
│ (task = Base.Threads.Task(%2))::Task
│ %4 = false::Core.Const(false)
│ Base.setproperty!(task, :sticky, %4)::Any
└── goto #3 if not false
2 ─ Core.Const(:(Base.Threads.put!(Main.:(var"##sync#41"), task)))::Union{}
3 ┄ Base.Threads.schedule(task)::Any
│ %9 = task::Task
│ %10 = Main.fetch(%9)::Any
└── return %10
) => Any
I think JuliaLang/julia#39773 is an attempt to enable various optimizations including type inference for multithreading contexts like this (am I right, @tkf ?).
That said, for the time being, I'm willing to make JET special case some of multithreading code and enable JET analysis on e.g. @spawn
blocks, but I won't extend the native type inference routine in any other aspects, like annotating return type of @spawn
code block, it's a job that JuliaLang/julia#39773 should do (technically, @spawn
ed code is internally represented as closure and I guess it's not too hard to implement a special analysis pass on it).
My question is, is it enough to handle @spawn
and @threads
macros for supporting analysis on multithreading code ? Or is there any other code snippet that is commonly used for multithreading ?
from jet.jl.
I've only ever used @spawn
and @threads
, and don't know about other ways.
Having JET work for these two macros would be nice, but there is also something to be said for simply waiting until Base Julia provides it.
It depends on how easy or hard it is to implement in JET, I suppose.
from jet.jl.
We've also noticed that report_opt
is handled differently in multithreaded context:
julia> function abmult(r::Int)
if r < 0
r = -r
end
# the closure assigned to `f` make the variable `r` captured
f = x -> x * r
return f
end;
julia> JET.@report_opt abmult(42)
═════ 3 possible errors found ═════
┌ @ REPL[97]:2 r = Core.Box(_7::Int64)
│ captured variable `r` detected
└──────────────
┌ @ REPL[97]:2 %7 < 0
│ runtime dispatch detected: (%7::Any < 0)::Any
└──────────────
┌ @ REPL[97]:3 -(%14)
│ runtime dispatch detected: -(%14::Any)::Any
└──────────────
julia> JET.@report_opt fetch(Threads.@spawn abmult(42))
═════ 3 possible errors found ═════
┌ @ task.jl:360 wait(t)
│┌ @ task.jl:343 Base._wait(t)
││┌ @ task.jl:301 lock(%13)
│││ runtime dispatch detected: lock(%13::Any)::Any
││└───────────────
││┌ @ task.jl:304 wait(%30)
│││ runtime dispatch detected: wait(%30::Any)::Any
││└───────────────
││┌ @ task.jl:307 unlock(%40)
│││ runtime dispatch detected: unlock(%40::Any)::Any
││└───────────────
Note that the boxed variable is not reported when @spawn
ed. We were hoping to use JET to help detect cases, where one has to interpolate into the @spawn
ed task in order to prevent boxing.
from jet.jl.
Related Issues (20)
- Define `@test_package` in addition to `report_package`
- Optional analyzer keyword argument gone HOT 2
- JET errors on interface definitions (as it should) - what to do?
- │ ArgumentError: Module JuliaFormatter does not correspond to a loaded package! HOT 2
- JET breaks SnoopCompile
- Add Error Analysis for OptimizationFailureReport
- Add a hook to analyze calls that appear at runtime HOT 1
- False positive for indexing Union{Tuple{},Vector{T}} HOT 2
- Ergonomics: descending into errors HOT 3
- List of Base reports HOT 1
- Line numbers do not Revise HOT 2
- Support JET annotations in source code HOT 4
- False positive: accessing variables from other, possibly undefined modules HOT 1
- Branches that restrict a type HOT 14
- Support "import Module as" syntax HOT 1
- Possible false positive with sort! HOT 4
- Support package extensions HOT 3
- main_uuid not defined HOT 10
- Misattributed instability due to recursion
- Feature request: Catch atomic errors
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jet.jl.