Comments (8)
from ad-delcont.
I don't know what to make of this though. On one hand, this example is not what I'd call a typical application of AD. But you're right, there's certainly a fundamental limitation in the code somewhere. I hope someone with more expertise in debugging laziness issues can shed some light on this.
from ad-delcont.
Yeah, there is some potential for lazyness to build up in some places, e.g. the Num
instance (+) = op2Num $ \x y -> (x + y, id, id)
. If you have a lot of additions you might end up with fibInt = fibN 1 50 = 12586269025
as a thunk 1+ 1 + 1 + ...
, which can easily blow up memory.
from ad-delcont.
Just replacing it by (+) = op2Num $ \x y -> let sum = x + y in sum
seq (sum, id, id)
doesn't fix it yet though. But since you're building up thunks of function compositions in the other elements of the tuple, it's hard to prevent a space leak there I'd guess since that can't be easily normalised.
from ad-delcont.
Thank you @turion . It really sounds like a radical redesign is in order. In its current version, ad-delcont
is mostly a straightforward port of the implementation from the Wang article, which they were able to pull off by using Scala which is eagerly-evaluated (IIRC).
from ad-delcont.
Somewhat simper example:
forkJoin :: Fractional a => a -> a
forkJoin x = (x+x)/2
forkJoinChain :: Fractional a => a -> a
forkJoinChain x = iterate forkJoin x !! 20
forkJoinChainAD :: Float -> (Float, Float)
forkJoinChainAD x = rad1g 0 1 forkJoinChain x
I don't think lazyness is a problem here. At worst it may cause memory consumption to be proportional to the amount of computation, which shouldn't be an issue with only tens of variables.
My guess would be that this piece of code causes combinatorial explosion:
op2_ zeroa plusa plusb f ioa iob = do
ra <- ioa
rb <- iob
ioa
and iob
are computations, that are evaluated every time they are needed. Each of them has two parents, each of them in turn also has two parents, etc.
Maybe AD
variable should also store the value and STRef
for gradient along with ContT
computation, maybe some flag to avoid repeated evaluations. Like laziness, but manual.
Anyway, I don't really grok the code, so it's all just wild guesses.
from ad-delcont.
Have you tried UseStrict to see if it goes away?
from ad-delcont.
@tonyday567 thanks but I just tried, -XStrict doesn't help.
from ad-delcont.
Related Issues (3)
- Radical redesign HOT 1
- Add tests
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ad-delcont.