bob-carpenter/ad-handbook

Automatic Differentiation Handbook

ad-handbook's Issues

Additional R build dependencies, possible missing file

Working on building the book. So far I've needed these R packages in addition to the ones in the README:

• ggplot2
• gridExtra
• rstan
• formatR

Now I'm getting this error:
! LaTeX Error: File 'tufte-book.cls' not found.
Maybe an uncommitted file?

Small issue in the intro

Saw your post on discourse and was checking this out. I noticed in the index.Rmd you write:

Automatic differentiation is a general technique for lifting a function computing values to one that also computes derivatives.

Two small issues that impact the understanding. 1) Could you explain or use different terminology for what "lifting" means (I found some references to lifting a function below) and 2) I see the dependent clause after 'function' was used to loosely define this, maybe just expand into a new sentence such as

Automatic differentiation is a general technique for lifting a function from computing values to also computing derivatives.

• https://fsharpforfunandprofit.com/posts/elevated-world/
• https://www.rdocumentation.org/packages/purrr/versions/0.2.5/topics/lift
• https://stackoverflow.com/questions/17965059/what-is-lifting-in-scala

Formatting for derivatives, adjoints, tangents etc. and some more matrix results?

Looking over the collection of known derivatives etc. should that section have a header and a little, "Hey these are some common cases if you want to use these."?

It might also be nice to format them as a list

[Operation]:

• Derivative:
• Adjoint:
• Tangent:

There's also a few matrix algebra ops I can write here if you want them.

Prereqs for building the book

I needed to install these packages in R:

1. tufte
2. reshape

I'm sure I have a bunch already installed.

Where's a good place to put this info? I was originally going to modify the README.md with a ## Before building the book subsection, but then thought it could be useful to have an install.R script or something else programmatic.

Contribution: section on implicit functions

Restating my post on https://discourse.mc-stan.org/t/help-write-an-autodiff-handbook/12640/13.

I propose grouping the chapters on ODEs, algebraic equations, and integrals, into one section on implicit functions. This is in part because for all these methods, we can apply the adjoint method to construct differentiation algorithms.

The content would be as follows:

Automatic differentiation for implicit functions

1. Algebraic equations: implicit function theorem, adjoint method, practical considerations
2. Optimization problems: extending results from the algebraic equations, equality constraint, inequality constraints, conditions for existence of derivatives and diagnostics
3. Ordinary differential equations: full Jacobian, adjoint method, tricks for users
4. Action of matrix exponential: adjoint method based on ODE case
5. Differential algebraic equations: adjoint method
6. Discrete sequences: adjoint method

We've already written a lot of this down, it's just matter of putting it together.