agentmodels/agentmodels.org

Modeling agents with probabilistic programs

agentmodels.org's Issues

Handle code boxes with only definitions better

Currently, when users try to run such boxes, they result in an error (returnify). We could avoid having such boxes at all, or make it impossible to run them.

viz.print => viz.auto

When I started looking through the code boxes, I found that viz.print failed everywhere with the error message "viz.print is not a function". Looks like it was renamed to viz.auto (and it looks like it works and produces sensible results when I substitute it in), see: probmods/webppl-viz@a2f58c4

Edit cloudflare js/html/css minification

Once we're done with frequent edits to the web book.

Fix: _.contains is not a function

reasoning about agents codeboxes

i've gone through the TODOs for the reasoning-about-agents chapter. the inference examples here are quite simple and explained in detail in the text. (feel free to add a graph of the posterior to these examples if it seems useful).

we need use src/mdpAgent.wppl for the gridworld examples.

we can also use the helpers for the Restaurant Choice problem that i've added to newGridworld.wppl.

finally, there's the POMDP example where we use beliefAgent.wppl. i've added a couple of additional examples to the TODOs. these could be done in separate codeboxes or the same one -- whatever seems best.

Gridworld state labels should not exceed state rectangles

Example:

Add URLs to all entries in bibliography.bib

display and formatting stuff

1. numbered lists (including table of contents and footnotes) should have regular numerals instead of roman ones.
2. there should be a default footer with a header for footnotes "## Foonotes" (see example in 5a), a heading for references, and then links to previous chapter, next chapter and table contents.
3. the table of contents page should have the form "section header" (no link) and then each chapter in section. instead of current setup where the section header is always a link to a chapter.

Update webppl+editor once webppl/editor incompatibility is resolved

That is, once the trampolineRunner PR is merged

generate codeboxes

1. Fix up the current script. We want the examples to be a testable part of webppl-agents. we should be able to run all the scripts (maybe excluding those with graphics) and check for errors.

Procrastination graphs

todos for graphs:

1. create a single python script for all three examples (instead of having copy-pasted scripts). We will re-use this script for the greedy bandits example in the same chapter (see below). so make the graphing code abstract enough to work with both cases.
2. Add legends (instead of text label)
3. x-axis label is "Day"
4. y-axis label for "predictLastMinute" should be "posterior on predictWorkLastMinute"
5. Title: "Inference for Procrastination Problem" (same title for all graphs)
6. add an option to put all graphs in same subplot (rather than having three different files).
7. use ggplot styling (see procrastination_alpha_inference.py)
8. Add graphs to the markdown in the appropriate location. (You may need to scale them down a bit before doing so --- this scaling down should be possible to automate when you save in matplotlib).

Chapter 5c: Possibly Greedy vs. Optimal Model graph

The current codebox compares the inferred utility for arm 0 (chocolate) for the optimal and possibly greedy models. We want a graph like the one for Procrastinate that plots both of these on one axis (with title, legend and so on). The x-axis is "time horizon" and the y axis is "Posterior Expectation for Utility of Arm0 (chocolate)". if that doesn't fit, then just say "Posterior Expectation for Utility of Arm0".

Chapter 5b. Graph showing scaling of optimal vs. myopic agent

The second codebox for Myopic agent on bandits compares the scaling of the optimal and myopic agent. The codebox uses range(9) but range(11) or even range(12) should not take too long. Generate the data for those ranges and make a figure that plots these as curves. The y-axis is "Runtime in seconds" and the x-axis is "Total # of Bandit Trials ("totalTime")".

Inline/side comment system, like Medium

I tried inlineDisqussions. This library has the disadvantage that it uses absolute positioning for the comment locations, so when our content expands (due to webppl results) the locations become inaccurate.

Example in Ch 3a is broken

features for display and visuals

1. TODO in 5a time inconsistency. redo exponential vs. hyperbolic graph in python.
2. how to get numbered lists with regular numerals (not roman numerals)?
3. there should be a default footer with a heading for endnotes (see example in 5a), a heading for references, and then links to previous chapter, next chapter and contents.
4. contents page should have "section header" (no link) and then each chapter in section. instead of current thing where section header is linked.
5. some codeboxes should have an extra button: "sample" which presses run continually (with some default time delay)

Grunt won't compile

When trying to update webppl with sudo sh ./scripts/update-webppl, all of the npm install commands work, but when it gets to grunt compile:../webppl-timeit:../webppl-dp:../webppl-viz:../webppl-gridworld, I get the error

Warning: Task "compile:../webppl-timeit:../webppl-dp:../webppl-viz:../webppl-gridworld" not found. Use --force to continue.
Aborted due to warnings.

Googling suggests that the fix is to edit a gruntfile, but I can't find one in the place I would expect.

something going wrong with updating webppl and wp-editor

the update is taking an extremely long time for each (longer than before). on my local jekyll serve, after update, i get "hasProperties" not found. this function is in src/gridworld.wppl. it seems that it's order has been changed in the minified js. so maybe something is going wrong with minification?

Fixing readme

uglifyjs is deprecated. Use uglify-js instead.

formal results from text

1. "exponential discounting is the unique smooth time consistent function". we want to make a precise statement of this form and then cite an explanation of it. (time inconsistency chapter).
2. the main theorem about the procrastination problem in kleinberg. give a gloss of this theorem.
3. state and cite the equivalence result for myopic/boundVOI agent on two-arm bandits.

chapter codebox -> wppl script

write a shell script that turns codeboxes into .wppl scripts. we'd need a way of labelling codeboxes so that the corresponding scripts have sensible names. something like:

in chapter X:

<!-- codeboxLabel:example1 -->
~~~~
foo();
~~~~

this would become a script named chapterX_example1.wppl.

Can't use print from the command line?

@stuhlmueller
When running the codeboxes from the command line, referencing print throws an error. Owain traced it to print being defined in the editor, which was really surprising to me. Installing and --require webppl-editoring doesn't seem to work. Anything you suggest to try instead?

Perhaps we just shouldn't use print in the codeboxes?

Use better names for gridworld library functions

For example, makeDonut, and makeHike should be something like makeDonutWorld and makeHikeWorld.

viz.print

looks like viz.print still has the old problem. error message is:
k is not a function

Code boxes that use viz don't terminate

fix chapter 4 codeboxes

multiple codeboxes are behaving badly

simplify codeboxes

arguments to GridWorld.draw should be (world, trajectory), where world can be POMDP and trajectory can include actions (which will be ignored).

Broken codeboxes in chapter 5c

A number of the codeboxes in the latter section of chapter 5c (http://agentmodels.org/chapters/5c-joint-inference.html) fail with the error:
makeRestaurantUtilityMDP is not defined

5e-joint-inference codebox #3 broken

_.all is not a function

HTTPS for agentmodels.org and webppl.org

Can you please set up these domains for HTTPS (valid certificates)?

I'm setting up my site using default GitHub pages, which only supports HTTPS. My resources don't load if there are invalid or missing certificates.

Update chapter summaries

diagrams

general points:

1. it's ideal if graphs can be at least partially coded (using JS, Python, or latex). But this might not be flexible enough and so it'll be better to just use keynote.
2. store whatever code / files you use to generate them in a separate folder in assets. put the chapter number in the diagram file name.
3. send me a graph as soon as you have a single example.

bandit graph locations:

1. chapter 3c (first TODO in bandit section)
2. chapter3c (second TODO). prizes should be "0" and "1" (binary bandits).
3. chapter 4: before the first bandit codebox at the end of the chapter. finalized.
4. chapter 5b. for first codebox (greedy), prizes should be zero and one. for myopic, code is finalized.
5. chapter 5c: first example. code is finalized.

(btw. some of these graphs are almost the same)

---

Stand-alone graphs:

• bayes net for the procrastinate problem. an image is here. there is an explanation of the relevant variables in Chapter 5a. on the diagram, actions should have quotes ("wait", "work") and utility variables (w and R-epsilon) should be italicized or in latex / equation style. the graph should go in the location of the handwritten version.
• bayes net for section introducing pomdps. you can just copy this but use lower case letters, subscripts instead of superscripts and U instead of R. (see the formal model section of chapter 3c. provide a brief explanation (two sentences) of the diagram.

Improve image in introduction

• It's pixelated
• The distance to the southern donut store is not noticeably shorter than the distance to the northern one

Add page for interactive editor with bundled packages

re-compile webppl-gridworld and nothing else

is it possible to re-compile webppl-gridworld and not all the other stuff (esp. webppl itself)? the aim is to make it fast to recompile if you only made a change to webppl-gridworld

Figure 1 in POMDP chapter missing the link from a to b'

You can see that b' depends on a by the equation immediately above Figure 1 which includes a.

Error when updating webppl

After pulling the latest version of agentmodels.org, when I run sudo sh ./scripts/update-webppl, I get the following error message which stops the update:

Warning: Command failed: browserify -t [./src/bundle.js --require ../webppl-timeit --require ../webppl-dp --require ../webppl-viz --require ../webppl-gridworld] -g brfs src/browser.js -o bundle/webppl.js
Error: Line 479: Unexpected token < (/Users/danielfilan/Documents/fhi_internship/agentmodels.org/node_modules/webppl-viz/src/index.js) while parsing file: /Users/danielfilan/Documents/fhi_internship/agentmodels.org/node_modules/webppl-viz/src/index.js
at Stream.end (/Users/danielfilan/Documents/fhi_internship/agentmodels.org/node_modules/webppl/node_modules/brfs/index.js:39:32)
at _end (/Users/danielfilan/Documents/fhi_internship/agentmodels.org/node_modules/webppl/node_modules/through/index.js:61:9)
at Stream.stream.end (/Users/danielfilan/Documents/fhi_internship/agentmodels.org/node_modules/webppl/node_modules/through/index.js:70:5)
at DestroyableTransform.onend (/usr/local/lib/node_modules/browserify/node_modules/readable-stream/lib/_stream_readable.js:545:10)
at DestroyableTransform.g (events.js:260:16)
at emitNone (events.js:72:20)
at DestroyableTransform.emit (events.js:166:7)
at endReadableNT (/usr/local/lib/node_modules/browserify/node_modules/readable-stream/lib/_stream_readable.js:960:12)
at nextTickCallbackWith2Args (node.js:441:9)
at process._tickCallback (node.js:355:17)
Use --force to continue.

Aborted due to warnings.

I'm not sure what the problem is: line 479 of node_modules/webppl-viz/src/index.js looks unremarkable to me.

Move generateAllCodeboxScripts.py to scripts/

Endnote-style citations

Example:

[Body]
Past work on IRL [1] has shown that...

[Bottom of page]
References:
1. Ng and Russell, Algorithms for...