getguesstimate/guesstimate-app

Create Fermi Estimates and Perform Monte Carlo Estimates

Guesstimate is a tool for performing estimates using monte carlo experiments. It can be used similarly to excel, but provides the option of providing ranges and distributions as values instead of individual points. Other metrics can do mathematical operations on these cells/metrics. After each new input is added or changed, a set of 5000 samples is randomly generated from each input and goes through the specified operations to produce confidence intervals in the output.

Using Guesstimate

See GetGuesstimate.com to use.

Create a Collection

Create a Metric with a Guess

You can enter individual numbers or ranges.

• [25,35]: A 90% confidence interval between 25 and 35.

Currently all ranges are converted into normal distributions by default, but can be changed to uniform distributions.

Create a Metric with a Function input

Use the = sign as the first character of a guesstimate input to make it a function (similar to Excel and Google Sheets). Each metric has a variable name which is made of two randomly chosen letters. When the function input is selected, you can either type a metric's symbol to use it in a calculation, or simply click on it.

The functions are processed using Math.js. As such, it has several mathematical constants and functions that are useable.

Different Visualization Methods

Metric Visualizations

There are a few different ways of visualizing metrics. These are the following:

• Normal: A view with metric names, basic distribution stats (mean & std deviation), and a picture of the distribution in the background.
• Basic: The same as normal, but without the distribution image. This helps speed up calculations.
• Scientific: The same as normal, but the distribution image is larger and there are a few extra statistics. Mean and standard deviation are labeled with more detail.
• Debugging: This is used for making the system, and is a view of the React props that each metric component contains. This may be interesting to you, but is not that useful for general use.

Arrows

Arrows between metrics can be turned on or off. They can be useful, but also add complication to the UI. In the future the arrow layout may improve significantly. By default they show when there are less than 20 metrics in a model, and are hidden when there are more.

Distribution View

Click on Metrics to see more information about their distributions. The Distribution shows a large view of the Distribution, information about different percentiles, and a description.

Limitations

There are many things that will be improved. Some of the more important limitations include:

1. Distribution types are limited to normal, uniform, and a few discrete types with math.js functions
2. Everything is public.
3. Only the creator of a collection is allowed to save edits to that collection.
4. Metrics can't be shared between collections.

Technical Information

Guesstimate frontend is a [Next.js] Application.

There are two important third party systems. Authentication is done with Auth0, through next-auth, and search/indexing is done with Algolia. This means that running it yourself involves some setup, but on the other hand it's pretty easy set up compared to what you may need in comparable systems. Both of these are free for moderate use. I recommend both, though should note that the way I set up Algolia was a bit hacky (not sure how to do it right).

There is also a small server in Rails, which is not yet on Github. I'd like to work on security a bit more before putting it on here (if select people want to work on it I'd be happy to share in a small group). However, this is quite tiny; just 2 models (users, spaces).

All models are stored and saved as 'spaces'. The spreadsheet content is all stored as json in Postgres. You can see the full requests if you look at the network tab in chrome.

However, while there are a few third party systems, everything will still work with just guesstimate-app, as long as you don't try to log in or save. In practice this means that you can do quite a bit of development, as you can edit any model on the site (just can't save them). It also makes much development quite simple when it's just on the website (not the server).

You can point your local instance to production APIs by setting NEXT_PUBLIC_API_ENV: NEXT_PUBLIC_API_ENV=production pnpm dev.

How to run

First, make sure that git and node are installed.

git clone https://github.com/getguesstimate/guesstimate-app.git

cd guesstimate-app

pnpm install

pnpm dev

There are often errors with specific things, but it depends on what is already installed on the computer. Later we could put it in a docker container or something.

guesstimate-app's Issues

Support peta prefix

I'm dealing with lots of numbers in the peta range but it switches over from Tera to 10^15 when passing 1000 tera. Could we have it as peta instead? :)

Private sheets

Save Failed warning

A user reported that when there was no internet connection, no error was shown for a few minutes. When they got internet again, they still couldn't save; instead they had to refresh (losing their data).

Problems:

1. Bug: The error should show the moment something is attempted to be saved and fails.
2. Feature: When the user comes back online, it should sync automatically. Maybe there should be a 5s retry attempt on failure.

Move Graph Editor Styling into graph component directory

Right now all styles are in in main.styl. They should be broken out.

Cursor position gets lost in editor

1. Rename a node
2. Click the the beginning of the name
3. Try changing the text in the beginning. The cursor will automatically move to the end of the line.

Drag & Drop Metric Components

UI Feature additions

1.Add Title of the page to the editor page.
2.Add a description field while creating a new page.

Time as a unit

Estimates in my field, more often than not, are about figuring out when something will be completed. I'd like to see units of time supported.

'New Function' should render an estimate close to the 'New Function' button and be Split from the result

Right now the 'New Function' button renders both the function and the dependent node in the same place. The dependent node should be placed underneath the other node.

Display issues on Firefox

tl;dr
Unreadable description tag
No horizontal scrollbar
Display issues after scrolling horizontally with cursor keys

---

Using the public model Drake Equation as an example,
Upon entering the page:

The description (or at least, I assumed it was) tag is scrunched up on the left.
There was also no scrollbar to scroll left and right, so I had to select a cell and then press the left and right cursor keys to do so.

When I tried to return (after having scrolled to the right) by pressing left however, this happened:

I am unable to view the description and the header seems to have glitched. I was able to scroll all the way to the first cell though.

This was on Firefox 43.0.2.

P.S: Unrelated to the issue, I think it would help your app if you make it easier for users to provide feedback (e.g. "contact us", and/or "fork me on GitHub"). Good luck!

UI to toggle sheet viseability

Once a sheet is created, it should be possible to change its visibility. This is probably not a crucial addition (not sure if Github has it), but it seems useful, especially for people who have already created private models.

I may spend some time with it, but if it takes too long, would be ok launching private support without this feature.

Enter data as distribution

One 'distribution' type should be 'data'. When the user clicks this, they can input raw data.

https://twitter.com/britt/status/686696399644143620

All models sent & in memory

Right now the server returns all models (called spaces in the code) at once. This is now about 2.5mb. Super slow and unnecessary. Should be refined.

Sidebar is too far to the left.

uniform distributions have spike at left end-point

When I put in a distribution such as [0.05, 0.09] as uniform, There's a large spike in the visualization at the right-most bin.

Load & Save Spaces from Firebase

FAQ Page

Solve latency via histograms

The site has very high latency right now. Perhaps this is due to the sampling technique employed. You seem to have around 1000 samples, do a double for loop when combining estimates, then randomly pick a subset of 1000. Suppose instead you use histograms, each represented as an array of pairs (value, probability). Now the same double for loop, produces pairs (value, probability). You have n^2 values vs before, so choose a subset of size n that are nicely spaced. 100 should suffice, but maybe you can get away with 20. "Nicely spaced" is the hardest part of this algorithm. If you want the histogram bars to be equally spaced the math gets a bit trickier.

As a side note, I think histograms much more accurately preserve distributions, per bit of representation.

Get Mocha/Chai (or Jest) tests set up

Do one test. Then set up Travis.ci.

Running a local version of guesstimate (for dev work, or otherwise...)

I'm setting up a local copy so I can test changes, etc.

It might be helpful to have a short walkthrough of how to do this for potential contributors.

1. You need node.js
2. Install package dependencies; run npm install from the root directory for guesstimate
3. ...

Save button doesn't always work

To reproduce:

• Go to http://www.getguesstimate.com/
• Click on 'Foobar is pretty cool though'
• Move a node
• Press 'save'

Convert Models into JSON

I'm thinking it would be nice to see the 'JSON' form of models. This could be edited, where whatever is added would then be the new model.

This would allow for crude alterations. Also, by making a pure text format, it would allow interoperability with other tools. So another program could produce models that would feed into Guesstimate, for instance.

From a user request:
"how can I work with other users on a shared model? I want to work with [name hidden] on his [name hidden] model, if possible"

Display issues in Edge

Edge 25.10586.0.0

The editing surface does not expand the cells as expected, making them unusable:

'New Estimate' should render an estimate close to the 'New Estimate' button

Right now, when someone clicks 'New Estimate', the estimate renders somewhere in the middle or bottom of the page. It should instead be in the top left.

Distinguish 'output' metric

Lots of models have 1-3 'output' metrics. It has been recommended that we allow these to be distinguished. This is something I've been thinking about and think it would be pretty simple to implement.

https://medium.com/@CoryMogk/cool-tool-initial-feedback-ec5ac90ffa3c#.nzrly5k2p

Color backgrounds of cells

From a request,
"Just wanted to see if there was a way to color (fill or outline) cells? Would be great to have this feature to highlight different cells for inputs."

Naming conventions

The naming conventions are always pretty important, I wanted to lay out what we have and ask if there are any strong opinions on this.

Model: One guesstimate spreadsheet
Metric: The thing that fills a cell and has a value ('people in iowa')
Guesstimate: The thing that goes into the number field of a Metric. In the future there could be many guesstimates for one metric, so you'll see in the code base there's a one-to-many relationship.
Simulation: A Monte Carlo simulation. Stores around 5k samples and some metadata. Belongs to a guesstimate.

In the codebase 'Model' is often written as 'space', which is what it used to be called. I'll try to change this sooner than later, as I could see that getting confusing.

'My Models' starts undefined

The user can take a second to load, so in the beginning, the 'My Models' button doesn't do anything.

Use a fancy table for models on home page

Text and Functions should show in HTML, not SVG

Right now when a node is clicked, an html box comes on top of it. This looks really neat. The nodes don't look as neat, they are kind of blurry.

Converting this into HTML will mean having images that probably aren't clickable (the cytoscape element behind is clickable instead). This would mean moving them and modifying their heights on dom operations.

Auto-Resizing of Histograms

Generalizing Code for calling Uniform and Normal Distributions

In order to implement new distributions, it would be helpful to refactor and rewrite some of the code used for distributions.

• Include jStat Package
• Change distribution type to use 'param1', 'param2', etc. instead of 'mean', 'stdev' (uniform is also 2-parameter, it can use the same syntax and structure!)
• Also change validators to use new naming for parameters; only validator needed for normal; param2>0
• [x](Check what else needs to be changed to allow this?)
• Replace code for Uniform distribution with jStat.uniform.sample(a, b)
• Allow direct specification of the jStat distribution name, so that it can be generalized easily.
• Use this as a template for Normal, so that it can be extended further (to other 2 parameter distributions, at least.)

Possible sampling error

From a intercom request:

' There’s currently a bad bug though in how chains of calculations are dealt with. As it stands, in a chain of calculations X -> Y -> Z, it appears that random numbers are sampled for X to calculate distributions in Y, but then to calculate the distribution of Z the values of Y are resampled. The effect is that correlations are ignored entirely. What should happen is that random numbers are sampled from X and used to calculate the whole chain of values (Y and Z). Here is a quick proof of the error: http://getguesstimate.com/models/2218. For two data cells I’ve labelled A and B (e.g. X = {A,B}), there are separate cells for C=A/(A+B) and D=B/(A+B) (so Y = {C,D}), and finally a cell that is the sum of the two fractions E=C+D. This should be identically 1, because A/(A+B) + B/(A+B) = 1. Instead, an error range is found for E because the values for C and D are resampled ignoring their anti-correlation. Doing the entire computation in one step, shown in the cell labelled F, works correctly because the values sampled for X={A,B} are used directly in the computation of F=A/(A+B) + C/(C+D). '

Add read only/edit mode for pages

Non-Index Page Breaks in Production

To Reproduce:

• Click here
• Click repo/new
• Refresh the page.

It breaks in production, but works locally it seems.

support more distributions

Great project.

Please consider supporting more distributions. The types below are useful for 'guesstimate' simulations:
Exponential, Pareto, Generalized extreme value, Truncated Normal, skew normal, t-distribution, Gompertz.

sms sending disabled

when hangouts is activated it does not allow your cellular carrier to send a receive messages to go into settings and use messaging will resolve the problem.. the only other option is Hangouts.

Private models (Discussion)

Hi,

This is really great and I'd like to start using it for all sort sort of things. However, it can be difficult to use it if models are all public (unless I'm missing something).

Ideally, we'd have SSL (you can use https://letsencrypt.org/ for free certs) and a flag to make models public or private. Some kind of sharing system ("let use X see this model") would be nice, but not critical.

Alternatively, I was going to self-host but this doesn't really seem possible at the moment because the Rails app isn't yet published (by the way: publish it; until you do it's pretty hard to get more contributors working on the app, and even if it's not perfect, others will help you make it better), and even in "remote API" mode I can't get it to build on my Mac (npm install fails with "No compatible version found: lodash._charsendindex@'>=3.0.0 <4.0.0'").

I'm going to keep trying to make it work locally and see if I can help implement private models, but it's a bit "catch 22" at the moment.

Thanks for doing this and thanks for sharing. It's a really impressive thing you've built!

Upgrade to react-bootstrap 0.25

https://www.npmjs.com/package/react-bootstrap

Get better router

Right now refreshing does not work

Bootstrap Navigation Dropdown with Repo List

As a user, I should see a bootstrap dropdown list of my models. See here:
http://react-bootstrap.github.io/components.html

This should go on the top left, next to the + button.

Estimates with variable certainty

A few people have requested being able to declare their 99% range or similar, instead of their 90% interval.

I'm thinking this should be implemented by a more detailed 'distribution editor' later on.

Saving Records / Undo / Checkpoints

It would be great to restore to previous versions. A user recommended we use a conflict-free replicated data type (CRDT), perhaps with swarm.

A few history-related features that would be useful:

• A list of what changed recently, similar to Trello
• Undo
• Manual snapshots
• Save and restore from JSON. (import & export)
• Saving snapshots automatically on the server
• Collaboration

Favicon

The homepage icon should be a favicon. I think this will be done in the webpack config.

Missing dependencies

$ cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)

running with latest d950de4

ran npm install, installed a bunch of stuff, exited cleanly running npm run start:

Error: Cannot find module 'lodash.defaults'

so npm install lodash.defaults, repeat for autoprefixer

these should probably be added as dependencies

Top Banner gets mis-sized when I select a cell in chrome

When I select a cell in chrome the top banner changes height and makes the interface seem a bit broken.

I've also noticed that it doesn't load at all when I try to use it in IE11.

SVG Arrows

App fails to start, "weird error 8"

I just checked out the app from git following the instructions from README.md, tried starting it using "npm run start" but it won't start. I'm not skilled enough to solve this problem myself, so if it's a stupid user error, ignore it. From the log on the command line:

$ npm run start

> [email protected] start /home/alex/Projects/guesstimate-app
> BUILD_DEV=0 NODE_ENV=development API_ENV=production webpack-dev-server


/home/alex/Projects/guesstimate-app/webpack.config.js:5
var elev = `<script> var _elev = window._elev || {};(function() {
           ^
SyntaxError: Unexpected token ILLEGAL
    at Module._compile (module.js:439:25)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at module.exports (/home/alex/Projects/guesstimate-app/node_modules/webpack/bin/convert-argv.js:80:13)
    at Object.<anonymous> (/home/alex/Projects/guesstimate-app/node_modules/webpack-dev-server/bin/webpack-dev-server.js:55:48)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
npm ERR! weird error 8
npm ERR! not ok code 0

Multiple unreadable models on index page

Two solutions for now:

1. Clicking the 'create model' link multiple times shouldn't result in multiple models.
2. We should filter out models with less than 1 metric from the main page.

hacking information

Some light information on how to set up a dev (or outward facing) deployment would be great!