basho/basho_docs

Basho Products Documentation

basho_docs's Issues

Changing the -name option

The documentation doesn't mention what will occur if a user starts a node, stops it, modifies the value of the -name option in vm.args, and then starts the node again.

There should be a note for relevant content advising:

1. This should not be done
2. If it is a devrel or similar, advise recovery is possible by simply removing the ring data.
3. If removing the ring data is not feasible, the user must riak-admin reip or (preferably) riak-admin cluster replace the nodes to have the new -name values honored and not error on node startup.

Target documentation pages:

• http://docs.basho.com/riak/1.2.1/cookbooks/Basic-Cluster-Setup/
• http://docs.basho.com/riak/latest/tutorials/fast-track/Building-a-Development-Environment/

Make Vector Clock changes much more explicit

In Riak 1.0, client-id header is basically ignored. We don't cover it very well in the docs. Should be very plain and explicit.

Talked about here: basho-labs/little_riak_book#8

Riak version finding on SmartOS

@jaredmorrow put together a script for finding which Riak version you're running on SmartOS. This should live somewhere on the docs.

https://gist.github.com/jaredmorrow/4999933

Add logstash, new PHP lib, others to community projects page

The community projects page is woefully out of date. This issue is a stub to add things, starting with the following:

• http://logstash.net/docs/1.1.5/outputs/riak
• https://npmjs.org/package/chinood
• https://npmjs.org/package/simpleriak
• https://github.com/marksteele/riak-php
• https://gist.github.com/4064937
• https://gist.github.com/4064990

Missing word in tutorials/querying/Secondary-Indexes

"Indexes reside across multiple machines. A covering a set of vnodes must be queried and the results merged."

Warn about qfun's fragility

The Erlang API section of http://docs.basho.com/riak/latest/references/appendices/MapReduce-Implementation/ explains the different options for specifying the function to be run in a map or reduce phase. Warnings should be placed around the qfun form to inform users about the fragility of using it.

The warning should address three concerns:

1. The module in which the function is defined must be present and exactly the same version on both the client and Riak nodes.
2. Any modules and functions used by this function (or any function in the resulting call stack) must also be present on the Riak nodes.
3. Errors about failures to ensure both 1 and 2 are often surprising, usually seen as opaque missing-function or function-clause errors. Especially in the case of differing module versions, this can be difficult to diagnose without expecting the issue and knowing of Module:info/0.

More information needed

So a couple of things:

• There should really be more (and structured) use cases for Riak on the wiki page, explaining common data modelling scenarios (especially including 2i for 'relations')
• Next to list_keys() and list_buckets() there should be more information about alternative approaches or links to more information if the use case really requires a listing of all keys or buckets.

Add info about Riak being enabled in openbsd ports tree

Looks like Riak 1.3 is in the openbsd ports tree (h/t to David Gwynne for the heads up).

http://marc.info/?l=openbsd-ports-cvs&m=136427560732173&w=2
http://marc.info/?l=openbsd-ports-cvs&m=136427572532189&w=2

While we don't officially support these packages, we should have some indication of how they exist in the docs.

Source link on each doc page to facilitate community fixes

In order to facilitate more rapid fixes for doc issues from the community it would be useful to have a link from each doc page back to its source here.

This would substantially lower the barrier between noticing an issue and directly fixing it.

This is an awesome way to do documentation.

"Installation on Closed Networks" not complete for Riak version 1.3.0

Not only "eleveldb" dependency but also "lager" dependency is tried to be pulled from github. This fails on closed networks and there is no workaround described in the docs.

Improve/Streamline the Fast Track

The Fast Track needs a lot of work, not the least of which is that. Some thoughts:

1. separate user/operator tracks
2. remove the link walking portion entirely
3. mapreduce needs simplified
4. examples could use code snippets rather than the HTTP interface
5. a shorter "Get Started in 5 Minutes" guide, perhaps the very first portion of the fast track

Add docs on Riak + SimpleSamlPHP to contrib section

Just a place holder so I don't forget too add this. David Gwynne and some hackers down at Queensland University have this in prod.

http://simplesamlphp.org/docs/1.10/riak:simplesamlphp-riak

HIPE notes for OSS version build instructions

We should note that HIPE works fine on linux, but causes some issues on FreeBSD and Solarisen, and so should be disabled on your erlang version for those platforms.

Installing Memory Backend Page Missing Critical Line

So the section on installing the memory backend[1] doesn't actually mention the one line of configuration you need to alter.

This section should mention that you need to set storage_backend to riak_kv_memory_backend in the riak_kv section.

http://docs.basho.com/riak/1.2.0/tutorials/choosing-a-backend/Memory/#Installing-the-Memory-Backend

Simplify Community section

Right now we have a community nav link, which splits into "resources" and everything else. We don't make very clear what exactly a "resource" is. I'd love to be able to combine some of these pages to reduce the total count, making it easier to navigate our community support.

Some of this information can be inlined into other sections. For example, we have "Start Here" that contains introductory material, then "Introductions" under Community. Same with "Developing with Riak". It may be topologically cleaner to fold those community resources into the core introduction narrative.

I originally followed the wiki layout when it came to community resources, only because they were being changed while I was trying to build a sitemap. But I'm still interested in cleaning this whole section up.

Mac OS instructions should give caveats for MacPorts Erlang.

I've not tried Homebrew, but MacPorts will only install the most recent version of Erlang. Your docs specifically say that the most recent Erlang can't be used.

So your Mac OS build instructions should really specify "kerl".

I've used "kerl" here and It seems to have done something, though I'm running into lots of other problems with your documentation so haven't actually been able to build anything.

Concepts need some love

Over the years things have changed in Riak. More importantly, we have collectively learned some things that render some of our existing concepts to be somewhat misinformed.

Go through the Concepts section with a fine-toothed comb and ensure that everything we say is accurate

http://docs.basho.com/riak/latest/references/appendices/concepts/

For example: "R + W > N ensures strong consistency in a cluster," under "Eventual Consistency" is incorrect.

Many claims we make with W are more appropriately made with DW or PW.

Moreover, simply reducing the wordiness of some of these concepts would go a long way to ensuring we stay up-to-date with the latest research.

When trying to install middleman for the docs...

I keep running into various issues. I'm putting together a blog entry based around what I'm doing to resolve it, but this is the part that's holding me up right now. It's based on breakage installing blockenspiel for the middleman requirement, a native component doesn't have the executable building.

Here's the issue I've created directly @ the blockspeil repo.

dazuma/blockenspiel#7

Document Riak Search statistics

We need notes added to the appropriate statistics output pages which document the various Riak Search statistics we have now.

Start here:

• Inspecting a Node
• HTTP Status

Neglected 404 from old docs need a little redirection

Looks like you can still get to http://wiki.basho.com/The-Riak-Community.html (which 404's) from the "Developer" drop down on wiki.basho.com. We need to add this to the redirects.

Document steps to rename a node on a running cluster

The http://docs.basho.com/riak/latest/references/Command-Line-Tools---riak-admin/ under the reip and cluster-replace sections should explain the steps to change a nodename safely on a running cluster.

Steps are:

1. Stop the node
2. Change vm.args to put in the new node name
3. Rename or remove the data/ring directory
4. Mark the old nodename as down from any running node. riak-admin down
5. Start the node
6. Once the node is started, have it join the existing cluster
7. Do not commit the cluster change
8. Force-replace
9. Confirm correct cluster topology using riak-admin cluster plan
10. Cluster commit to commit the changes

Documentation doesn't mention the R15B03 version of Erlang

The documentation, e.g. the Fast Track Tutorial says explicitly that the R15B02 version of Erlang shouldn't be used, but it doesn't mention the R15B03 version.

This is highly confusing for beginners (like myself), who end up assuming that the documentation is simply out of date, and therefore try using R15B03 (which causes crashes when you start riak, at least for me).

Package version for manual deb download is stale

On http://docs.basho.com/riak/latest/tutorials/installation/Installing-on-Debian-and-Ubuntu/ , 1.2.0 needs to be 1.2.1.

Or, just finish #24

"Fast Track" is very slow.

http://docs.basho.com/riak/latest/tutorials/fast-track/

claims

"From start to finish, this will probably take you around 45 minutes."

So far, this has been pretty frustrating.

Apart from the errors in the instructions, it's not clear to me why I can't use the precompiled version. Would it be possible to include enough of a Makefile to support "make devrel" in the precompiled release?

For reference, I'm trying to run Riak on Mac OS for experimentation. Eventual deployment would be on Linux.

bucket property description

sites http://wiki.basho.com/Buckets.html or http://wiki.basho.com/Configuration-Files.html mention basic read/write properties but miss explanation

I would also change w description so it clearly states that vnode doesn't have to save object to persistent storage, it suffices to just receive request (below is my proposition)

rw - quorum for both operations (get and put) involved in deleting an object (default is set at the bucket level)
r - (read quorum) how many replicas need to agree when retrieving the object
w - (write quorum) how many vnodes must confirm reviving write request before returning a successful response
dw - (durable write quorum) how many replicas to commit to durable storage before returning a successful response
pr - (primary read quorum) works like r but requires that the nodes read from are not fallback nodes
pw - (primary write quorum) how many replicas to commit to primary nodes before returning a successful response

I ve found above on http://wiki.basho.com/HTTP-Delete-Object.html which is weird place to look for this.

plus I would like to add more explanation what exactly is primary node

Adding riak apt-repository has wrong command:

http://docs.basho.com/riak/latest/tutorials/installation/Installing-on-Debian-and-Ubuntu/#Installing-From-Apt-Get

I think there was a copy and paste error from my original instructions:

https://github.com/basho/internal_wiki/wiki/Riak-Package-Repositories

Currently it is:

sudo bash -c "echo deb http://apt.basho.com lsb_release -sc main > /etc/apt/sources.list.d/basho.list"

as per my doc it should be:

sudo bash -c "echo deb http://apt.basho.com `lsb_release -sc` main > /etc/apt/sources.list.d/basho.list"

but in retrospect I think the cleanest way should be:

sudo bash -c "echo deb http://apt.basho.com $(lsb_release -sc) main > /etc/apt/sources.list.d/basho.list"

Move Riak CS project wiki docs to public docs

Riak CS still contains important documents in it's github wiki pages. To avoid two sets of docs getting out of sync, I propose moving all project wiki information into basho docs, and removing the user-focused wiki pages entirely (to avoid both version skew and clarify where all information lives).

Riak CS Wiki

Addition to Default MDC documentation to disable fullsync

http://docs.basho.com/riakee/latest/cookbooks/Multi-Data-Center-Replication-Configuration/#Settings

Indicate fullsync_interval = disabled and fullsync_on_connect = false are both required to fully disable fullsync.

Currently, the Description for fullsync_interval states "To disable full synchronization, use: disabled". However, a fullsync will still occur when a new cluster is connected IF fullsync_on_connect is set to true.

More emphasis on PW parameter.

I feel like even seasoned users of Riak might not quite understand partitioning and how it can affect their data.

Particularly of interest for me is one of the ways you can avoid concurrent writes to a partition with a bunch of fallback nodes[1]: use pw=QUORUM.

This is very useful but as far as I can see PW is only just basically defined in the docs and not why pw might be useful. (And for example, take a look here: http://docs.basho.com/riak/latest/references/appendices/concepts/Riak-Glossary/#Quorum where pw is not even mentioned.)

1. that is, say you have a single node partition, which would have fallback vnodes representing the rest of the cluster

Broken links on Dynamo Paper page

While on this page: http://docs.basho.com/riak/latest/references/dynamo/ links to glossary items are broken. For example while the consistent hashing entry is here:

http://docs.basho.com/riak/latest/references/appendices/concepts/Riak-Glossary/#Consistent-Hashing

the current link on the dynamo paper points to:

http://docs.basho.com/references/appendices/concepts/Riak-Glossary/#RiakGlossary-ConsistentHashing

lsb_release inline command to add Basho repository on Debian fails

Inline reference to lsb_release -sc needs to be $(lsb_release -sc).

Add note for Retrieving a List of All Users section of RiakCS

On the RiakCS Account Management page, in the 'Retrieving a List of All Users' section there should be some sort of note indicating that @endpoints variable in s3curl.pl will need to be edited to include the hostname that you're trying to get the users list from. That particular change is not intuitive and has caused a couple people a few hours of headache.

document typo?

does't -> doesn't

$ git diff
diff --git a/source/riak/tutorials/choosing-a-backend/Bitcask.md b/source/riak/tutorials/choosing-a-backend/Bitcask.md
index 9208f60..c3f2ec1 100644
--- a/source/riak/tutorials/choosing-a-backend/Bitcask.md
+++ b/source/riak/tutorials/choosing-a-backend/Bitcask.md
@@ -144,7 +144,7 @@ in your [[app.config|Configuration Files]].
 ]}

-<div class="note"><div class="title">Bitcask does't actually set O_SYNC on
+<div class="note"><div class="title">Bitcask doesn't actually set O_SYNC on
 Linux</div><p>At the time of this writing, due to an unresolved Linux <a
 href="http://permalink.gmane.org/gmane.linux.kernel/1123952">kernel issue</a>
 related to the <a

HTTP route in examples should be /buckets/BUCKET/keys/KEY instead of /riak/BUCKET/KEY

http://docs.basho.com/riak/latest/references/apis/http/HTTP-Fetch-Object/ vs http://docs.basho.com/riak/latest/tutorials/querying/Basic-Operations/
trainees are getting confused /riak/* vs /buckets/*

"Installing Riak from Source" broken because tarball isn't a git checkout.

http://docs.basho.com/riak/latest/tutorials/installation/Installing-Riak-from-Source/

Has a pretty basic problem. The source tarball does not have a .git directory, so it's not a git working copy.

$ cd riak-1.3.0
$ make rel
fatal: Not a git repository (or any of the parent directories): .git
./rebar get-deps
env: escript: No such file or directory

The instructions should either be changed to specify cloning the source from github or "make rel" should be fixed or the tarball should be updated to include a .git file.

Syslog is now supported

http://docs.basho.com/riak/latest/cookbooks/faqs/logs-faq/#can-i-make-riak-use-syslog-instead-of-file

Restructure interaction between m/r and tombstones

Per this thread, you need to plan for tombstones in your map reduce jobs. We don't have any docs explaining why this is or how to do it.

listing riak-cs uses requires additional step

riakcs/cookbooks/Account-Management/#Retrieving-a-List-of-All-Users:

Listing riak-cs users requires that s3curl.pl be modified to add the custom endpoint:

my @endpoints = ('my.riakcs.com',
                  's3.amazonaws.com',
                  's3-us-west-1.amazonaws.com',
                  's3-us-west-2.amazonaws.com',
                  's3-us-gov-west-1.amazonaws.com',
                  's3-eu-west-1.amazonaws.com',
                  's3-ap-southeast-1.amazonaws.com',
                  's3-ap-northeast-1.amazonaws.com',
                  's3-sa-east-1.amazonaws.com', );

Update Community Section w/ All Riak Meetups

Send link to me when done for website update.

Re-check bitcask capacity planning doc math

We should recheck the bitcask capacity planning doc math.

Riaknostic now included by default

This fact needs to be reflected in the URL below.

http://docs.basho.com/riak/latest/references/appendices/Inspecting-a-Node/#Riaknostic

A Short Guide to Writing Guides is not public?

I tried to get to https://github.com/basho/internal_wiki/wiki/A-Short-Guide-to-Writing-Guides from the https://github.com/basho/basho_docs readme and I get a 404. Not sure if it is private and should be public or? thanks, kjm

Bitcask merge window setting clarifications.

@tisba from IRC notes that merge windows set in multi-backend don't work. We should note that must be set in the global bitcask section of the app.config file and clarify how setting precedence works when using multi-backend.

From Joe:

re: bitcask changeable options
when a bitcask backend is started, it is passed (Dir, Opts)
whenever it needs to obtain the value of any option, it first checks its Opts structure - which is maintained per backend, and if not found there then get_env(bitcask
when running in a multi-backend config the entire block of options listed under multi-backed is passed in as Opts
so only options not set in the multi-backend subset can be changed on the fly
with the exception of max_file_size and expiry_secs - they are explicitly copied from global to local during startup

there is only 1 merge worker for the entire node
so it can't use per-backend settings for merge window
the multi-backend settings are passed to the bitcask backend when it starts
but merging is controlled from an app-level worker, not from the backend
any merge_window setting below the multi-backend bit is probably ignored

I think the problem here is that a lot of settings were lumped under
"bitcask" before multi-backend existed
which made sense in that context
but the merge worker doesn't have access to the private vars of individual backend instance
It looks like when each vnode starts it starts the bitcask app if it isn't already
and when the bitcask app starts, it starts a merge_worker
the only get_env I see in the merge_worker code is the one for merge_window
perhaps we should suggest making bitcask_merge_window a top-level option to disambiguate
so then we can say "anything you can set in the bitcask section can be set in a multi-backend bitcask"

OSX install instructions specify wrong package to untar after download

http://docs.basho.com/riak/latest/tutorials/installation/Installing-on-Mac-OS-X/

Both the 64 and 32 bit install instructions ask you to untar 1.2.1 after you've just downloaded 1.3.0

Add FAQ entry about m/r returning JSON on occasion

As talked about here on the mailing list.

Add link to Basho Docs GH repo

I 'm having trouble finding a link to the Basho Docs repo on GH. Unless I'm blind with RICON prep madness and am missing something obvious, we need to make this more apparent.

Monitoring manual

Would be useful to have a page with a run down of what can indicate a problem with a Riak node/cluster (disk space, some internal riak parameters (what they mean if not obvious), etc.)

Sane default thresholds for parameters would be nice. :-)

Something similar to the following:
http://docs.mongodb.org/manual/administration/monitoring/

custom error page

Currently custom error page for 404 is not activated though it is in the repository.

http://docs.basho.com/blahblahblah

Craft Lager Documentation

Currently, valid Lager settings are scattered all over God's Green Earth.

• https://github.com/basho/lager/blob/master/README.org
• https://github.com/basho/lager_syslog/blob/master/README.org
• https://github.com/jbrisbin/lager_amqp_backend
• https://github.com/kivra/lager_loggly
• https://github.com/blinkov/lager_smtp

This might take a fair bit of work. A possible avenue of attack is:

1. Start with a bullet list of all options in Configuration-Files.md
2. Create a document, possibly starting with Lager Readme, pulling in all resources.
3. Fill out deeper details and links in Configuration-Files.md
4. Profit

Clean up/Expand Backend docs

Right now, the backend reference/cookbooks are conflated with the tutorials on how to choose a backend. These serve different purposes and should be separated.

We should keep and simplify the "choosing a backend" tutorials, then create a more detailed set of docs which cover Configuring, Tuning, Recovery and possibly other deeper analyses, like Leveldb's "implementation details" section.