Giter Site home page Giter Site logo

unsplit's Introduction

#unsplit - A framework for resolving Mnesia netsplits#

Copyright (c) 2010 Erlang Solutions Ltd.

Version: 0.5

Unsplit is a framework for registering merge functions that will be called any time mnesia tries to heal from netsplits.

The default behaviour of mnesia is not to attempt automatic merge after a 'partitioned network' event. It detects and reports the condition, but leaves it up to the user to resolve the problem.

Mnesia itself offers a few remedies: restart from backup, or elect to unconditionally reload tables from one or more nodes - in both cases data loss is very likely.

Unsplit starts a subscription on the 'partitioned network' event, and forces Mnesia to merge the "islands" that have been separated. It inserts itself into the schema merge transaction, claiming table locks on all affected tables. It then runs user-provided merge callbacks for each table, fetching data from one side, comparing the objects, and writing back the data that should be kept.

Writing an Unsplit method

Unsplit methods are table-specific, although a default method can be set using the unsplit application environment variable

{default_method, {Module, Function, ExtraArgs}}

The given method is called with the following arguments:

apply(Module, Function, [init, Table, Attributes | ExtraArgs]) -> ret()

to set up the merge. Then, data will be fetched using a given fetch strategy, and the fetched data will be handed to the merge function as:

apply(Module, Function, [data(), state()]) -> ret()

The return value, ret() is defined as:

ret() :: stop
       | {ok, state()}
       | {ok, actions(), state()}
       | {ok, actions(), strategy(), state()}

actions() :: [action()]

action() :: {write, Objects} | {delete, Objects}

strategy() :: all_keys | {ModS, FunS}

If a custom fetch strategy function is given, it will be called as

apply(ModS, FunS, [Table, RemoteNode, state()]) -> {ok, data(), state()}

Note that state() is whatever the merge function creates. The unsplit framework treats it as an opaque object, but the fetch function needs to do the same, or be aware of its definition.

The format of data() can be anything that the merge function accepts, but if the built-in strategy all_keys is used, it will have the format:

data() :: [{[object()], [object()]}]

where each 2-tuple represents the data matching a given key on each side of the split. Thus, [{[{mytab, 1, a}], []}] would mean that the object {mytab, 1, a} only exists on the side where the merge process is running, but is not found on the other side. [{[{mytab,2,a}], [{mytab,2,b}]}] would mean that conflicting versions of the object {mytab,2} were found.

##Modules##

unsplit
unsplit_lib
unsplit_reporter
unsplit_server
unsplit_vclock

unsplit's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

unsplit's Issues

Compilation error

After cloning, I simply run make but I got the following errors:
$ make
./rebar get-deps
==> edown (get-deps)
==> unsplit (get-deps)
./rebar update-deps
==> edown (update-deps)
==> unsplit (update-deps)
Updating edown from {git,"git://github.com/esl/edown.git","HEAD"}
./rebar compile
==> edown (compile)
src/edown_xmerl.erl:37: can't find include lib "xmerl/include/xmerl.hrl"
src/edown_doclet.erl:32: can't find include lib "edoc/include/edoc_doclet.hrl"
src/edown_xmerl.erl:92: record xmlElement undefined
src/edown_doclet.erl:46: can't find include lib "xmerl/include/xmerl.hrl"
src/edown_xmerl.erl:93: variable 'Attrs1' is unbound
src/edown_doclet.erl:256: undefined macro 'NO_APP'
src/edown_xmerl.erl:93: variable 'Parents1' is unbound
src/edown_xmerl.erl:138: record xmlElement undefined
src/edown_doclet.erl:108: record doclet_gen undefined
src/edown_xmerl.erl:163: record xmlAttribute undefined
src/edown_doclet.erl:114: record doclet_gen undefined
src/edown_xmerl.erl:164: record xmlAttribute undefined
src/edown_doclet.erl:115: record doclet_gen undefined
src/edown_xmerl.erl:165: variable 'HRef' is unbound
src/edown_doclet.erl:116: record doclet_gen undefined
src/edown_xmerl.erl:167: record xmlAttribute undefined
src/edown_xmerl.erl:168: record xmlAttribute undefined
src/edown_doclet.erl:117: record doclet_gen undefined
src/edown_xmerl.erl:178: record xmlAttribute undefined
src/edown_doclet.erl:118: record doclet_gen undefined
src/edown_xmerl.erl:178: record xmlAttribute undefined
src/edown_xmerl.erl:179: record xmlAttribute undefined
src/edown_doclet.erl:120: record doclet_toc undefined
src/edown_xmerl.erl:179: record xmlAttribute undefined
src/edown_doclet.erl:121: record doclet_toc undefined
src/edown_xmerl.erl:180: variable 'Alt' is unbound
src/edown_doclet.erl:124: record context undefined
src/edown_xmerl.erl:180: variable 'Src' is unbound
src/edown_doclet.erl:125: record context undefined
src/edown_xmerl.erl:234: record xmlAttribute undefined
src/edown_doclet.erl:126: record context undefined
src/edown_xmerl.erl:235: variable 'Lang1' is unbound
src/edown_doclet.erl:131: function title/2 undefined
src/edown_xmerl.erl:235: record xmlAttribute undefined
src/edown_doclet.erl:188: record xmlElement undefined
make: *** [compile] Error 1

I've installed erlang-edoc and erlang-xmerl packages but the problem persists :-/

Pluggable logging, instead of io:fwrite calls

Hello,

I'm planning to add a feature to 'unsplit' that allows me to plug in a logging function other than io:fwrite (e.g., lager or something custom), with the plugged-in function used throughout all unsplit modules.

The basic plan is to store a setting in the application's env identifying the logging function (a {Module, FunctionName} tuple). A table-specific setting specified in the {user_properties, ...} of a Mnesia table could be used additionally, but that's not something I would find useful for my purposes; if you or the user community thinks that has use, it's something I can tackle.

Calls to io:fwrite and error_logger:format would be replaced with calls to io_lib:format, along with passing the resulting string to whatever log function has been configured (with the default being io:fwrite). Log functions would be expected to have this type specification:

-type unsplit_log_fun() :: fun((string()) -> 'ok').

Pull request forthcoming, but wanted to begin with a heads up, along with my thanks for a very useful project!

  • Alex

crashing on bag table

unsplit_server line 254
[_] = A = mnesia:read({Tab,K}), % assert that A is non-empty
crashes if table is of type bag and has more than one record for key=K.

About inconsistency resolution

Hey @uwiger,

I am currently trying to get a grasp on mnesia and related techniques for inconsistency resolution between erlang nodes in a cluster after netsplits (I am OK with having eventual consistency). Mnesia is only used in RAM. What catched my attention about unsplit in particular are these lines of code which are with some adjustments also used in syn. Even though I don't understand these algorithms to their full extend as far as I can see it can also be used to generate a consistent state by simply using the state of the largest 'island' of connected nodes.

First I wanted to ask you about your experience with unsplit and the mentioned code from above in production systems. Also it would be nice to know if it is already possible to implement the above strategy of largest 'islands' with unsplit.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.