marianoguerra/tanodb

distributed in memory key value store on top of riak_core, example for the little riak core book

Mariano Guerra's Index Page

Hi! I'm Mariano Guerra, an Information's System Engineer from Córdoba, Argentina

I'm the Co-founder of Instadeq a product to do data analysis, visualization and collaboration for individuals and teams and GlooData an Extensible Conversational UI for Interactive Components

I'm writing the book WebAssembly from the Ground Up with Patrick Dubroy

Previously Co-founder of Event Fabric

I do a Weekly Newsletter about the Future of Coding

Main Projects

(most recent first)

• WebAssembly From The Ground Up: Book about compiling to WebAssembly with vanilla Javascript and ohm.js
• Data The Ultimate: spawnfest project. DTU is a data format that, unlike YAML, allows to define pretty decent programming language
• Youtube Notes: Take notes from Youtube Videos without leaving your keyboard
• Review Jam / Advent of Future of Code: One task per day to be solved with a Future of Coding Project
• Creemos en la Red: Teach web development to underrepresented groups in Argentina
• BEAMoji: a transpiler from Erlang to an Erlang dialect that uses emojis for identifiers
• Elixir Flavoured Lisp Flavoured Erlang: a transpiler from Lisp Flavoured Erlang to Elixir
• Elixir Flavoured Erlang: a transpiler from Erlang to Elixir
• Efene Programming Language: A Python/JS inspired dialect of Erlang
• Creemos en la Web: Teach web development to underrepresented groups in Córdoba, Argentina
• Riak Core Lite (Contributor): A framework to build dynamo-style distributed systems
• Riak Core on Partisan on Elixir Tutorial: Tutorial on how to build a distributed key-value store in Elixir using Riak Core on top or Partisan
• Riak Core Tutorial: Tutorial on how to build a distributed key-value store in Erlang using Riak Core
• Little Riak Core Book: Small book describing Riak Core and how to build a distributed key-value store in Erlang using Riak Core
• Founders in Germany: List of English resources for international founders in Germany
• Interfix Programming Language: a language for the Erlang VM with an experimental syntax
• json.human.js: render JSON in a human readable way
• rst2html5: generate HTML5 from ReStructuredText
• All Projects

Me Around the Web

• gh/marianoguerra
• @marianoguerra at hachyderm.io
• @warianoguerra
• marianoguerra.org

Talks

(most recent first)

• Outlining a UI for Tremor - Tremor Con 2022
• WebAssembly Demysteified - Heilbronn Nerd BBQ October 2022
• Fireside chat on BEAM Languages with Erlang co inventor Robert Virding & Mariano Guerra - Code BEAM America 2021
• Seeing Like a Linguist - Code BEAM Coruña 2022 (Slides, Video)
• Instadeq: A Live Programming Environment for End User Data Analysis and Visualization - Live 2021 (Slides)
• The Platform of the System - Code BEAM Brasil 2021 (Slides)
• Elixir Flavoured Erlang - Erlang User Group Meetup (Slides)
• BEAM Language Ecosystem: Past, Present and Future - Code BEAM Brasil 2020 (Slides)
• Programming by any other name - Bob 2020 - Berlin (Slides)
• Un Paseo por el Laberinto de las Representaciones - ElixirConfLA Medellin 2019 (Slides)
• Slides: A Tour Through the Distributed System Zoo - Code BEAM Milan 2018
• Video: Implementing Languages on the BEAM - Code BEAM SF 2018
• Video: A Tour Through the Distributed System Zoo - Code BEAM SF 2018 (Slides)
• Video: Building distributed applications: riak_core vs partisan | London Riak Meetup (Slides)
• Sistemas Distribuidos: ¿Para Qué? - EFLBA2017
• The BEAM community and efene
• Gossip protocols, Epidemic Broadcast and Eventual Consistency in practice - Erlang Factory Lite 2016
• How BEAM Languages are Made
  • Basel Erlang Meetup 2016
  • Karlsruhe Elixir Meetup 2016
  • BEAM Buenos Aires Meetup 2016
• From 0 to a Working Distributed System With riak_core - Ricon San Francisco 2015
• Riak Core in Small Bites - Berlin Erlang Factory Lite 2014
• Hagamos un lenguaje de programación con pypy por Mariano Guerra - PyConAr 2013
• Efene Programming Language - Brussels Erlang Factory Lite 2012
• Efene a programming language for the erlang VM - London Erlang Factory 2010

Interviews & Podcasts

• My NoCode Story: #053: NoCode Dashboards w/Mariano Guerra 2022
• "Aha! Moment" Demo Video with Instadeq - No Code Ops 2021
• Episode 86: EMx 085: Riak Core and Partisan with Mariano Guerra - Elixir Mix 2019
• Efene with Mariano Guerra - Episode 47 - Python Podcast 2016
• Mariano Guerra on the Efene and Erlang VM as Language Platform - InfoQ 2010

Language Projects

• Firestore Query Parser
• HTML Is A Programming Language
• Efene Programming Language
• Elixir Flavoured Erlang: an Erlang to Elixir Transpiler
• BEAMoji
• Elixir Flavoured Lisp Flavoured Erlang
• OTP.ex: Erlang/OTP transpiled to Elixir using Elixir Flavoured Erlang
• Interfix Programming Language
• OTL: One true language
• Forth implemented in C, JS, WebAssembly and compiled from C to asm.js and WebAssembly
• Sonic Blocks: make music with blocks and Sonic PI
• Band Blocks: make music with blocks and band.js
• plang: scheme like language implemented with PyPy
• squim: a scheme/kernel language implemented in JS

Academic-ish Stuff

• SPLASH's LIVE 2021 Accepted Submission: Instadeq A Live Programming Environment for End User Data Analysis and Visualization
• ACM SIGPLAN International Workshop on Erlang 2021: Program Committee Member
• Google Summer of Code 2020: Mentor for Reference implementation for riak-core-lite based distributed application

Parser Projects

• transito: edn parser
• erldn: erlang edn parser
• qrly: jquery-like selectors for HTML/XML in Erlang
• legojs: build HTML with CSS-like selectors in JS
• emel: build HTML with CSS-like selectors in Erlang
• fnt: jquery-templates in Erlang

Deprecated Projects

• Emesene Instant Messenger (MSN Network Closed)

tanodb's Issues

Listing Keys from a Bucket no longer works

Due to a change in what riak_core expects, the instructions for the chapter 'Listing Keys from a Bucket' no longer works because of the following causing a badmatch:

tanodb/apps/tanodb/src/tanodb_coverage_fsm.erl

Lines 27 to 29 in 3fba494

	init(_, [ReqId, From, Request, Timeout]) ->
	State = #state{req_id=ReqId, from=From, request=Request},
	{Request, allup, 1, 1, tanodb, tanodb_vnode_master, Timeout, State}.

Changing it to this fixes it:

{Request, allup, 1, 1, tanodb, tanodb_vnode_master, Timeout, riak_core_coverage_plan, State}.

Season II planned?

Hi

Thanks for this tanodb.

I wonder if there could be some following up steps, such as:

• eleveldb/erocksdb as backend
• crdt
• more riak_core related logic

rebar3 release - fails

rebar3 release gives me:

===> release successfully created!
===> Unable to copy from /Users/thanos/work/tanodb/_build/default/rel/tanodb/releases/0.1.0/vm.args to /Users/thanos/work/tanodb/_build/default/rel/tanodb/etc/vm.args because of {copy_failed,
                                                                                                                                                                              enoent}

/Users/thanos/work/tanodb/_build/default/rel/tanodb/releases/0.1.0/vm.args was not created:

Admins-MacBook-Air:tanodb thanos$  ls -lR /Users/thanos/work/tanodb/_build/default/rel/tanodb/releases/
total 16
drwxr-xr-x  8 thanos  staff   272 Apr  9 11:44 0.1.0
-rw-r--r--  1 thanos  staff  1929 Apr  9 11:44 RELEASES
-rw-r--r--  1 thanos  staff     9 Apr  9 11:44 start_erl.data

/Users/thanos/work/tanodb/_build/default/rel/tanodb/releases//0.1.0:
total 208
-rw-r--r--  1 thanos  staff   5332 Apr  9 11:44 start_clean.boot
lrwxr-xr-x  1 thanos  staff     43 Apr  9 11:41 sys.config.orig -> /Users/thanos/work/tanodb/config/sys.config
-rw-r--r--  1 thanos  staff  35881 Apr  9 11:44 tanodb.boot
-rw-r--r--  1 thanos  staff   1041 Apr  9 11:44 tanodb.rel
-rw-r--r--  1 thanos  staff  46874 Apr  9 11:44 tanodb.script
lrwxr-xr-x  1 thanos  staff     40 Apr  9 11:41 vm.args.orig -> /Users/thanos/work/tanodb/config/vm.args

couldn't find 'nodename' in ../tanodb_config/tanodb.conf

I met following problem when I try to do rebar3 run

:~/tanodb/tanodb (ping-as-a-service-fix-1)$ rebar3 run
===> Verifying dependencies...
===> Compiling tanodb
===> Running cuttlefish schema generator
===> Starting relx build process ...
===> Resolving OTP Applications from directories:
/home/id/tanodb/tanodb/_build/default/lib
/home/id/tanodb/tanodb/apps
/usr/local/lib/erlang/lib
/home/id/tanodb/tanodb/_build/default/rel
===> Resolved tanodb-0.1.0
===> Dev mode enabled, release will be symlinked
===> release successfully created!
vm.args needs to have a -name parameter.
-sname is not supported.
couldn't find 'nodename' in ../tanodb_config/tanodb.conf
$PWD is /home/id/tanodb/tanodb

I have tried follow "Little Riak Core Book" to install through template, but I still got above problem.
Then I use commit "ping-as-a-service-fix-1", problem is same
I also checked file, it is there.
~/tanodb/tanodb/config (ping-as-a-service-fix-1)$ ll vm.args

My rebar3 version is
~$rebar3 -v
rebar 3.2.0 on Erlang/OTP 18 Erts 7.0

Anyone met this before?
Thanks a lot!

riak_core_util not being added to the Erlang code path

Hi there,

I suspect that the Erlang code path is not being set up correctly. An indication could be the fact that I receive the following error message when attempting to execute tanodb:put("foo","bar")..

 no function clause matching riak_core_util:chash_key("foo")

But if I perform a search, the beam file is on the path. Checking the source confirms that the function is defined.

[/common/tanodb%]find ./ | grep riak_core_util
.//_build/default/lib/riak_core/ebin/riak_core_util.beam
.//_build/default/lib/riak_core/src/riak_core_util.erl

The full error message was as follows:

(tanodb3@127.0.0.1)6> tanodb:put("foo","bar").
22:20:12.505 [error] gen_fsm <0.1365.0> in state prepare terminated with reason: no function clause matching riak_core_util:chash_key("foo") line 247
22:20:12.506 [error] CRASH REPORT Process <0.1365.0> with 0 neighbours exited with reason: no function clause matching riak_core_util:chash_key("foo") line 247 in gen_fsm:terminate/7 line 626
22:20:12.506 [error] Supervisor tanodb_write_fsm_sup had child undefined started with {tanodb_write_fsm,start_link,undefined} at <0.1365.0> exit with reason no function clause matching riak_core_util:chash_key("foo") line 247 in context child_terminated
{error,timeout}
(tanodb3@127.0.0.1)7>

I can dynamically add the code path using code:add_patha/1 but I'd like to actually fix the problem. I don't have any prior knowledge of 'rebar3' and I'm unfamiliar with the build system.

If you could send me a pointer, I'll submit a P/R to fix the issue.

B

Ping as a Service (PaaS) - rebar3 run fails

Under the section

Ping as a Service (PaaS)

and subsection

Testing it

trying to do

rebar3 run

fails:

===> release successfully created!
Exec: /Users/thanos/erlangs/18.1/erts-7.1/bin/erlexec -boot /Users/thanos/work/tanodb/_build/default/rel/tanodb/releases/0.1.0/quantumdb -boot_var ERTS_LIB_DIR /Users/thanos/erlangs/18.1/erts-7.1/../lib  -mode embedded  -config /Users/thanos/work/tanodb/_build/default/rel/tanodb/generated.conf/app.2016.04.19.22.07.13.config -args_file /Users/thanos/work/tanodb/_build/default/rel/tanodb/generated.conf/vm.2016.04.19.22.07.13.args -vm_args /Users/thanos/work/tanodb/_build/default/rel/tanodb/generated.conf/vm.2016.04.19.22.07.13.args  -- console
Root: /Users/thanos/work/tanodb/_build/default/rel/tanodb
Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:4:4] [async-threads:64] [kernel-poll:true]

Setup running ...
Directories verified. Res = {[ok],[]}
Setup finished processing hooks ...
22:07:14.421 [info] Application lager started on node '[email protected]'
22:07:14.423 [info] Starting reporters with []
22:07:14.423 [info] Application exometer_core started on node '[email protected]'
22:07:14.423 [info] Application poolboy started on node '[email protected]'
22:07:14.423 [info] Application pbkdf2 started on node '[email protected]'
22:07:14.423 [info] Application eleveldb started on node '[email protected]'
22:07:14.429 [info] Application sasl started on node '[email protected]'
22:07:14.429 [info] Application basho_stats started on node '[email protected]'
22:07:14.434 [info] Application os_mon started on node '[email protected]'
22:07:14.439 [info] Application riak_sysmon started on node '[email protected]'
22:07:14.441 [info] Application crypto started on node '[email protected]'
22:07:14.446 [info] alarm_handler: {set,{{disk_almost_full,"/"},[]}}
22:07:14.516 [info] New capability: {riak_core,vnode_routing} = proxy
22:07:14.533 [info] New capability: {riak_core,staged_joins} = true
22:07:14.548 [info] New capability: {riak_core,resizable_ring} = true
22:07:14.561 [info] New capability: {riak_core,fold_req_version} = v2
22:07:14.578 [info] New capability: {riak_core,security} = true
22:07:14.591 [info] New capability: {riak_core,bucket_types} = true
22:07:14.607 [info] New capability: {riak_core,net_ticktime} = true
22:07:14.656 [info] Stopped  application riak_core.
[os_mon] cpu supervisor port (cpu_sup): Erlang has closed
{"Kernel pid terminated",application_controller,"{application_start_failure, tanodb,{bad_return,{{tanodb_app,start,[normal,[]]},{'EXIT',{undef,[{cowboy_router,compile,[[{'_',[{\"/ping\", tanodb_http_ping,[]}]}]],[]},{quantumdb_app,init_http,0,[{file,\"/Users/thanos/work/tanodb/_build/default/lib/quantumdb/src/tanodb_app.erl\"},{line,36}]},{quantumdb_app,start,2,[{file,\"/Users/thanos/work/tanodb/_build/default/lib/tanodb/src/quantumdb_app.erl\"},{line,13}]},{application_master,start_it_old,4,[{file,\"application_master.erl\"},{line,273}]}]}}}}}"}

when you run rebar3 run (using R18)

vm.args needs to have a -name parameter.
-sname is not supported.
couldn't find 'nodename' in ../tanodb_config/tanodb.conf
$PWD is /Users/gyanendraaggarwal/tanodb