zilverline/sequent

CQRS & event sourcing framework for Ruby

Sequent

Sequent is a CQRS and event sourcing framework written in Ruby.

Getting started

See the official site at https://www.sequent.io/

New to Sequent? Getting Started is the place to be!

Contributing

Fork and send pull requests

Documentation

See the official site at https://www.sequent.io/

Want to help improve the documentation? Please let us know how we can improve by creating an issue

If you want to help write the documentation fork and send pull request.

You can start the documentation locally via:

cd docs
bundle install
bundle exec jekyll serve --livereload

Open localhost:4000

Releasing

Change the version in lib/version.rb. Commit this change.

Then run rake release. A git tag will be created and pushed, and the new version of the gem will be pushed to rubygems.

Running the specs

First create the database if you did not already do so:

createuser -D -s -R sequent
SEQUENT_ENV=test bundle exec rake sequent:db:create

Run rspec spec to run the tests.

Changelog

The most notable changes can be found in the Changelog

License

Sequent is released under the MIT License.

sequent's Issues

Add explicit assertion for payload variables

I noticed while writing specs that the sequence number wasn't being validated and I think it's related to payload_variables being rejected before serialization.

sequent/lib/sequent/core/event.rb

Lines 23 to 36 in f1f9667

	def payload
	result = {}
	instance_variables
	.reject { |k| payload_variables.include?(k) }
	.select { |k| self.class.types.keys.include?(to_attribute_name(k))}
	.each do |k|
	result[k.to_s[1 .. -1].to_sym] = instance_variable_get(k)
	end
	result
	end
	protected
	def payload_variables
	%i{@aggregate_id @sequence_number @created_at}
	end

My thoughts are updating command_handler_helpers.rb and event_handler_helpers.rb to also include aggregate_id and sequence_number but want to make sure there's not a better location to add this.

diff --git a/lib/sequent/test/command_handler_helpers.rb b/lib/sequent/test/command_handler_helpers.rb
index f866793..361ff02 100644
--- a/lib/sequent/test/command_handler_helpers.rb
+++ b/lib/sequent/test/command_handler_helpers.rb
@@ -143,8 +143,17 @@ module Sequent
 
         Sequent.configuration.event_store.stored_events.zip(expected_events.flatten(1)).each_with_index do |(actual, expected), index|
           next if expected.class == Class
-          _actual = Sequent::Core::Oj.strict_load(Sequent::Core::Oj.dump(actual.payload))
-          _expected = Sequent::Core::Oj.strict_load(Sequent::Core::Oj.dump(expected.payload))
+
+          _actual = Sequent::Core::Oj.strict_load(Sequent::Core::Oj.dump(actual.payload)).merge({
+            'aggregate_id' => actual.aggregate_id
+            'sequence_number' => actual.sequence_number
+          })
+
+          _expected = Sequent::Core::Oj.strict_load(Sequent::Core::Oj.dump(expected.payload)).merge({
+            'aggregate_id' => expected.aggregate_id
+            'sequence_number' => expected.sequence_number
+          })
+
           expect(_actual).to eq(_expected), "#{index+1}th Event of type #{actual.class} not equal\nexpected: #{_expected.inspect}\n     got: #{_actual.inspect}" if expected
         end
       end

Custom validation ?

How to use custom validation to talk to external service to check validation in Command ?

Heroku deployment

When deploying to Heroku, i've got problem with migration.
Heroku postgres doesn't allow us create database, so what's command should i use to migrate event_store to existing database ?

Getting started guide creates an unexpected Exception

When you first describe the code for the Usernames AggregateRoot on line 177 of modelling the domain you describe:

def self.instance(id = ID)
  repository.load_aggregate(id)
  ...
  end

But, the AggregateRoot base model doesn't have access to respository in the same way that the CommandHandler base class does. So, when running the tests it raises an exception:

1) AuthorCommandHandler AddAuthor creates a user when valid input
     Failure/Error: repository.load_aggregate(id)
     
     NameError:
       undefined local variable or method `repository' for Usernames:Class
     # ./lib/usernames/usernames.rb:8:in `instance'

Later, you describe the Usernames AggregateRoot class and the method has changed to:

  def self.instance(id = ID)
    Sequent.configuration.aggregate_repository.load_aggregate(id)
   ...
  end

Which works, but the small change isn't called out in the docs and it's unclear why it would use an unnamed repository method earlier instead of always using Sequent.configuration.aggregate_repository.

Spec error in generated spec_helper.rb under Ruby 2.4.6

Under 2.4.6:

$ bundle exec rspec

An error occurred while loading ./spec/app/projectors/post_projector_spec.rb.
Failure/Error: require_relative '../../spec_helper'

SyntaxError:
  /Users/tom/development/msbit/sequent-test/spec/spec_helper.rb:35: syntax error, unexpected keyword_ensure, expecting keyword_end
    ensure
          ^
  /Users/tom/development/msbit/sequent-test/spec/spec_helper.rb:44: syntax error, unexpected keyword_ensure, expecting keyword_end
      ensure
            ^
  /Users/tom/development/msbit/sequent-test/spec/spec_helper.rb:47: syntax error, unexpected keyword_end, expecting end-of-input
    end
       ^
# ./spec/app/projectors/post_projector_spec.rb:1:in `require_relative'
# ./spec/app/projectors/post_projector_spec.rb:1:in `<top (required)>'

An error occurred while loading ./spec/lib/post/post_command_handler_spec.rb.
Failure/Error: require_relative '../../spec_helper'

SyntaxError:
  /Users/tom/development/msbit/sequent-test/spec/spec_helper.rb:35: syntax error, unexpected keyword_ensure, expecting keyword_end
    ensure
          ^
  /Users/tom/development/msbit/sequent-test/spec/spec_helper.rb:44: syntax error, unexpected keyword_ensure, expecting keyword_end
      ensure
            ^
  /Users/tom/development/msbit/sequent-test/spec/spec_helper.rb:47: syntax error, unexpected keyword_end, expecting end-of-input
    end
       ^
# ./spec/lib/post/post_command_handler_spec.rb:1:in `require_relative'
# ./spec/lib/post/post_command_handler_spec.rb:1:in `<top (required)>'
No examples found.


Finished in 0.00036 seconds (files took 0.07237 seconds to load)
0 examples, 0 failures, 2 errors occurred outside of examples

Under 2.5.5:

$ bundle exec rspec
...

Finished in 0.15 seconds (files took 0.81271 seconds to load)
3 examples, 0 failures

Likely due to:

rescue/else/ensure are now allowed to be used directly with do/end blocks.

added as part of 2.5.0.

This might simply be a matter of changing:

Sequent require Ruby version 2.3.7 or later.

on https://www.sequent.io/docs/getting-started.html

Webhooks and events

Here I am with other design questions.

Let's say I use a web API and they send a lot of events with webhooks. For example, with Stripe, I would receive invoice.created once a month for each customer with monthly payments.

1. Should I receive the webhook and immediately send the data to the EventStore or should I create commands?

2. What would the aggregate_id be? Should I use the id Stripe provides or should I generate one that is unique to our system?

Thanks again for your time. The more I learn about Sequent and CQRS, the more I enjoy it.

Strict apply values for events

Sequent currently ignores apply ThisHappened, does_not_exist: 'value' attributes that are
not declared on an Event. The same applies if you put a value in that is not of the correct type. Type checking and casting is done in the CommandService, but this leaves room for programming errors.

Since Events are forever it is better to be more strict about this so no accidental errors occur. apply should fail if the attributes do not exist, or the types are wrong.

Sharding EventStore ?

Do you have any idea on sharding our EventStore ? ;)
Note: We have PgBouncer to keep consistent connection pooling, so sharding by just adding more databases is a good strategy.
But Sequent should have a built-in strategy to switch connection at runtime to find correct database to add to EventStore.
The command_service could have interface:

Sequent.command_service.switch(:shard).execute_commands(...)

About changing business rule

I have one question about ES in general and Sequent specifically.
Old persisted events are recorded due to business rule.
So how about changing business rules could be handled ?
We have two things to do:

• Changing codes in command validation
• More processing when building view from old events data (so that invalid state couldn't be built)

How do you think about this problem ?

Event changing ?

In Sequent, what's the correct way to deal with

• Event schema changing
• Event deleting

Thanks .

Sagas？

Hi,

I'm also interested in implementing CQRS/ES in Ruby and searched into this repo.

May I ask how would you support sagas/process managers? Because I didn't notice any related codes.

Support online migrations for new projections

I'd really like to find a way to deploy new projections (not rebuild existing ones) without downtime.

I've been banging my head against this for a while and haven't come up with a good solution. We are happy to implement a new migration strategy and contribute it back or to sponsor development of this feature - but maybe we can start with a discussion re approach.

Originally, I was hoping to "cheat" a bit and have an online migration that gets caught up with all existing events and then quickly grabs a full table lock on the events table and then triggers a "reload" of all processes to pick up the projector. The problem is, that's basically the same as an offline migration (will incur downtime), it's just more tightly automated.

So far, the best option I can think of is to introduce PG listen/notify and to have projection handlers that listen for new events.

Has anyone given this some thought that they'd care to share?

Support MySQL?

How difficult would it be to support MySQL?

Rails 5.0, 5.1

Any plans to support new rails ?

sequent generator does not work in 3.1.0

gem install sequent
sequent

expected

Please specify a command

actual

cannot load such file -- parser/current (LoadError)

Generated projects are missing a README file

Calling load_aggregate outside of a CommandHandler leads to cache desyncs

Right now the unit-of-work cache is only cleared when the command service completes a unit of work. That means that if in Thread 1 I call load_aggregate and then in Thread 2 call execute_command for that aggregate, I will now have two different copies of the aggregate in each thread, since it is cached in Thread 1.

I think the intention might be that applications should not load aggregates outside of command handlers, but whether that's true or not is not clear.

There are probably at least three approaches to addressing this issue, either:

• update the documentation to plainly describe the limitations on aggregate access
• move the unit-of-work cache out of AggregateRepository and into CommandService -- effectively, AggregateRepository#load_aggregate would no longer cache aggregates, but a unit-of-work cache would still be maintained with uncommitted events within command processing.
• update the cache to check whether any new events have occurred on the aggregate since it was cached, then incrementally apply any new events to the aggregates if necessary.

How to delete entity

I am using this project to teach myself CQRS/ Event Sourcing. I am trying to implement a Delete command and Event for one of aggregates. I don't see an example of how to do this in the specs or example rails app. What is the recommended way on how to implement this feature?

Explain better how update aggregate works

From #136

What does the aggregate repository do? Does add_aggregate write to the DB? If so, which DB and what records are made? What does an update_aggregate operation look like?

Add to the docs that there is no need to update_aggregate since the events applied are automatically saved

Errors in Getting Started guide

Undefined route after Account creation

After configuring the DB Connection pool for sinatra in the Getting Started guide p3 we should get a "success" when creating a new account through the web form. But the post method directs to the as yet undefined route at /authors/id/#{author_id} after the author is created.

Missing flash output

While we create a flash notice in the controller there is no flash being printed in the template.

We forget to require AuthorProjector after we define it.

We create a new file in app/projectors/author_projector.rb but do not require it in the blog.rb file. So, when we go to run the migration we get and error: NameError: uninitialized constant AuthorProjector

There is no definition of the author events

When we implement the projector for authors in part 3 and submit the form to create a new author we get an error:

Sequent::Core::EventPublisher::PublishEventError - Event Handler: AuthorProjector
Event: #<AuthorNameSet:0x00007fd4ff075db0 @aggregate_id="91cd938a-9d0b-47ef-9026-5ef03a412ef7", @sequence_number=2, @created_at=Thu, 25 Apr 2019 15:29:30 +0200>
Cause: #<ArgumentError: Empty list of attributes to change>:

as back in part 2 we stubbed out events for AuthorNameSet and AuthorEmailSet but we never define the attributes.

===

Sorry these are coming in piecemeal, it's just as I get some spare time to follow through the tutorial - which is excellent otherwise!

FakeEventStore should raise a non unique error

The event records table in the actual event store raises a duplicate key error for the aggregate id, but the FakeEventStore does not.

bundle exec rake sequent:migrate:online fails citing no sequent_versions table

Hi there! I'm trying to add this to an existing rails application that already has some non event sourced seed data managed by activerecord. I run through all of the steps on the getting started section

bundle install
bundle exec rake sequent:db:create
RACK_ENV=test bundle exec rake sequent:db:create
bundle exec rake sequent:db:create_view_schema

Everything is successful at this point, I did have to move a few things around and add some config options telling sequent where my database.yml lived, but then when i run

bundle exec rake sequent:migrate:online

I get the following error:

ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR:  relation "sequent_versions" does not exist
LINE 1: SELECT  "sequent_versions".* FROM "sequent_versions" ORDER B...
                                          ^
: SELECT  "sequent_versions".* FROM "sequent_versions" ORDER BY version desc LIMIT $1
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:611:in `async_exec_params'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:611:in `block (2 levels) in exec_no_cache'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activesupport-5.2.3/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:610:in `block in exec_no_cache'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:581:in `block (2 levels) in log'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:580:in `block in log'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activesupport-5.2.3/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:571:in `log'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:609:in `exec_no_cache'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:598:in `execute_and_clear'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:81:in `exec_query'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:478:in `select'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:70:in `select_all'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/query_cache.rb:106:in `select_all'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/querying.rb:41:in `find_by_sql'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/relation.rb:560:in `block in exec_queries'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/relation.rb:584:in `skip_query_cache_if_necessary'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/relation.rb:547:in `exec_queries'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/relation.rb:422:in `load'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/relation.rb:200:in `records'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/relation.rb:195:in `to_ary'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/relation/finder_methods.rb:532:in `find_nth_with_limit'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/relation/finder_methods.rb:517:in `find_nth'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/activerecord-5.2.3/lib/active_record/relation/finder_methods.rb:125:in `first'
/Users/jared/.rvm/gems/ruby-2.6.4/bundler/gems/sequent-559b2f259d5a/lib/sequent/migrations/view_schema.rb:87:in `current_version'
/Users/jared/.rvm/gems/ruby-2.6.4/bundler/gems/sequent-559b2f259d5a/lib/sequent/migrations/view_schema.rb:304:in `drop_old_tables'
/Users/jared/.rvm/gems/ruby-2.6.4/bundler/gems/sequent-559b2f259d5a/lib/sequent/migrations/view_schema.rb:275:in `rollback_migration'
/Users/jared/.rvm/gems/ruby-2.6.4/bundler/gems/sequent-559b2f259d5a/lib/sequent/migrations/view_schema.rb:163:in `rescue in migrate_online'
/Users/jared/.rvm/gems/ruby-2.6.4/bundler/gems/sequent-559b2f259d5a/lib/sequent/migrations/view_schema.rb:147:in `migrate_online'
/Users/jared/.rvm/gems/ruby-2.6.4/bundler/gems/sequent-559b2f259d5a/lib/sequent/rake/migration_tasks.rb:91:in `block (3 levels) in register_tasks!'
/Users/jared/.rvm/gems/ruby-2.6.4/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
/Users/jared/.rvm/gems/ruby-2.6.4/bin/ruby_executable_hooks:24:in `eval'
/Users/jared/.rvm/gems/ruby-2.6.4/bin/ruby_executable_hooks:24:in `<main>'

I looked in my database and I do have a view_schema group with a schema_versions table, but for some reason the migration task can't see it. I don't think this has something to do with the existing database i have set up, because I can drop everything and run the sequent specific commands first and it still fails.

Any thoughts?

Thanks!

https://www.sequent.io is not picking up the certificate

Store aggregate_id as UUID

aggregate_id's are currently being stored as string by default and in docs. This can be a postgres uuid column.

`create_records` does not work in ActiveRecord 5.1

It results in a TypeError: can't quote Array

Rake tasks and rails integration

So far it looks like there is no obvious way to have drop-in installation into rails environment.
Is it by design?
Also, what would you say, if I'll make a PR with all rake tasks prefixed with sequent: namespace ?

Uniqueness constraints

What's the best way to validate uniqueness constraints of an attribute in Domain ?

Errors in Getting Started guide

In the getting started, When trying to create an author nothing happens.
You actually forgot to register the author_command_handler in the sequent initializer.

  config.migrations_class_name = 'Migrations'

  config.command_handlers = [
    PostCommandHandler.new,
    AuthorCommandHandler.new,
  ]

  config.event_handlers = [
    PostProjector.new
  ]
end

Explain tables in sequent_schema

From #136

• event_records
• command_records
• stream_records

Not compatible with Rails 5.2.1

rails new foldername --api -T -d postgresql

Then add sequent to the Gemfile and run bundle install. Output is as follows.

Fetching gem metadata from https://rubygems.org/.
Resolving dependencies...
Bundler could not find compatible versions for gem "activerecord":
  In snapshot (Gemfile.lock):
    activerecord (= 5.2.1)

  In Gemfile:
    rails (~> 5.2.1) was resolved to 5.2.1, which depends on
      activerecord (= 5.2.1)

    sequent was resolved to 3.0.0, which depends on
      activerecord (< 5.2, >= 5.0)

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

Bundler could not find compatible versions for gem "pg":
  In snapshot (Gemfile.lock):
    pg (= 1.0.0)

  In Gemfile:
    pg (< 2.0, >= 0.18)

    sequent was resolved to 2.1.0, which depends on
      pg (~> 0.18)

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

Getting started guide uses the wrong directory for Sinatra templates

By default, sinatra will use the app/views/ directory for templates.

In the Getting started - building a web application you instruct the creation of an era file in app/view/index.erb

The correct path is used later for app/views/authors/show.erb and app/views/authors/index.erb so I assume this is a typo.

Errors encountered in Getting Started guide

At the end of section 1.1, bundle exec rspec runs into some uninitialized constant errors.

• ruby 2.6.3p62
• PostgreSQL 11.2
• sequent (3.2.2)

$ bundle exec rspec

An error occurred while loading ./spec/app/projectors/post_projector_spec.rb.
Failure/Error: require 'sequent/test'

NameError:
  uninitialized constant Sequent::ApplicationRecord
# ./spec/spec_helper.rb:6:in `require'
# ./spec/spec_helper.rb:6:in `<top (required)>'
# ./spec/app/projectors/post_projector_spec.rb:1:in `require_relative'
# ./spec/app/projectors/post_projector_spec.rb:1:in `<top (required)>'

An error occurred while loading ./spec/lib/post/post_command_handler_spec.rb.
Failure/Error: require 'sequent/test'

NameError:
  uninitialized constant Sequent::ApplicationRecord
# ./spec/spec_helper.rb:6:in `require'
# ./spec/spec_helper.rb:6:in `<top (required)>'
# ./spec/lib/post/post_command_handler_spec.rb:1:in `require_relative'
# ./spec/lib/post/post_command_handler_spec.rb:1:in `<top (required)>'
No examples found.


Finished in 0.0002 seconds (files took 0.2669 seconds to load)
0 examples, 0 failures, 2 errors occurred outside of examples

Repo here: https://github.com/reentim/blog

Design of the event handler

I am getting more and more comfortable in using Sequent and I think I came to the point where I understand what you mean by "Sequent is an opinionated CQRS and event sourcing framework". There are many ways to implement CQRS and opinions are needed to keep a framework lightweight without too much configuration and keep it's maintenance manageable.

There's one design choice that I try to work with right now and I'd like to discuss this with you.

Workflows

Workflows can be used to do other stuff (then updating a Projection) based on Events. Common tasks run by Workflows are:

1. Execute other Commands
2. Schedule something to run in the background

In Sequent Workflows are committed in the same transaction as committing the Events.

Since Workflows have nothing to do with Projections they do not run when doing a Migration.

First of all, I totally see how workflows and projections are both event handlers. When events happen, things happen. I also appreciate the elegance of everything being committed in the same transaction. In my understanding, a projection has no reason to fail and make the entire transaction to rollback. It should go smoothly. But my reasoning is not the same with workflows. The example provided, enqueuing a delayed job to send an email, has side effects. Which is also fine if it's the last step. Enqueuing a job is a very safe operation and is very unlikely to fail and rollback the transaction.

There are small and bigger issues I'm trying to solve here.

The first one is enqueuing a background job. It allows the transaction to be committed and a more "risky" task can be executed and the result will not impact previous events. A small annoyance is that often, my jobs will start executing even before the transaction has finished to commit. So sending a command from that background job using an aggregate_id that is not yet committed will fail. That's ok, I have retry mechanisms and the second try will succeed. But how to distinguish an aggregate that its command failed and one that is simply not ready yet?

Should there be some kind of event handler that can execute only if the events are actually committed? Let's say I'm sending an email after the UserCreated event and because of another command in the same queue fails. The database transaction will rollback but I can't take back the email. If it's ran in a background job, I can't cancel the job if it's already finished.

Should it be the Sequent::Core::EventStore's responsibility to use a transaction in store_events and publish events when it's actually committed? This is not a crazy idea if we embrace eventual consistency of the read side of the app. The Sequent::Core::CommandService could have it's own transaction provider that is not based on the database and could offer rollback features to workflows. But that will totally break the current behavior explained here: https://www.sequent.io/docs/concepts/command-service.html#

If I go ahead with my own refactor of Sequent::Core::EventStore and Sequent::Core::CommandService, would you be interested in a Pull Request or this is far from the direction you want to take with Sequent?

Question - Why store Commands in the Event Store?

I really enjoyed reading the other design questions in here and I'm trying to understand this all a lot better.

I'm left wondering why sequent stores the commands in the DB as well as the events? There is no clear way to interact with the commands, or did I miss something?

State in Workflows?

Hi, first of all, a big thank you for all the hard work on Sequent.

I'm wondering how you recommend handling state in workflows. By that I mean if a workflow needs to keep state to do it's work, where should that be kept? In a separate projection? In a Record but one that is managed by the workflow? A normal ActiveRecord model? Something else?

As a (contrived) motivating example:
Imagine a Workflow for calculating achievements:

class GranterOfAchievements < Sequent::Workflow
  on ProductPurchased do |event|
    purchase_count = get_state_somehow(event.customer_id)
    new_count = purchase_count + 1
    store_state_somehow(event_customer_id, new_count)

    if new_count == 10
      command = GrantAchievement(
        aggregate_id: event.customer_id, 
        name: 'First Ten Yay'
      )
      Sequent.command_service.execute_commands(command)
    end
  end
end

I understand that it's better to have stateless Workflows, but it's not always possible. What is the Sequent take on this?

Thanks

Be more flexible with versions of active_record etc..

I wanted to add sequent to my project, but because I am on rails 5.2.1, and a version of rubocop meant that the parser gem would be too high version number, I was unable to add it.

I still plan on using it in the future, but would prefer to wait until you officially support this version of rails rather than me blindly just upping the version number in a PR etc..

Date format should not be locale specific

The format of a Date parameter should not be hard-coded as the Dutch date format (%d-%m-%Y). Like with DateTime the format should be iso8601.

`sequent:db:create_event_store` will fail for some DB username

GIVEN

the connected DB username is either:
(i) identify with any keyword defined in PG, or
(ii) a case sensitive string.
Let's assume it is a random string "Aw1PrIwW"

WHEN

try to run the sequent migration tasks which help create schema:
rake sequent:db:create_event_store

EXPECTED

it will run as expected, Schema like sequent_schema should be created.

CURRENTLY

No, it will fail. rake command will be aborted by following error message:

ActiveRecord::StatementInvalid: PG::UndefinedObject: ERROR:  role "aw1prIww" does not exist
: CREATE SCHEMA IF NOT EXISTS sequent_schema AUTHORIZATION Aw1PrIwW

My Guess on Why

at lib/sequent/support/database.rb this method self.create_schema

The User role part of the Sql query is not double quoted, so a username 'Aw1PrIwW' will be treated as 'aw1prIww'. Then, system cannot find this user role.

ActiveRecord 6

I am trying Sequent with the beta1 version of ActiveRecord 6.0.0, in a Rails app. It seems to work just fine. Is there any roadmap to make sure Sequent is fully compatible when the official version of Rails 6.0 comes out?

Right now, I am testing with my own fork because I had to edit the gemspec to resolve Bundlers' dependencies conflicts.

Test DB not created

After following the steps in Getting Started, the final step bundle exec rspec fails with the following error:

  FATAL:  database "projectname_test" does not exist

It's not clear how to create the test DB from the docs.

Should `Boolean` really be a class?

Currently, the Boolean constant used for validating true/false is a class:

sequent/lib/sequent/core/ext/ext.rb

Lines 32 to 36 in 0ef301d

	class Boolean
	def self.deserialize_from_json(value)
	value.nil? ? nil : (value.present? ? value : false)
	end
	end

Is there a reason that this was made a class rather than a module? It doesn't seem particularly meaningful to instantiate Boolean, and having it be a class is a hard conflict with a not-uncommon solution to a related problem:

module Boolean; end
TrueClass.include Boolean
FalseClass.include Boolean

Can Boolean be changed to a module?

Dropping support for ruby < 2.5 in Sequent 4.0

We will be dropping support for ruby < 2.5 in the next major release because:

• It is EOL soon: https://www.ruby-lang.org/en/downloads/
• So we can use nice stuff like yield_self

Using commands and aggregates

First of all, I'm sorry if this isn't the right place for the kind of questions I have. Couldn't find the official way to contact you for questions (IRC? Slack? email?).

I successfully created an app that creates an account, based on your getting started guides. So far, I like how sequent is built and how I interact with it. I am already very familiar with ActiveRecord, so I really appreciate that it is used for the event store.

Now, I try to understand what are correct and bad ways to implement defaults and building relationships between aggregates. For example, an account can hold a lot of things. There's credential info, it can also hold identity info, contact info, licence, preferences, etc. With Rails and traditional CRUD, I could create multiple models for each one of those concepts. Then, I would use an after_create hook to create the associated records for the user.

Would all commands and events be created under the same account aggregate? I looks very convenient to me.

ValueObject to store json

I'd like to pass a hash into some events and commands.

I'm struggling to find the best way to write a custom ValueObject that serializes to json and back.

Mind pointing me in the right direction? Thanks!

Support patch versions for projections

Currently there is a single strategy of building projections:

The new projection is created in a background process by replaying all events. At the end the application needs a little downtime to get the events added after the new projection was created.

This strategy is fast and works great if the projections don't need many events (let's say less then 5.000.000) to rebuild its state. Also having a single strategy is simple and easy to understand.

During a lot of migrations we are just adding data to the new projection. Data that previously did not exist. In this case it makes no sense to rebuild the entire projection again. Although downtime is still minimized rebuilding a projection in the background can take significant amount of time. For example an event store of > 200M events and replaying ~ 15M events can take up to an hour in the background.

So what we like to have is a way of allowing new data in projections (adding columns to a table) without needing to rebuild the entire projection. This can be achieved by adding a patch version.

Proposal to implement this:

class ViewSchemaMigrations < Sequent::Migrations::Projectors
  def self.version
    '1.1'
  end
  def self.versions
    {
      '1' => [InvoiceProjector, EstimateProjector],
      '1.1' => [InvoiceProjector],
    }
  end
end

We then also need the following sql files:

invoice.sql containing the initial table creation.
invoice_1.1.sql containing the addition of the columns.

The Sequent::Migrations::ViewSchema::Versions needs to have an extra column patch_version to store the current patch version (default 0).

In Sequent::Migrations::ViewSchema.migrate_offline we can implement applying the patch version.

We also need to think about how to deal with rollbacks and if we want to support the following scenario: upgrading from version '1.0' via '1.1' directly to version '2.0'.

cc @stephanvd @bforma @mvandiepen

Command replaying ?

Imagine we could have ability to drop out event store ! It's possible by replaying our commands .
I think this has potential for being able to swap out EventStore .
Any thought about this ?

Questions about schema in a Rails application

First of all, let me just say that from my (so far) limited experience with this project, it's pretty great.

Here's my situation: we're interested in pursuing an event-sourced architecture for the application we're working on. It seems that we want to stick with Rails for a handful of reasons, instead of using a Sinatra app as the examples illustrate. So far, I have everything working – I'm able to execute commands, observe that the resulting events are being stored and my projections being created – but there are a few bits and pieces that I’m a little unsure about, mostly when it comes to schema management.

1. How should we be managing schema in a Rails environment? The documentation seems to be a little confusing when it comes to migrations in general – it says “it is strongly recommended to copy this schema into your first migration and then use migrations to keep it up to date”, then proceeds to say “view schema is not maintained through migrations” in the next section, and then the following section talks about “rake tasks for migrations”.

Does this mean that the Sequent schema should be loaded via a migration (e.g. db/migrate/xxxxxxxxxxx_create_sequent_schema.rb) as you would normally in a Rails app, but the view schema (for projections) should be loaded directly via something like rake db:schema:load? Or should both of these things be handled via a schema load task and traditional Rails migrations are out of the picture completely? If that’s the case, then the migration_path option for the “Rake Tasks for migrations” confuses me.

2. How should we set the schema search path in Rails? It’s generally done in the database.yml, but it seems like managing the view schema version in that would be difficult / redundant.

Honestly, if you can just give me a “For Dummies” explanation of how to handle schema for Sequent in Rails (which it seems you guys have done, judging by some replies in other issues) I would really be appreciative – as would others, I imagine.

Thanks!

Errors capture (fail ~ apply)

Is it useful to create a table for Error ? I mean treat Error in AggregateRoot as a business event also. The only difference is that it won't be used to replay to rebuild state.

In this case, the fail behaviour should be the same as apply.

Can't load schema when the DB already exists

For our production deploy, the database already exists, but is empty. I can't run the sequent:migrate:online task as I get an error that the event_records table does not exist. Dropping the DB and running sequent:db:create is not an option as the app does not have permission to destroy the DB.

It would be ideal to have a sequent:schema:create task that assume the DB is already present.

Hi. Thanks for a great framework! Sequent looks to be very useful for us.

Hi. Thanks for a great framework! Sequent looks to be very useful for us.

I've been starting to learn how to use it, but have found that some things in the docs are a little unclear, making it slow to get started. I've kept track of the questions that I've had in mind whilst going along, so perhaps it would be helpful when revising the docs for me to post them here:

• Is it better to have the attributes (e.g. the title of the Post) as fields on the command/event or to have a PostAttributes value object to encapsulate them so that the command/event has that as a single attribute instead? What are the pros and cons of each approach?
• How and when are the events applied when the aggregate is reconstructed from the event stream? The initialize method will be called each time a new object is instantiated, but there is logic in the examples that looks like it should only run when the initial AddPost command is handled.
• What does an UpdatePost command/event look like if I am using a HTML form to do the update (many fields)
• Why are AggregateRoots and Projectors different classes? I know that this is the point of the CQRS pattern, but the docs and examples show no use case where the projectors do anything other than a 1-1 mapping of Projector to AggregateRoot. What would that look like?
• What is an EventHandler? Some examples have this, but the current docs don't. I see from the changelog that it's been renamed to Projector, but the examples were a source of confusion until I found this.
• What does the aggregate repository do? Does add_aggregate write to the DB? If so, which DB and what records are made? What does an update_aggregate operation look like?
• There are 3 DB tables in the sequent_schema: event_records, command_records and stream_records. What are they used for and when? Events I assume are used to rebuild the aggregates, but commands I don't see mention of being used and I don't know what streams are at all.
• Initially, it was not clear to me that the aggregate_id needed to be manually generated. I was assuming it would be automatically made on repository.add_aggregate
• Why is it not possible to do a zero downtime migration to add a new projection (leaving the existing ones alone)? Is this theoretically possible in future or will it never be?
• Is it OK to rename aggregate_id as id in the projections so as to be consistent with our other services? Does this matter or not?

Hopefully that's useful. I'm not very familiar with the CQRS pattern so perhaps some of these questions arise from seeing this project as my first experience with event sourcing. I'm happy to help with writing the documentation if that would be useful, but I don't know the answers to all of the above questions yet.

Originally posted by @mattgibson in #121 (comment)