whhe/mongo-replicator

A real-time replication tool for MongoDB replica set using change streams.

Mongo Replicator

Mongo Replicator is a real-time replication tool for MongoDB replica set. It use Change Streams to fetch real-time data changes, with which users can sync data to other containers in real time.

Requirements

Change Stream require 3.6 or higher version of MongoDB, and is only available for replica set.

For watch scope, deployment or database level is available for 4.0 or higher version of MongoDB.

For ChangeStreamOptions, startAtOperationTime require MongoDB 4.0, and startAfter require MongoDB 4.2.

Concepts

Mongo Replicator consists of Collector, Replicator and Operator.

Collector

Collector represents the data source to watch.

You can set the watch scope when creating a Collector instance. According to the params, the data source can be a deployment (either a replica set or a sharded cluster), several databases or collections.

NewCollector(uri string, databases []string, collections []string) *Collector 

Collector has a Collect method to perform the watch action.

(c *Collector) Collect(opts ...*options.ChangeStreamOptions) (*mongo.ChangeStream, error)

See ChangeStreamOptions to customize your change streams.

Replicator

Replicator is the unified interface to perform the replication. It is initialized by an Operator and has only one method Replicate.

func (r *Replicator) Replicate(e model.ChangeEvent) error 

Operator

Operator interface defined the methods need to be implemented for the target container to deal with change event documents.

type Operator interface {
	Insert(model.ChangeEvent) error
	Delete(model.ChangeEvent) error
	Replace(model.ChangeEvent) error
	Update(model.ChangeEvent) error
	Drop(model.ChangeEvent) error
	Rename(model.ChangeEvent) error
	DropDatabase(model.ChangeEvent) error
	Invalidate(model.ChangeEvent) error
}

You can implement your own Operator to customize the replication logic.

Usage and Example

See example and godoc for reference.

License

Apache 2.0 License