diego3g / umbriel Goto Github PK

View Code? Open in Web Editor NEW

506.0 6.0 72.0 936 KB

✉️ Umbriel is a simple mailing platform built with Node.js, TypeScript & Prisma.

License: MIT License

Shell 0.22% TypeScript 98.49% JavaScript 0.45% Handlebars 0.65% Dockerfile 0.19%

node ddd typescript prisma

umbriel's Introduction

Umbriel

Umbriel is a simple mailing platform built with Node.js, TypeScript & Prisma.

umbriel's People

Contributors

Stargazers

Watchers

Forkers

robsonmobile vitorformiga reyadrahman santos-jp7 gabriellopes00 bernardogeneroso renan-garcia pyfundation naumjefferson optydeviocourses fjpiedade hheenry uzlog profencer adroaldopagliari mixtan andrianbalanescu suryatmodulus frnangelim akira-development shaunstanislauslau dropefernandes beingsane joaoalcdev diogocezar tanjiaoyun hardjunior alanjholiveira allandiego davydcardoso lehcosta bismarck4lves maiconcosta isaamante martinsedm diogofgomespinheiro rythelle eduardodemartinigiacomini alexrogalskiy octalpixel mkuchak observabilidade renan5g danielhesse kodiyak midianet harryhan24 saurabhkashyap filipemacedo saurabharch alexssmusica guiradi ilyosdev cordiz84 alexdemenezes treepo1 leandrodaher cristinaraila samuelsankys keisuke169 renatobrasilfone vitugc1 magominimalista higorcouto gabriel3p victor0814gui raul743 rodolfobelo19 carlosdiego369 brunoramux david-sousa-web lcsouzamenezes

umbriel's Issues

Update contact

Update contact details as name and email.

Message sending progress

Store the progress of the message sending inside the message stats.

This information maybe should not be calculated on every request.

Create Permissions & roles structure

Today, authenticated users can do anything inside Umbriel but it would be good if we could have some predefined roles with custom permissions.

Messages stats with distinct users

Today if a user clicks twice it counts twice on metrics, it should count only one per user.

Get all senders and templates routes

Count subscribers of tag route

Create a route that returns the amount of recipients based on a list of tag ids.

Sender confirmation

Every time a sender gets created or updated, we should send a validation email through AWS SES.

AWS SES do not allow any email to send messages, we need to validate them before start sending emails.

Invite team member

Let admins invite other members to Umbriel. The admin inputs the name and email, and the invited member receive an email with instructions to access.

We can use the RegisterUser use case that is not being used today.

Segments structure

Allow the creation of segments.

Segments are group of tags.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Warning

These dependencies are deprecated:

Datasource	Name	Replacement PR?
npm	`@babel/plugin-proposal-class-properties`
npm	`eslint-plugin-node`
npm	`request`

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

docker-compose

docker-compose.yml

bitnami/zookeeper 3

bitnami/kafka 3

dockerfile

Dockerfile

github-actions

.github/workflows/lint.yml

actions/checkout v2

actions/setup-node v2

bahmutov/npm-install v1

.github/workflows/porter_umbriel_api.yml

actions/checkout v2.3.5

.github/workflows/porter_umbriel_kafka.yml

actions/checkout v2.3.5

.github/workflows/porter_umbriel_queue.yml

actions/checkout v2.3.5

.github/workflows/porter_umbriel_sns_webhook.yml

actions/checkout v2.3.5

.github/workflows/test.yml

actions/checkout v2

bahmutov/npm-install v1

postgres 13

.github/workflows/tsc.yml

actions/checkout v2

actions/setup-node v2

bahmutov/npm-install v1

npm

package.json

@prisma/client 3.0.2

aws-sdk 2.978.0

bcryptjs 2.4.3

bullmq 1.40.4

cors ^2.8.5

dotenv ^10.0.0

dotenv-flow ^3.2.0

express 4.17.1

html-to-text 8.0.0

inline-css ^3.0.0

ioredis 4.28.0

jsonwebtoken 8.5.1

kafkajs ^1.15.0

nodemailer 6.7.0

request ^2.88.2

sns-validator ^0.3.4

uuid 8.3.2

@babel/cli 7.14.8

@babel/core 7.15.0

@babel/node 7.14.9

@babel/plugin-proposal-class-properties 7.14.5

@babel/preset-env 7.15.0

@babel/preset-typescript 7.15.0

@types/bcryptjs 2.4.2

@types/cors 2.8.12

@types/dotenv-flow 3.2.0

@types/express 4.17.13

@types/html-to-text 8.0.1

@types/inline-css 3.0.1

@types/jest 27.0.1

@types/jsonwebtoken 8.5.5

@types/nodemailer 6.4.4

@types/request 2.48.7

@types/sns-validator 0.3.1

@types/supertest 2.0.11

@types/uuid 8.3.1

@typescript-eslint/eslint-plugin 4.33.0

@typescript-eslint/parser 4.33.0

babel-plugin-module-resolver 4.1.0

eslint 7.32.0

eslint-config-prettier 8.3.0

eslint-config-standard 16.0.3

eslint-plugin-import 2.24.2

eslint-plugin-import-helpers 1.1.0

eslint-plugin-node 11.1.0

eslint-plugin-prettier 4.0.0

eslint-plugin-promise 5.1.0

jest 27.3.1

pg 8.7.1

plop 2.7.4

prettier 2.4.1

prisma 3.0.2

supertest 6.1.6

ts-jest 27.0.7

ts-node-dev 1.1.8

tsconfig-paths 3.11.0

typescript 4.4.3

Check this box to trigger a request for Renovate to run again on this repository

Public route for unsubscribe (sent in emails)

Allow contact to unsubscribe by clicking on the unsubscribe link inside the email.

Metrics & Graphs

Create a dashboard with some metrics & graphs:

How many new subscriptions each day during the last week (apply custom period)?
what more?

Message public share URL

Allow to share the message content in a public URL.

Send a test email before sending to mass

Replace `RequiredFieldsValidator` with per field validation

Hey @gabriellopes00, I merged the PR #69 but I think that RequiredFieldsValidator is not a good idea. I think we should add validation per field instead a validator that check for empty values within all request data. Can we fix that?

That's causing our tests to fail as userId is present in the request due to the EnsureAuthenticatedMiddleware but in the register route, this value is null so the tests are failing.

Filter false email opens on GMail

https://stackoverflow.com/a/57729879
https://www.gmass.co/blog/false-opens-in-gmail/

Save message editable content as well

Today we only save the message content in HTML format when the user creates it, but this prevents us from updating that message before sending it as the HTML is not editable by DraftJS on front-end.

We should save the message content in Draft format until the message is sent.

When the message is sent, we should fill another column inside the messages table like finalBody or finalContent that will have the message final HTML so we don't lose track about the message content in Draft format.

Add Controller Validations

Can the controller validations be made by creating a validator component within each module that requires a validation ? Example: Inside Account's modules, the RegisterUser controller needs a validator. Would be a good idea create a component called validator with the rules to validate the email, the user name, to compare password and password_confirmation... inside account's module, and inject that dependency inside de controller. This way, inside the controller, just call this.validator.validate(email...). What do you think ?

Add orderings to lists

All the search routes does not include a default order.

Return tag ID on contact details route

Today we are returning the ID from the relation between Tag and Contact and not the tag ID itself.

Handle bounce events

What happens when a bounce event is captured?

Should the contact be unsubscribed from that tag?
Should him be globally unsubscribed?
Should we wait until 3 bounces before unsubscribing?

Send message to specific contact list (by CSV)

Allow user to send a message to a specific list of contacts by importing a CSV during the message creation. This action should not save any relation between that contacts to tags inside Umbriel.

After the email is sent, that list will not be reachable again, only the contacts individually that will be persisted to Umbriel database.

UserRepository is updating in message with User Datas.

Wasn't the save supposed to update in prisma.user?

async save(user: User): Promise<void> {
    const data = await UserMapper.toPersistence(user)
    await prisma.message.update({
      where: {
        id: user.id,
      },
      data,
    })
}

Delete contact (hard delete)

Delete contact and all its data.

The main point here is that it would not be good to lose track of that contact inside message stats, so I think the message stats should be stored instead of calculating it from recipients every time it's requested.

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

Resubscribe contact

Undo unsubscribe of the contact.

Save only necessary data on events meta

Today when we receive notifications of reads, clicks or opens from AWS SES we save all the event metadata on the database but we don't need the whole object, we need only some specific data like, on clicks we wan't to know which link was clicked.

Update sender

Update sender data as name and email.

Implement the presentation layer

For now, we didn't implement a proper presentation layer with view models.

Maybe we could move the controllers to presentation layer if we want to leave the "package by feature" architecture.

Template variables

For now, the template only received one variable: {{ message_content }}, but it would be good to have other variables as:

{{ message_content }} - The full content of a broadcast or sequence message
{{ address }} - Your company's address, as set in your account settings
{{ unsubscribe_url }} - Unsubscribe URL (use to customize the link text)
{{ subscriber_preferences_url }} - Update Subscriber Preferences URL (use to customize the link text)
{{ subscriber.first_name }} - The first name of the subscriber
{{ subscriber.email_address }} - The email address of the subscriber

Allow to set account settings

Account name;
Address;

That's it (for now).

Summary

The HTTP service got a high CPU usage due to AWS SNS notifications;
The queue service got a high CPU usage due to queue processing to send emails;
The HTTP service got a high memory usage when adding the batch of jobs to the queue;

Errors

When receiving some AWS events, the HTTP service was logging some errors with this trace:

PrismaClientKnownRequestError2 [PrismaClientKnownRequestError]: Unique constraint failed on the fields: (`message_id`,`contact_id`)
    at cb (/home/node/app/node_modules/@prisma/client/runtime/index.js:34800:17)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async PrismaRecipientsRepository.saveWithEvents (/home/node/app/dist/modules/broadcasting/repositories/prisma/PrismaRecipientsRepository.js:41:5)
    at async RegisterEvent.execute (/home/node/app/dist/modules/broadcasting/useCases/RegisterEvent/RegisterEvent.js:60:5)
    at async RegisterEventController.handle (/home/node/app/dist/modules/broadcasting/useCases/RegisterEvent/RegisterEventController.js:90:22)
    at async /home/node/app/dist/infra/http/adapters/ExpressRouteAdapter.js:15:26 {
  code: 'P2002',
  clientVersion: '2.28.0',
  meta: { target: [ 'message_id', 'contact_id' ] }
}

Remediations

Infrastructure

Split the HTTP service that receives SNS notifications in a new service with (100mCPU, 144mb RAM and 35% CPU autoscale threshold);
Increased the amount of CPU of Queue service to 400m CPU;

Application

Do some benchmarks with Bee Queue module and different concurrency factors;
Add jobs to queue in batches instead of creating a huge jobs array;
Try to reduce the amount of queries during notification receiving;