zenika/lilaobot

A bot used to grab calls for offers from our customers and make them visible in slack channels

Welcome to lilaobot 👋

lilaobot allows call for offers from various sources (currently OBMS and Little Big Connection are envisionned) to be gathered from sources and to be published into Slack channel

Work in progress 🚧

Setup

Deployment

On each commit, Github Action (.github/workflows/main.yml) applies Terraform conf (located at ./terraform), to deploy the GCP Functions (located at ./functions)

Terraform state is saved in a Google Cloud Storage named lilaobot-terraform-state

Requirements

• a GCS bucket to store Terraform's state
• a GCP Service Account, with following roles:
  • Cloud Function Developer
  • Service Account User
  • Storage Object Admin
  • Storage Admin
• Secrets on Github repository:
  • GCP_SA_KEY (service account credentials, a JSON file)
  • GCP_PROJECT (project id)
• Create an OAuth Client ID Credentials through GCP Console -> APIS & Services -> Credentials
• Secrets on GCP through GCP Console -> Security -> Secret Manager:
  • oauth2-client-id: Client ID from previously created OAuth Credentials
  • oauth2-client-secret: Client ID from previously created OAuth Credentials
  • slack-bot-token: The token of the Slackbot Application (https://api.slack.com/authentication/basics)

Resources

This project is based on this article from Google, without the image part.

Below, the goal of this actual project:

How to Use

After deployment you can activate the bot through this URLs : https://europe-west1-lilaobot.cloudfunctions.net/oauth2init

BEWARE This will send all your new mails into Zenika Slack channel named project-lilaobot

NOTE This URLs is retrievable in the GCP Console -> Cloud Functions -> Trigger -> HTTP.

Once this url is invoked (and authorization is granted) the function is authorized to read user mails.

How does it works ?

Two main flows are visible here: initialization and transfer of message

Author

👤 Logan Hauspie

• Twitter: @lhauspie
• Github: @lhauspie

🤝 Contributing

Contributions, issues and feature requests are welcome!
Feel free to check issues page.

Show your support

Give a ⭐️ if this project helped you!

---

This README was generated with ❤️ by readme-md-generator

lilaobot's Issues

create license file

Decide and create the license file

oauth module is not tested

Actually oauth module is not tested, it should be, maybe by mocking libraries with example data (retrieve from real world usage).
It will greatly help understanding the code.

Deploy lilaobot

Should be deployed in Zenika GCP services

Read all emails from last historyId after recieving Pub/Sub message

Pub/Sub Message contains the email address and an historyId. Today we only read the last recieved email. By this way we might miss some email if several emails are recieved ate the almost same time.

It could be nice to read all the recieved email since the last historyId becaus an historyId can represent a batch of emails.

Convert last Little Big Connection offers into RSS

Should read the content of Little Big Connection with a login that has rights to get offers

Que pensez-vous d'une archi de ce style ?

Merge slack-publisher and onNewMessage, to have the full workflow.

Reminder :

• delete the slack-publisher
• change the terraform to not deploy it
• delete the Cloud Function in GCP

endWatch http endpoint doesn't exist in Google Cloud Functions

Seems like there is some kind of terraforming I forgot

Messaged duplicated for forwarded mails

Messages are duplicated only when mails are coming from a forward.

Cleanup branches !

There are at least ten branches which are not currently merged in main.
Can you please remove the branches you have created that contain code merged into main (be it directly or through a cherry-pick or any other cumbersome git-fuckery ?

Branches to analyze are

• @Le-Bit by Cyril Labitte origin/feat/prettier
• @Le-Bit by Cyril Labitte origin/fix/rework-testing
• @AcevedoR origin/fix/fix-gmail-api-mappings
• @AcevedoR origin/fix/fix-onNewMessage
• @AcevedoR origin/fix/config-import-broken
• @AcevedoR origin/feat/onNewMessage-impl
• @AcevedoR origin/fix/callback-save-token
• @AcevedoR origin/docs/add-architecture-diagram
• @AcevedoR origin/feat/adding-initWatch-function
• @AcevedoR origin/fix/callback-function
• @AcevedoR origin/feat/implementing-callback-function
• @sixdouglas origin/master-backup-with-douglas-work
• @sixdouglas origin/fix/gmailNotifierStep1

Can you either deet branch remotely or comment with the PR linked to this branch ?

Error when there is only one message in list

Since #46 implementation, we read a pageToken.
But when here is only one message, pageToken is not sent by GMail API, and as a consequence listMessages fails.

Error is

TypeError: Cannot read property 'pageToken' of undefined at Object.exports.listMessages (/workspace/lib/gmail-api-client.js:67:35) at exports.onNewMessage (/workspace/index.js:119:43) at processTicksAndRejections (internal/process/task_queues.js:95:5)

Fix dependabot alerts

There is some dependabot alert about aiohttp. It might be nice to fix it.

Renew Gmail token with scheduled job

The purpose of this scheduled job is to renew periodically the initial authorisation given by the google user.

Add alerting on errors

We are not alerted when there is errors in GCP logs

Retrieve configuration from GCP Secrets

Today the configuration in centralized in files named config.js and config.json.

It could be nice to replace these files by a GCP Secrets to avoid configuration in code base.

Message may be received multiple times

I have sent a mail to lilaobot email address, which has been sent 4 times to Slack. maybe we could avoid that, no?

Write a readme

because we have none

Disable gmail watching

Allow users that previously gived the authorization to wtahc his/her email box to disable the watcher.

IaC: get:secrets on Function Service Account

I had to mannualy add permission to access secrets on the default Service Account used to execute the Function :

gcloud secrets add-iam-policy-binding slack-bot-token \
    --role roles/secretmanager.secretAccessor \
    --member serviceAccount:[email protected]

It could be handled by Terraform, or be a requirement.

GCP Quotas: insure 'Free tiers' quotas are respected

It would be a good idea to enforce quotas to limit the project to free usage, especially on storage

Setup the gmail inbox, and tweak the code to only get mails with specific label.

• Add gmail filter to add label automatically
• Update the watcher code to set the corresponding label

Cannot read HTML mails

• Actually if we receive a mail with HTML, process will fail.
• Slack does not support HTML

Stop using a pure gmail address

In our context, we should use a @zenika.com email address.

I'm currently discussing with internal IT department to have one provisionned

Implement the Watch Renewer, the refresh token scheduler.

The purpose of this scheduled job is to renew periodically the initial authorization given by the google user at init-time.