jackjoe/mailgun_logger

Simple Mailgun log persistence in Phoenix/Elixir

mailgun_logger's Issues

Unable to delete Accounts

02:42:14.783 [info] DELETE /accounts/8
02:42:14.801 [info] Sent 500 in 18ms
02:42:14.802 [error] #PID<0.3618.0> running MailgunLoggerWeb.Endpoint (connection #PID<0.3547.0>, stream id 21) terminated
Server: :5050 (http)
Request: POST /accounts/8
** (exit) an exception was raised:
** (Ecto.ConstraintError) constraint error when attempting to delete struct:

* events_account_id_fkey (foreign_key_constraint)

If you would like to stop this constraint violation from raising an
exception and instead add it as an error to your changeset, please
call foreign_key_constraint/3 on your changeset with the constraint
:name as an option.

Getting this error when trying to delete accounts

Search filter on domain/account

image name and volume dir in service db in docker-compose.yml seems wrong

https://github.com/jackjoe/mailgun_logger/blob/master/README.md?plain=1#L120 refers to postgresql it should be postgres

https://github.com/jackjoe/mailgun_logger/blob/master/README.md?plain=1#L128 refers to /var/lib/mysql it should be /var/lib/postgresql/data

Feature request: show inbound or outbound in e-mail list

Would be nice to see if an e-mail was inbound (via routes) or outbound.

Error during import

Hi,

During import it loops over the pagination for some time before we get:

10:28:44.175 [error] Task #PID<0.3726.0> started from #PID<0.3727.0> terminating

** (FunctionClauseError) no function clause matching in MailgunLogger.Events.prepare_raw/2

(mailgun_logger 0.0.3) lib/mailgun_logger/events/events.ex:94: MailgunLogger.Events.prepare_raw(%{"delivery-status" => %{"bounce-code" => "4.4.1", "code" => 550, "description" => "This is the mail system at host xxxx@yyyy.\r\n\r\nI'm sorry to have to inform you that your message could not\r\nbe delivered to one or more recipients. It's attached below.\r\n\r\nFor further assistance, please send mail to postmaster.\r\n\r\nIf you do so, please include this problem report. You can\r\ndelete your own text from the attached returned message.\r\n\r\n The mail system\r\n\r\nxxxx@yyyy: delivery temporarily suspended: connect to\r\n xxxx@yyyy[195.105.217.14]:25: Connection refused\r\n", "message" => "X-Postfix; delivery temporarily suspended: connect to xxxx@yyyy[195.105.217.14]:25: Connection refused"}, "event" => "failed", "flags" => %{"is-delayed-bounce" => true}, "id" => "2owk2AOdTwC-sgXzCx0ZIA", "log-level" => "warn", "reason" => "bounce", "recipient" => "xxxx@yyyy", "severity" => "temporary", "timestamp" => 1597763806.283901}, 1)

(elixir 1.10.2) lib/enum.ex:1396: Enum."-map/2-lists^map/1-0-"/2

(elixir 1.10.2) lib/enum.ex:1396: Enum."-map/2-lists^map/1-0-"/2

(mailgun_logger 0.0.3) lib/mailgun_logger/events/events.ex:77: MailgunLogger.Events.save_events/2

(elixir 1.10.2) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2

(elixir 1.10.2) lib/task/supervised.ex:35: Task.Supervised.reply/5

(stdlib 3.11.2) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

Function: #Function<2.1964153/0 in MailgunLogger.run/0>

Args: []

I can't see what in that particular event is causing the issue?

DB Connection issue

I see the mailgun api get results, but then get a couple of ERRORS that look related to DB. Tried this on my Mac as well as Linux. My docker-compose.yml is using defaults listed on website.

mailgun_app | 21:27:25.503 [error] MyXQL.Connection (#PID<0.3243.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0.3706.0> timed out because it queued and checked out the connection for longer than 15000ms
mailgun_app |
mailgun_app | #PID<0.3706.0> was at location:
mailgun_app |
mailgun_app | :prim_inet.recv0/3
mailgun_app | (myxql 0.4.3) lib/myxql/client.ex:171: MyXQL.Client.recv_packets/4
mailgun_app | (myxql 0.4.3) lib/myxql/connection.ex:430: MyXQL.Connection.prepare/2
mailgun_app | (myxql 0.4.3) lib/myxql/connection.ex:81: MyXQL.Connection.handle_prepare/3
mailgun_app | (db_connection 2.2.2) lib/db_connection/holder.ex:316: DBConnection.Holder.holder_apply/4
mailgun_app | (db_connection 2.2.2) lib/db_connection.ex:1240: DBConnection.prepare/4
mailgun_app | (db_connection 2.2.2) lib/db_connection.ex:1233: DBConnection.run_prepare/4
mailgun_app | (db_connection 2.2.2) lib/db_connection.ex:1245: DBConnection.run_prepare_execute/5
mailgun_app |
mailgun_app | 21:27:25.543 [debug] QUERY ERROR db=0.0ms

...

mailgun_app | 21:27:25.543 [debug] QUERY ERROR db=0.0ms
mailgun_app | rollback []
mailgun_app | 21:27:25.546 [error] Task #PID<0.3706.0> started from #PID<0.3705.0> terminating
mailgun_app | ** (DBConnection.ConnectionError) socket closed (the connection was closed by the pool, possibly due to a timeout or because the pool has been terminated)
mailgun_app | (ecto_sql 3.4.5) lib/ecto/adapters/myxql.ex:242: Ecto.Adapters.MyXQL.insert/6
mailgun_app | (ecto 3.4.6) lib/ecto/repo/schema.ex:661: Ecto.Repo.Schema.apply/4
mailgun_app | (ecto 3.4.6) lib/ecto/repo/schema.ex:263: anonymous fn/15 in Ecto.Repo.Schema.do_insert/4
mailgun_app | (ecto 3.4.6) lib/ecto/multi.ex:646: Ecto.Multi.apply_operation/5
mailgun_app | (elixir 1.10.4) lib/enum.ex:2111: Enum."-reduce/3-lists^foldl/2-0-"/3
mailgun_app | (ecto 3.4.6) lib/ecto/multi.ex:630: anonymous fn/5 in Ecto.Multi.apply_operations/5
mailgun_app | (ecto_sql 3.4.5) lib/ecto/adapters/sql.ex:875: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
mailgun_app | (db_connection 2.2.2) lib/db_connection.ex:1427: DBConnection.run_transaction/4
mailgun_app | Function: #Function<2.33026069/0 in MailgunLogger.run/0>

view email broken

View email when text email

body-plain vs body html, implement fallback

Unable to access log issues

Is this software not available at present?

Mailgun Logger Setup

We are looking to setup MG Logger and need a little help troubleshooting.

Current steps we have taken..
Setup Linux EC2 on AWS
Installed docker and docker compose
Ran Docker Compose and copied the text from github into docker-compose.yml
Ran docker-compose up..
Got a success message with MailgunLogger running
If I go to the EC2 Public IPv4 address and add the :5050 at the end, the browser just spins.

This gave the following output...

[ec2-user@ip-172-31-46-57 ~]$ docker compose up
docker: 'compose' is not a docker command.
See 'docker --help'
[ec2-user@ip-172-31-46-57 ~]$ docker-compose up
[+] Running 2/2
 ⠿ Container ec2-user-db-1   Recreated                                     1.5s
 ⠿ Container ec2-user-web-1  Recreated                                    10.3s
Attaching to ec2-user-db-1, ec2-user-web-1
ec2-user-db-1   | 2022-04-13 11:57:41+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.28-1debian10 started.
ec2-user-db-1   | 2022-04-13 11:57:41+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
ec2-user-db-1   | 2022-04-13 11:57:41+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.28-1debian10 started.
ec2-user-db-1   | 2022-04-13T11:57:41.788115Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.28) starting as process 1
ec2-user-db-1   | 2022-04-13T11:57:41.798175Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
ec2-user-db-1   | 2022-04-13T11:57:41.979531Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
ec2-user-db-1   | 2022-04-13T11:57:42.186708Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
ec2-user-db-1   | 2022-04-13T11:57:42.186823Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
ec2-user-db-1   | 2022-04-13T11:57:42.188579Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
ec2-user-db-1   | 2022-04-13T11:57:42.222528Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.28'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server - GPL.
ec2-user-db-1   | 2022-04-13T11:57:42.223223Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
ec2-user-web-1  | |> MIGRATE_AND_SEED
ec2-user-web-1  | |> Starting dependencies...
ec2-user-web-1  | |> Starting repos...
ec2-user-web-1  | 11:57:44.357 [info] Migrations already up
ec2-user-web-1  | |> Running seed script...
ec2-user-web-1  | |> Success!
ec2-user-web-1  | 11:57:51.717 [info] Running MailgunLoggerWeb.Endpoint with cowboy 2.9.0 at 0.0.0.0:5050 (http)
ec2-user-web-1  | 11:57:51.731 [info] Access MailgunLoggerWeb.Endpoint at http://localhost

Extract out to stand-alone elixir lib

This is very timely! I'd love to see the logging bits extracted out into its own library so I can use it directly in an existing Phoenix app. Perhaps separate out the view related bits as well.

Problems with display time

Hello

What I belong to is CST time, I see on the mailgun log is 8 time, but the backstage pull time is UTC, I tried to change the mysql time zone into CST, but what mailgun_logger show or do I have any way to display the log as CST time so that I can proofread the log time correctly? Thank you!

Manual Trigger run on docker container based mailgun_logger

13:56:14.410 [error] Task #PID<0.4911.0> started from #PID<0.4909.0> terminating
** (Protocol.UndefinedError) protocol Enumerable not implemented for {:error, 502, "Bad Gateway"} of type Tuple. This protocol is implemented for the following type(s): DBConnection.PrepareStream, DBConnection.Stream, Date.Range, Ecto.Adapters.SQL.Stream, File.Stream, Function, GenEvent.Stream, HashDict, HashSet, IO.Stream, Jason.OrderedObject, List, Map, MapSet, Pager.Page, Range, Scrivener.Page, Stream
(elixir 1.13.1) lib/enum.ex:1: Enumerable.impl_for!/1
(elixir 1.13.1) lib/enum.ex:143: Enumerable.reduce/3
(elixir 1.13.1) lib/enum.ex:4144: Enum.map/2
(mailgun_logger 2202.4.3) lib/mailgun_logger/events/events.ex:112: MailgunLogger.Events.save_events/2
(mailgun_logger 2202.4.3) lib/mailgun_logger.ex:44: MailgunLogger.process_account/1
(elixir 1.13.1) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2
(elixir 1.13.1) lib/task/supervised.ex:34: Task.Supervised.reply/4
(stdlib 3.17) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Function: #Function<2.78929076/0 in MailgunLogger.run/0>
Args: []

Hi, getting this error when clicking on trigger new run.

column message_id in table events is too short

i saw errors stating that the column message_id in table events is too short (100) for some mails coming from proton.me
i fixed it with this query
alter table events alter column message_id type varchar(255) using message_id::varchar(255)

kaput

Docker startup fails because of missing S3 aws configuration (store_messages is on)

Hello,
I tried to run the docker compose setup, but it fails.
If I understand the message right, store_messages should be set to off,
otherwise, I need to configure an AWS S3 bucket.
(According README.MD)

I did not find any information how to set these configs from outside by docker environment vars.
If I check the code, it looks like, store_messages will not been read from ENV vars.

Maybe the docker-compose has a missing part,
besides the postgres typo issues POSTGRES_ instead of POSTGRESS_

mailgun_logger-web-1  | |> MIGRATE_AND_SEED
mailgun_logger-web-1  | |> Starting dependencies...
mailgun_logger-web-1  | |> Starting repos...
mailgun_logger-web-1  | |> Running seed script...
mailgun_logger-web-1  | 13:21:30.110 [info] Migrations already up
mailgun_logger-web-1  | |> Success!
mailgun_logger-web-1  | 13:21:36.777 [notice] Application mailgun_logger exited: exited in: MailgunLogger.Application.start(:normal, [])
mailgun_logger-web-1  |     ** (EXIT) an exception was raised:
mailgun_logger-web-1  |         ** (RuntimeError) If `store_messages` is on, you need to configure ex_aws
mailgun_logger-web-1  |             (mailgun_logger 2308.1.1) lib/mailgun_logger/application.ex:33: MailgunLogger.Application.check_store_raw/0
mailgun_logger-web-1  |             (mailgun_logger 2308.1.1) lib/mailgun_logger/application.ex:9: MailgunLogger.Application.start/2
mailgun_logger-web-1  |             (kernel 8.5.4.1) application_master.erl:293: :application_master.start_it_old/4
mailgun_logger-web-1  | {"Kernel pid terminated",application_controller,"{application_start_failure,mailgun_logger,{bad_return,{{'Elixir.MailgunLogger.Application',start,[normal,[]]},{'EXIT',{#{'__exception__' => true,'__struct__' => 'Elixir.RuntimeError',message => 
mailgun_logger-web-1  | <<\"If `store_messages` is on, you n
mailgun_logger-web-1  | eed to configure ex_aws\">>},[{'Elixir.MailgunLogger.Application',check_store_raw,0,[{file,\"lib/mailgun_logger/application.ex\"},{line,33},{error_info,#{module => 'Elixir.Exception'}}]},{'Elixir.MailgunLogger.Application',start,2,[{file,\"lib/mailgun_logger/application.ex\"},{line,9}]},{application_master,start_it_old,4,[{file,\"application_master.erl\"},{line,293}]}]}}}}}"}
mailgun_logger-web-1  | Kernel pid terminated (application_controller) ({application_start_failure,mailgun_logger,{bad_return,{{'Elixir.MailgunLogger.Applica
mailgun_logger-web-1  | tion',start,[normal,[]]},{'EXIT',{#{'__exception__' => true,'__struct__' => 'Elixir.RuntimeError',message => <<"If `store_messages` is on, you need to configure ex_aws">>},[{'Elixir.MailgunLogger.Application',check_store_raw,0,[{file,"lib/mailgun_logger/application.ex"},{line,33},{error_info,#{module => 'Elixir.Exception'}}]},{'Elixir.MailgunLogger.Application',start,2,[{file,"lib/mailgun_logger/application.ex"},{line,9}]},{application_master,start_it_old,4,[{file,"application_master.erl"},{line,293}]}]}}}}})
mailgun_logger-web-1  | 
mailgun_logger-web-1  | Crash dump is being written to: erl_crash.dump...
mailgun_logger-web-1  | done
mailgun_logger-web-1 exited with code 1

trim filter fields

when using the filter fields, the value is not trimmed. So searching for @domain.be (note the whitespace), no matches are found.

graphs page error

date format msyql function port

error when viewing stored message

when viewing some mails i get "Something went wrong"

web_1 | 13:18:29.959 [info] GET /events/1838/stored_message
web_1 | 13:18:30.170 [info] Sent 500 in 210ms
web_1 | 13:18:30.170 [error] #PID<0.8130.0> running MailgunLoggerWeb.Endpoint (connection #PID<0.8129.0>, stream id 1) terminated
web_1 | Server: servername:80 (http)
web_1 | Request: GET /events/1838/stored_message
web_1 | ** (exit) an exception was raised:
web_1 | ** (FunctionClauseError) no function clause matching in String.contains?/2
web_1 | (elixir 1.15.7) lib/string.ex:2538: String.contains?(nil, "text/plain")
web_1 | (mailgun_logger 2312.4.0) lib/mailgun/events.ex:73: Mailgun.Events.get_stored_message_content/1
web_1 | (mailgun_logger 2312.4.0) lib/mailgun_logger_web/controllers/event_controller.ex:31: MailgunLoggerWeb.EventController.stored_message/2
web_1 | (mailgun_logger 2312.4.0) lib/mailgun_logger_web/controllers/event_controller.ex:1: MailgunLoggerWeb.EventController.action/2
web_1 | (mailgun_logger 2312.4.0) lib/mailgun_logger_web/controllers/event_controller.ex:1: MailgunLoggerWeb.EventController.phoenix_controller_pipeline/2
web_1 | (phoenix 1.7.7) lib/phoenix/router.ex:430: Phoenix.Router.call/5
web_1 | (mailgun_logger 2312.4.0) deps/plug/lib/plug/error_handler.ex:80: MailgunLoggerWeb.Router.call/2
web_1 | (mailgun_logger 2312.4.0) lib/mailgun_logger_web/endpoint.ex:1: MailgunLoggerWeb.Endpoint.plug_builder_call/2

the stored message is just plain txt nothing fancy, other mails work fine.

Password reset not working

Installing via docker compose

Hi! Thank you very much for this project.
I am trying to install it on my server via docker compose. However I can't get this to work.
Could you point me in the right direction?

here is my log:

> db_1   | 
db_1   | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1   | 
db_1   | 2020-10-05 15:32:12.877 UTC [1] LOG:  starting PostgreSQL 13.0 (Debian 13.0-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1   | 2020-10-05 15:32:12.878 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1   | 2020-10-05 15:32:12.878 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1   | 2020-10-05 15:32:12.939 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1   | 2020-10-05 15:32:12.975 UTC [25] LOG:  database system was shut down at 2020-10-05 15:31:51 UTC
db_1   | 2020-10-05 15:32:12.995 UTC [1] LOG:  database system is ready to accept connections
web_1  | |> MIGRATE_AND_SEED
web_1  | |> Starting dependencies...
web_1  | |> Starting repos...
web_1  | 15:32:15.212 [error] MyXQL.Connection (#PID<0.217.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:15.212 [error] MyXQL.Connection (#PID<0.218.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:16.364 [error] MyXQL.Connection (#PID<0.217.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:17.981 [error] MyXQL.Connection (#PID<0.218.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:18.182 [error] Could not create schema migrations table. This error usually happens due to the following:
web_1  | 
web_1  |   * The database does not exist
web_1  |   * The "schema_migrations" table, which Ecto uses for managing
web_1  |     migrations, was defined by another library
web_1  |   * There is a deadlock while migrating (such as using concurrent
web_1  |     indexes with a migration_lock)
web_1  | 
web_1  | To fix the first issue, run "mix ecto.create".
web_1  | 
web_1  | To address the second, you can run "mix ecto.drop" followed by
web_1  | "mix ecto.create". Alternatively you may configure Ecto to use
web_1  | another table for managing migrations:
web_1  | 
web_1  |     config :mailgun_logger, MailgunLogger.Repo,
web_1  |       migration_source: "some_other_table_for_schema_migrations"
web_1  | 
web_1  | The full error report is shown below.
web_1  | 
web_1  | ** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2970ms. This means requests are coming in and your connection pool cannot serve them fast enough. You can address this by:
web_1  | 
web_1  |   1. By tracking down slow queries and making sure they are running fast enough
web_1  |   2. Increasing the pool_size (albeit it increases resource consumption)
web_1  |   3. Allow requests to wait longer by increasing :queue_target and :queue_interval
web_1  | 
web_1  | See DBConnection.start_link/2 for more information
web_1  | 
web_1  |     (ecto_sql 3.4.5) lib/ecto/adapters/sql.ex:593: Ecto.Adapters.SQL.raise_sql_call_error/1
web_1  |     (elixir 1.10.2) lib/enum.ex:1396: Enum."-map/2-lists^map/1-0-"/2
web_1  |     (ecto_sql 3.4.5) lib/ecto/adapters/sql.ex:686: Ecto.Adapters.SQL.execute_ddl/4
web_1  |     (ecto_sql 3.4.5) lib/ecto/migrator.ex:641: Ecto.Migrator.verbose_schema_migration/3
web_1  |     (ecto_sql 3.4.5) lib/ecto/migrator.ex:484: Ecto.Migrator.lock_for_migrations/4
web_1  |     (ecto_sql 3.4.5) lib/ecto/migrator.ex:406: Ecto.Migrator.run/4
web_1  |     (ecto_sql 3.4.5) lib/ecto/migrator.ex:149: Ecto.Migrator.with_repo/3
web_1  |     (mailgun_logger 0.0.4) lib/release_tasks.ex:51: anonymous fn/2 in JackJoe.ReleaseTasks._migrate/0
web_1  | 15:32:23.943 [info] [swarm on production@4be8c176bbca] [tracker:init] started
web_1  | 15:32:24.059 [error] MyXQL.Connection (#PID<0.3414.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:24.059 [error] MyXQL.Connection (#PID<0.3413.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:24.059 [error] MyXQL.Connection (#PID<0.3408.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:24.060 [error] MyXQL.Connection (#PID<0.3407.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:24.060 [error] MyXQL.Connection (#PID<0.3412.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:24.060 [error] MyXQL.Connection (#PID<0.3411.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:24.060 [error] MyXQL.Connection (#PID<0.3418.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:24.060 [error] MyXQL.Connection (#PID<0.3417.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:24.061 [error] MyXQL.Connection (#PID<0.3416.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:24.061 [error] MyXQL.Connection (#PID<0.3415.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:24.063 [info] Running MailgunLoggerWeb.Endpoint with cowboy 2.8.0 at 0.0.0.0:5050 (http)
web_1  | 15:32:24.064 [info] Access MailgunLoggerWeb.Endpoint at http://localhost
web_1  | 15:32:24.064 [debug] [:production@4be8c176bbca][Elixir.Quantum.JobBroadcaster] Loading Initial Jobs from Config
web_1  | 15:32:24.064 [debug] [:production@4be8c176bbca][Elixir.Quantum.ExecutionBroadcaster] Unknown last execution time, using now
web_1  | 15:32:24.064 [debug] [:production@4be8c176bbca][Elixir.Quantum.ExecutionBroadcaster] Adding job #Reference<0.912682946.2221932548.182105>
web_1  | 15:32:24.065 [debug] [:production@4be8c176bbca][Elixir.Quantum.ExecutionBroadcaster] Adding job #Reference<0.912682946.2221932548.182114>
web_1  | 15:32:24.065 [debug] [:production@4be8c176bbca][Elixir.Quantum.ExecutionBroadcaster] Continuing Execution Broadcasting at -576405090210 (2020-10-06T07:00:00)
web_1  | 15:32:25.429 [error] MyXQL.Connection (#PID<0.3413.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:25.495 [debug] Processing with MailgunLoggerWeb.PingController.ping/2
web_1  |   Parameters: %{}
web_1  |   Pipelines: [:ping]
web_1  | 15:32:25.953 [error] MyXQL.Connection (#PID<0.3408.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused
web_1  | 15:32:25.994 [error] MyXQL.Connection (#PID<0.3417.0>) failed to connect: ** (DBConnection.ConnectionError) connection refused

Thank you!

EDIT: @damianmytek I just wrapped your log output into the backtick thing, I did not redact anything else.

Create index for a fresh start not working

16:45:33.115 [info] create index events_message_to_index
** (MyXQL.Error) (1071) Specified key was too long; max key length is 3072 bytes
(ecto_sql 3.7.2) lib/ecto/adapters/sql.ex:760: Ecto.Adapters.SQL.raise_sql_call_error/1
(elixir 1.13.1) lib/enum.ex:1593: Enum."-map/2-lists^map/1-0-"/2
(ecto_sql 3.7.2) lib/ecto/adapters/sql.ex:852: Ecto.Adapters.SQL.execute_ddl/4
(ecto_sql 3.7.2) lib/ecto/migration/runner.ex:343: Ecto.Migration.Runner.log_and_execute_ddl/3
(ecto_sql 3.7.2) lib/ecto/migration/runner.ex:117: anonymous fn/6 in Ecto.Migration.Runner.flush/0
(elixir 1.13.1) lib/enum.ex:2396: Enum."-reduce/3-lists^foldl/2-0-"/3
(ecto_sql 3.7.2) lib/ecto/migration/runner.ex:116: Ecto.Migration.Runner.flush/0
(stdlib 3.17) timer.erl:166: :timer.tc/1

bamboo

No emails / bamboo for now? @pierot

errors in red

Something went wrong

Using both the compose information provided and running the containers manually I get the following error when trying to access the logger:

17:47:03.697 [info] GET /setup
17:47:03.700 [info] Sent 500 in 3ms
17:47:03.701 [error] #PID<0.5344.0> running MailgunLoggerWeb.Endpoint (connection #PID<0.5342.0>, stream id 2) terminated
Server: 127.0.0.1:5050 (http)
Request: GET /setup
** (exit) an exception was raised:
** (ArgumentError) assign @view_module not available in template.
Please make sure all proper assigns have been set. If this
is a child template, ensure assigns are given explicitly by
the parent template as they are not automatically forwarded.
Available assigns: [:changeset, :conn, :inner_content]
(phoenix_html 3.2.0) lib/phoenix_html/engine.ex:187: Phoenix.HTML.Engine.fetch_assign!/2
(mailgun_logger 2022.3.0) lib/mailgun_logger_web/templates/layout/setup.html.eex:2:
MailgunLoggerWeb.LayoutView."setup.html"/1
(phoenix_view 1.1.0) lib/phoenix/view.ex:473: Phoenix.View.render_to_iodata/3
(phoenix 1.6.6) lib/phoenix/controller.ex:772: Phoenix.Controller.render_and_send/4
(mailgun_logger 2022.3.0) lib/mailgun_logger_web/controllers/setup_controller.ex:1:
MailgunLoggerWeb.SetupController.action/2
(mailgun_logger 2022.3.0) lib/mailgun_logger_web/controllers/setup_controller.ex:1:
MailgunLoggerWeb.SetupController.phoenix_controller_pipeline/2
(phoenix 1.6.6) lib/phoenix/router.ex:355: Phoenix.Router.call/2
(mailgun_logger 2022.3.0) lib/plug/error_handler.ex:80: MailgunLoggerWeb.Router.call/2

I'm afraid, I don't know enough about the underlying technologies to determine what's happening. If I'm missing something please ask and I will do my best to provide details.

DB issue.

I have issue in downloading logs to db despite new trigger. I see success message "Run triggered successfully. Data should be coming in"... So far no data in the db.

Best Regards,

21:09:22.511 [info] GET /events
21:09:22.516 [info] Sent 200 in 5ms
21:09:24.271 [info] GET /events
21:09:24.279 [info] Sent 200 in 8ms
21:09:25.118 [info] GET /
21:09:25.122 [info] Sent 200 in 3ms
21:09:26.874 [info] POST /trigger_run
21:09:26.877 [info] Sent 302 in 2ms
21:09:26.895 [info] GET /
21:09:26.898 [info] Sent 200 in 2ms
21:09:30.212 [info] GET /events
21:09:30.218 [info] Sent 200 in 6ms
21:11:54.871 [error] Task #PID<0.17811.0> started from #PID<0.17810.0> terminating
** (MyXQL.Error) (1406) Data too long for column 'message_subject' at row 1
(ecto_sql 3.6.1) lib/ecto/adapters/myxql.ex:270: Ecto.Adapters.MyXQL.insert/6
(ecto 3.6.1) lib/ecto/repo/schema.ex:717: Ecto.Repo.Schema.apply/4
(ecto 3.6.1) lib/ecto/repo/schema.ex:342: anonymous fn/15 in Ecto.Repo.Schema.do_insert/4
(ecto 3.6.1) lib/ecto/multi.ex:716: Ecto.Multi.apply_operation/5
(elixir 1.11.0) lib/enum.ex:2181: Enum."-reduce/3-lists^foldl/2-0-"/3
(ecto 3.6.1) lib/ecto/multi.ex:690: anonymous fn/5 in Ecto.Multi.apply_operations/5
(ecto_sql 3.6.1) lib/ecto/adapters/sql.ex:1005: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
(db_connection 2.4.0) lib/db_connection.ex:1512: DBConnection.run_transaction/4
Function: #Function<2.19295648/0 in MailgunLogger.run/0>
Args: []
21:53:25.518 [info] GET /events
21:53:25.522 [info] Sent 200 in 3ms
21:54:12.898 [info] GET /
21:54:12.903 [info] Sent 200 in 4ms
21:54:15.396 [info] GET /events
21:54:15.401 [info] Sent 200 in 5ms
21:54:18.495 [info] GET /
21:54:18.499 [info] Sent 200 in 4ms
21:54:20.209 [info] POST /trigger_run
21:54:20.210 [info] Sent 302 in 1ms
21:54:20.222 [info] GET /
21:54:20.224 [info] Sent 200 in 1ms
21:57:20.211 [error] Task #PID<0.19744.0> started from #PID<0.19743.0> terminating
** (stop) exited in: Task.await(%Task{owner: #PID<0.19744.0>, pid: #PID<0.19745.0>, ref: #Reference<0.2477322715.1366294529.219296>}, 180000)
** (EXIT) time out
(elixir 1.11.0) lib/task.ex:643: Task.await/2
(elixir 1.11.0) lib/enum.ex:1399: Enum."-map/2-lists^map/1-0-"/2
(mailgun_logger 0.2.0) lib/mailgun_logger.ex:27: MailgunLogger.run/0
(elixir 1.11.0) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2
(stdlib 3.13.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Function: &MailgunLogger.run/0
Args: []
21:58:47.518 [info] GET /events
21:58:47.520 [info] Sent 200 in 1ms

message to field needs to be larger erros

Hi, not sure it is something wrong with mail gun but i received this error recently:

19:14:50.165 [error] Task #PID<0.16064.0> started from #PID<0.16056.0> terminating
** (MyXQL.Error) (1406) Data too long for column 'message_to' at row 1
    (ecto_sql 3.7.2) lib/ecto/adapters/myxql.ex:274: Ecto.Adapters.MyXQL.insert/6
    (ecto 3.7.2) lib/ecto/repo/schema.ex:744: Ecto.Repo.Schema.apply/4
    (ecto 3.7.2) lib/ecto/repo/schema.ex:367: anonymous fn/15 in Ecto.Repo.Schema.do_insert/4
    (ecto 3.7.2) lib/ecto/multi.ex:716: Ecto.Multi.apply_operation/5
    (elixir 1.13.1) lib/enum.ex:2396: Enum."-reduce/3-lists^foldl/2-0-"/3
    (ecto 3.7.2) lib/ecto/multi.ex:690: anonymous fn/5 in Ecto.Multi.apply_operations/5
    (ecto_sql 3.7.2) lib/ecto/adapters/sql.ex:1021: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
    (db_connection 2.4.2) lib/db_connection.ex:1562: DBConnection.run_transaction/4
Function: #Function<2.78929076/0 in MailgunLogger.run/0>
    Args: []
    
    Increasing the size of the message to field in the events table solved it for now.