elixir-ecto/ecto_sql

SQL-based adapters for Ecto and database migrations

Ecto SQL

Ecto SQL (documentation) provides building blocks for writing SQL adapters for Ecto. It features:

• The Ecto.Adapters.SQL module as an entry point for all SQL-based adapters
• Default implementations for Postgres (Ecto.Adapters.Postgres), MySQL (Ecto.Adapters.MyXQL), and MSSQL (Ecto.Adapters.Tds)
• A test sandbox (Ecto.Adapters.SQL.Sandbox) that concurrently runs database tests inside transactions
• Support for database migrations via Mix tasks

To learn more about getting started, see the Ecto repository.

Running tests

Clone the repo and fetch its dependencies:

$ git clone https://github.com/elixir-ecto/ecto_sql.git
$ cd ecto_sql
$ mix deps.get
$ mix test.all

Note that mix test.all runs the tests in test/ and the integration_tests for each adapter: pg, myxql and tds.

You can also use a local Ecto checkout if desired:

$ ECTO_PATH=../ecto mix test.all

You can run tests against a specific Ecto adapter by using the ECTO_ADAPTER environment variable:

$ ECTO_ADAPTER=pg mix test

MySQL and PostgreSQL can be installed directly on most systems. For MSSQL, you may need to run it as a Docker image:

docker run -d -p 1433:1433 --name mssql -e 'ACCEPT_EULA=Y' -e 'MSSQL_SA_PASSWORD=some!Password' mcr.microsoft.com/mssql/server:2017-latest

Running containerized tests

It is also possible to run the integration tests under a containerized environment using earthly:

$ earthly -P +all

You can also use this to interactively debug any failing integration tests using the corresponding commands:

$ earthly -P -i --build-arg ELIXIR_BASE=1.8.2-erlang-20.3.8.26-alpine-3.11.6 --build-arg MYSQL=5.7 +integration-test-mysql
$ earthly -P -i --build-arg ELIXIR_BASE=1.8.2-erlang-20.3.8.26-alpine-3.11.6 --build-arg MSSQL=2019  +integration-test-mssql
$ earthly -P -i --build-arg ELIXIR_BASE=1.8.2-erlang-20.3.8.26-alpine-3.11.6 --build-arg POSTGRES=11.11 +integration-test-postgres

Then once you enter the containerized shell, you can inspect the underlying databases with the respective commands:

PGPASSWORD=postgres psql -h 127.0.0.1 -U postgres -d postgres ecto_test
MYSQL_PASSWORD=root mysql -h 127.0.0.1 -uroot -proot ecto_test
sqlcmd -U sa -P 'some!Password'

License

Copyright (c) 2012 Plataformatec
Copyright (c) 2020 Dashbit

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

ecto_sql's Issues

Cannot use Ecto to write to timestamptz columns

Precheck

• investigated issue with timestamp vs timestamptz columns, and am aware of (and agree with) Ecto just using timestamp
• checked on IRC channel to see if solution other than custom extensions is possible

Environment

• Elixir 1.7.3 on OTP 21
• Postgres 10.5
• Ecto 3.01 / ecto_sql 3.0.0
• postgrex 0.14
• Operating system: Fedora 28

Current behavior

If I have a legacy table (defined in another system) with a column of type timestamptz, I cannot use Ecto to write to that column.

    TzTest.changeset(%TzTest{}, %{tz_col: DateTime.utc_now()}) |> Repo.insert()

results in this in the log:

11:44:59.028 [debug] QUERY ERROR db=13.7ms queue=1.1ms
INSERT INTO "tz_test" ("tz_col") VALUES ($1) RETURNING "id" [~N[2018-11-05 16:44:58.996802]]

even though schema is defined as

  schema "tz_test" do
    field(:tz_col, :utc_datetime_usec)
  end

Full source available in test project at https://github.com/jamesvl/ecto_tz_test

Expected behavior

Insert or updates write a DateTime value.

Public function removal in minor version

Why was this function removed in a minor version?

https://github.com/elixir-ecto/ecto_sql/blob/v3.1.1/lib/mix/ecto_sql.ex#L8

When running multiple migrations and one fails with a CompileError, schema_migrations is not updated

Precheck

• Do not use the issues tracker for help or support requests (try Elixir Forum, Stack Overflow, IRC or mailing lists, etc).
• For proposing a new feature, please start a discussion on elixir-ecto.
• For bugs, do a quick search and make sure the bug has not yet been reported.
• Finally, be nice and have fun!

Environment

• Elixir version (elixir -v): 1.7.4
• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): PostgreSQL 10.4, 11.1
• Ecto version (mix deps): 3.0.6 (ecto_sql 3.0.4)
• Database adapter and version (mix deps): postgrex 0.14.1
• Operating system: macOS Mojave

Current behavior

Create a new project and add two new migrations:

$ mix phx.new hello
$ cd hello
$ mix phx.gen.schema User users
$ mix phx.gen.schema Account accounts

Edit the second migration, priv/repo/migrations/*_create_accounts.exs, and change "create table" to "create_table":

defmodule Hello.Repo.Migrations.CreateAccounts do
  use Ecto.Migration

  def change do
    create_table(:accounts) do
            timestamps()
    end
  end
end

Then create the database and run the migrations:

$ mix ecto.create
$ mix ecto.migrate

You'll get a CompileError, similar to this:

[info] == Running 20190111231353 Hello.Repo.Migrations.CreateUsers.change/0 forward
[info] create table users
[info] create index users_email_index
[info] == Migrated 20190111231353 in 0.0s
** (CompileError) priv/repo/migrations/20190111231403_create_accounts.exs:12: undefined function create_table/2
    (stdlib) lists.erl:1338: :lists.foreach/2
    (stdlib) erl_eval.erl:680: :erl_eval.do_apply/6
    (elixir) lib/code.ex:715: Code.load_file/2
    (ecto_sql) lib/ecto/migrator.ex:476: Ecto.Migrator.extract_module/2
    (ecto_sql) lib/ecto/migrator.ex:433: anonymous fn/5 in Ecto.Migrator.do_migrate/4
    (elixir) lib/enum.ex:3281: Enumerable.List.reduce/3
    (elixir) lib/enum.ex:1968: Enum.reduce_while/3
    (ecto_sql) lib/ecto/migrator.ex:427: Ecto.Migrator.migrate/4
    (ecto_sql) lib/ecto/adapters/sql.ex:814: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
    (db_connection) lib/db_connection.ex:1349: DBConnection.run_transaction/4
    (ecto_sql) lib/ecto/adapters/sql.ex:723: Ecto.Adapters.SQL.lock_for_migrations/5
    (ecto_sql) lib/ecto/migrator.ex:318: Ecto.Migrator.lock_for_migrations/3
    (ecto_sql) lib/mix/tasks/ecto.migrate.ex:110: anonymous fn/4 in Mix.Tasks.Ecto.Migrate.run/2

If you check the database, you'll see the users table was created, but schema_migrations has no rows in it. If you fix the error and run the migration again, it'll fail because users already exists.

$ mix ecto.migrate
[info] == Running 20190111231353 Hello.Repo.Migrations.CreateUsers.change/0 forward
[info] create table users
** (Postgrex.Error) ERROR 42P07 (duplicate_table) relation "users" already exists
    (ecto_sql) lib/ecto/adapters/sql.ex:620: Ecto.Adapters.SQL.raise_sql_call_error/1
    (elixir) lib/enum.ex:1314: Enum."-map/2-lists^map/1-0-"/2
    (ecto_sql) lib/ecto/adapters/sql.ex:707: Ecto.Adapters.SQL.execute_ddl/4
    (ecto_sql) lib/ecto/migration/runner.ex:301: Ecto.Migration.Runner.log_and_execute_ddl/3
    (ecto_sql) lib/ecto/migration/runner.ex:110: anonymous fn/2 in Ecto.Migration.Runner.flush/0
    (elixir) lib/enum.ex:1925: Enum."-reduce/3-lists^foldl/2-0-"/3
    (ecto_sql) lib/ecto/migration/runner.ex:108: Ecto.Migration.Runner.flush/0
    (stdlib) timer.erl:166: :timer.tc/1
    (ecto_sql) lib/ecto/migration/runner.ex:26: Ecto.Migration.Runner.run/7
    (ecto_sql) lib/ecto/migrator.ex:211: Ecto.Migrator.attempt/7
    (ecto_sql) lib/ecto/migrator.ex:113: anonymous fn/4 in Ecto.Migrator.do_up/4
    (ecto_sql) lib/ecto/migrator.ex:193: anonymous fn/3 in Ecto.Migrator.run_maybe_in_transaction/5
    (ecto_sql) lib/ecto/adapters/sql.ex:814: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
    (db_connection) lib/db_connection.ex:1349: DBConnection.run_transaction/4
    (ecto_sql) lib/ecto/migrator.ex:192: Ecto.Migrator.run_maybe_in_transaction/5
    (elixir) lib/task/supervised.ex:89: Task.Supervised.do_apply/2
    (elixir) lib/task/supervised.ex:38: Task.Supervised.reply/5
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

Expected behavior

I would expect schema_migrations to be updated after the first migration but prior to the second migration being loaded and failing to compile.

Dialyzer complains that type `Ecto.Adapter.on_conflict` does not exist.

Environment

Elixir 1.8.1
Ecto SQL 3.0.5
Ecto 3.0.7

Current behavior

Dialyzer complains that type Ecto.Adapter.on_conflict does not exist.

See the type being used here.

Expected behavior

The type should be fixed.

Make it possible to use log functions

Could it be possible to make the log functions available so you can format the data you receive from telemetry?
I would like to reformat the uuids before printing the logging.
But since you can no longer update the data you can no longer make use of logging of the printing code from this library.
So now I have to copy all the code from this library with log: false setting. And print the logging myself with this code and all the functions that are referenced in it.

Logger.log(
          level,
          fn -> log_iodata(measurements, metadata) end,
          ansi_color: sql_color(query_string)
        )

ArgumentError in Ecto.Repo.Registry.lookup

Environment

• Elixir version: Elixir 1.8.2 (compiled with Erlang/OTP 21)
• Erlang version: OTP-22.0.3
• Database and version: PostgreSQL 11.3
• Ecto version: 3.1.5
• Database adapter and version: postgrex 0.14.3
• Operating system: macOS 10.14.6

Current behavior

In an older application that has a test_helper.exs file like this one (Phoenix 1.1.x generator):

ExUnit.start
Mix.Task.run "ecto.create", ["--quiet"]
Mix.Task.run "ecto.migrate", ["--quiet"]
Ecto.Adapters.SQL.Sandbox.mode(MyApp.Repo, :manual)

The first time tests are run after a new migration is added, the run fails with this stacktrace:

$ mix test
Generated myapp app
** (ArgumentError) argument error
    (stdlib) :ets.lookup_element(Ecto.Repo.Registry, #PID<0.1011.0>, 3)
    (ecto) lib/ecto/repo/registry.ex:18: Ecto.Repo.Registry.lookup/1
    (ecto) lib/ecto/adapter.ex:127: Ecto.Adapter.lookup_meta/1
    (ecto_sql) lib/ecto/adapters/sql/sandbox.ex:486: Ecto.Adapters.SQL.Sandbox.lookup_meta!/1
    (ecto_sql) lib/ecto/adapters/sql/sandbox.ex:387: Ecto.Adapters.SQL.Sandbox.mode/2

Expected behavior

Ideally the repo should still be up and usable after the Ecto migration runs, or there could be a more helpful error message. Since the migration is silent, this showed up as flaky tests that were quite hard to track down.

mix ecto.migrate -n argument has no effect

Environment

• Elixir version (elixir -v):

  Erlang/OTP 21 [erts-10.2] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [hipe] [dtrace]
  
  Elixir 1.7.4 (compiled with Erlang/OTP 21)
  

• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): PostgreSQL v11.1
• Ecto version (mix deps): 3.0.2, ecto_sql 3.0
• Database adapter and version (mix deps): postgrex v0.14.1
• Operating system: macOS

Current behavior

With multiple pending migrations, run mix ecto.migrate -n 1. Rather than only running a single migration, it runs all pending migrations.

Expected behavior

The -n 1 flag should work and cause Ecto to only migrate up by a single version, rather than running all pending migrations.

Wording in docs is slightly confusing...

I feel like the docs here are a bit confusing.

I think this could be read as execute/2 is one of those commands that are non-reversible in change/0 (even though it provides an up/0 and down/0).

This confused me quite a bit, actually, and I wasted a bunch of time trying to find some examples of execute/2's usage in migrations.

Thoughts? Should we make it more clear?

Or am I just terrible at reading comprehension? 🤣

mix ecto.rollback throws an error

Environment

• Elixir version (elixir -v): master
• Database and version: PostgreSQL 10.5
• Ecto version (mix deps): master
• Database adapter and version (mix deps): postgrex master

Current behavior

hexpm[wm-ecto-3]% mix ecto.rollback
[debug] QUERY OK source="schema_migrations" db=1.1ms
SELECT s0."version"::bigint FROM "schema_migrations" AS s0 FOR UPDATE []
[info] == Running Hexpm.Repo.Migrations.AddServiceToUsers.down/0 forward
[info] execute "DELETE FROM users WHERE username = 'hexdocs'"
[info] execute "ALTER TABLE users ALTER password SET NOT NULL"
[info] alter table users
[info] == Migrated in 0.0s
** (ErlangError) Erlang error: :timeout_value
    (stdlib) proc_lib.erl:933: :proc_lib.stop/3
    (elixir) lib/gen_server.ex:917: GenServer.stop/3
    (ecto_sql) lib/mix/tasks/ecto.rollback.ex:93: anonymous fn/4 in Mix.Tasks.Ecto.Rollback.run/2
    (elixir) lib/enum.ex:768: Enum."-each/2-lists^foreach/1-0-"/2
    (elixir) lib/enum.ex:768: Enum.each/2
    (mix) lib/mix/task.ex:319: Mix.Task.run_task/3
    (mix) lib/mix/cli.ex:79: Mix.CLI.run_task/2
    (elixir) lib/code.ex:767: Code.require_file/2

The rollback actually succeeds but there shouldn't be an error.

Expected behavior

No error.

Document that telemetry measures are reported in nanoseconds?

Environment

• Elixir version (elixir -v): 1.8.2
• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): PostgreSQL 11.2
• Ecto version (mix deps): 3.1.6
• Database adapter and version (mix deps): ecto_sql 3.1.1 and postgrex 0.14.0
• Operating system: MacOS 10.14

Current behavior

I'm using ecto 3.1.6, ecto_sql 3.1.1 and postgrex 0.14.0 with PostgreSQL 11.2.

When I add an IO.inspect here, a particular query shows :query_time of 2587 μs. But the telemetry measurements reported to my handler show query_time: 2587000. So it appears to me that the times are being reported in nanoseconds. I don't see this documented anywhere.

Expected behavior

I would like to document somewhere that the measurements are in nanoseconds, as it took me some puzzling to figure this out. Can someone advise me on where that documentation should go? So far I don't see where these numbers are actually generated.

Link to documentation in README is broken

Environment

Master branches of ecto and ecto_sql.

Current behavior

The link to the ecto_sql documentation (https://hexdocs.pm/ecto_sql) is broken.

Expected behavior

Documentation for ecto_sql should be displayed. It currently does not because ecto_sql is not a published package on hex. This also breaks the Ecto.Adapters.SQL link in the ecto README.

The test files in the sql folder are missing the `_test` suffix

Environment

• Ecto SQL: master

Proposal: static telemetry event naming

Hello folks! I've been participating in the various conversations around telemetry project, and I'd like to propose a change in the Event naming scheme that Ecto uses...

Right now it is configurable, so a user could change the event name to [:my_app, :query]. The impact of this is that it's impossible to attach to Ecto events w/o some configuration input from the user (since the event name is needed up-front to subscribe to the event stream. This situation doesn't improve the ergonomics at all, but does introduce a burden on the user & the reporter library.

I'd like to suggest that the event name be static, so that all event name sent from Telemetry for Ecto instrumentation is always the same, for example:

:telemetry.execute([:ecto, :query], total, metadata)

This change would mean that the raw events are discoverable & consumable directly by Telemetry reporters, while still allowing customization via metadata or through telemetry_metrics. Any user-specific info, like a custom metric name or the Ecto Repo belong in the metadata.

Thanks!

cc/ @josevalim @arkgil

FYI, I think this should also be the generic recommendation for building Telemetry instrumentation into libraries.

Adding a postgres index concurrently corrupts the index

We recently moved to ecto 3, and have suddenly had issues with concurrent index creation. The migration will hang indefinitely, and if you inspect the index in psql it is marked INVALID. This occurs even with non-unique indexes, which is unusual because it usually happens when a constraint is violated during index create. My guess is something is racing in the new multi-process migration system but couldn't piece it together.

Another possible wrinkle is we actually run our migration through distillery custom commands as seen here: https://github.com/bitwalker/distillery/blob/master/docs/guides/running_migrations.md. I tried fiddling with the pool sizes and diffing that code against the actual mix ecto.migrate task but nothing popped out.

Environment

• Elixir version (elixir -v): 1.7.3
• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): Postgresql 9.6.8 (running on amazon aurora)
• Ecto version (mix deps): 3.0.3
• Database adapter and version (mix deps): postgrex 0.14
• Operating system: alpine linux in docker

Current behavior

Ecto migrate for concurrent indexes hangs and corrupts the index

Expected behavior

Ecto can create indexes concurrently

How to add a lock timeout to migrations?

We ran into a scenario recently where we had downtime for the following reason:

1. We had a long running transaction open
2. We had a migration that attempted to take an ACCESS EXCLUSIVE lock (happens for all ALTER TABLE commands)

This caused all queries to be queued behind the migration, which was waiting to get a lock.

We'd like to automatically set a lock timeout on migrations to avoid this problem in future. The command is:

SET lock_timeout TO '5s'

We could add this manually inside every migration we create but that is tedious and prone to people forgetting. Is there a way to use ecto to automatically execute this SQL at the start of every DDL transaction?

Disable migration table lock when `:concurrent` option is used

Rather than warn or hang, it would be ideal if migrations using the :concurrent option would automatically turn off the migration table lock for that migration only.

I agree that this new lock introduced in 3.1 is a great default to have for most codebases and migrations. But when you need to run something concurrently, it seems unwise to turn this lock off for all migrations, when you only really need it off for a single migration.

More context:

• elixir-ecto/ecto#3006
• c101091

Use `type/2` with window functions

Environment

• Elixir version (elixir -v): 1.7.4 OTP 21
• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): PostgreSQL 10.5
• Ecto version (mix deps): 3.0.0
• Database adapter and version (mix deps): postgrex 0.14.1
• Operating system:

Current behavior

Doing avg(d.production) by default returns a %Decimal{}, to have it return a float I usually do type(avg(d.production), :float) or use fragment, both work fine.

However, I've tried multiple ways of having the same behavior while using window functions:

• over(type(avg(d.production), :float), :interval) throws unknown window function type/2. See Ecto.Query.WindowAPI for all available functions
• type(over(avg(d.production), :interval), :float) throws type(over(avg(d.production()), :interval), :float) is not a valid query expression.

Expected behavior

The select returns that value as float

Migration failure with latest PostgreSQL release 11.4

Environment

• Elixir version (elixir -v): 1.8.1
• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): PostgreSQL 11.4
• Ecto version (mix deps): 3.1.5
• Database adapter and version (mix deps): pg
• Operating system: MacOS Mojave

Current behavior

Considering this table

defmodule MyApp.Repo.CreatePosts do
  use Ecto.Migration

  def change do
    create table(:likes) do
      add :user_id, references(:users, on_delete: :nothing)
      add :post_id, references(:posts, on_delete: :nothing)

      timestamps()
    end

    create index(:likes, [:user_id])
    create index(:likes, [:post_id])

    create unique_index(:likes, [:user_id, :post_id], name: :unique_user_post_index)
  end
end

and a follow-up migration that just aims to change the on_delete behaviour:

defmoule MyApp.Repo.ChangeOnDeleteOfUsersAndPostsOnLikes do
   use Ecto.Migration

   def change do
     alter table(:likes) do
        modify :user_id, references(:users, on_delete: :delete_all), from: references(:users, on_delete: :nothing)
        modify :post_id, references(:posts, on_delete: :delete_all), from: references(:posts, on_delete: :nothing)
     end
   end
end

Now fails in PostgreSQL since 11.4, but worked fine in previous versions. The error message given is:

* (Postgrex.Error) ERROR 42P07 (duplicate_table) relation "unique_user_post_index" already exists

Expected behavior

Not sure if this isn't just a bug introduced with this release of PostgreSQL or if this is a bug on ecto_sql. What I did find out however is that the following SQL is being generated:

ALTER TABLE "likes" 
DROP CONSTRAINT "likes_user_id_fkey", 
ALTER COLUMN "user_id" TYPE bigint, 
ADD CONSTRAINT "likes_user_id_fkey" FOREIGN KEY ("user_id") REFERENCES "users"("id") ON DELETE CASCADE, 
DROP CONSTRAINT "likes_post_id_fkey", 
ALTER COLUMN "post_id" TYPE bigint, 
ADD CONSTRAINT "likes_post_id_fkey" FOREIGN KEY ("post_id") REFERENCES "posts"("id") ON DELETE CASCADE

Which fails when run on the psql console as well, but the following SQL executes and has the desired result:

ALTER TABLE "likes" 
DROP CONSTRAINT "likes_user_id_fkey", 
ADD CONSTRAINT "likes_user_id_fkey" FOREIGN KEY ("user_id") REFERENCES "users"("id") ON DELETE CASCADE, 
DROP CONSTRAINT "likes_post_id_fkey", 
ADD CONSTRAINT "likes_post_id_fkey" FOREIGN KEY ("post_id") REFERENCES "posts"("id") ON DELETE CASCADE

So, the question is: Should a migration that changes/resets the foreign key reference on a column also declare the column type that doesn't change? I understand that changing the reference to another table might result in a change of type for the column and that it's kinda hard to figure out if this does beforehand. However, the required information is all there in the from: option....

ecto.migrate crashing

Environment

• Elixir version (elixir -v):Elixir 1.7.4 (compiled with Erlang/OTP 21)
• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): (PostgreSQL) 11.1
• Ecto version (mix deps): ecto 3.0.5 (Hex package) (mix)
• Database adapter and version (mix deps): ecto_sql 3.0.3
• Operating system: macOS 10.14.2 (18C54)

Current behavior

Running ecto.migrate runs all migrations and finishes...with a crash.

I have a alias setup to run migrations for a few repos. But mix crashes after the first repo is migrated.

ie. mix ecto.migrate --repo A --repo B will migrate repo A but exit before migrating B.

It seems that the offender is this. I can comment that out and it works as expected.

Expected behavior

My team and I would like to be able to migrate all repos without extra typing. :)

mix ecto.migrate --log-sql is broken

Environment

• Elixir version (elixir -v): master
• Ecto version (mix deps): master
• Database adapter and version (mix deps): postgrex master

Current behavior

Note, without --log-sql everything is fine.

hexpm[wm-ecto-3]% mix ecto.migrate --log-sql
[debug] QUERY OK source="schema_migrations" db=1.1ms
SELECT s0."version"::bigint FROM "schema_migrations" AS s0 FOR UPDATE []
[info] == Running Hexpm.Repo.Migrations.AddServiceToUsers.up/0 forward
[info] alter table users
[error] an exception was raised logging %DBConnection.LogEntry{call: :prepare_execute, connection_time: 12329000, decode_time: 14000, params: [], pool_time: nil, query: %Postgrex.Query{cache: :reference, columns: nil, name: "", param_formats: nil, param_oids: nil, param_types: nil, ref: nil, result_formats: nil, result_oids: nil, result_types: nil, statement: ["ALTER TABLE ", [34, "users", 34], 32, [[], "ADD COLUMN ", [34, "service", 34], 32, "boolean", [[" DEFAULT ", "false"], []]], []], types: nil}, result: {:ok, %Postgrex.Query{cache: :reference, columns: nil, name: "", param_formats: [], param_oids: [], param_types: [], ref: #Reference<0.2121130605.262930433.40304>, result_formats: [], result_oids: nil, result_types: nil, statement: ["ALTER TABLE ", [34, "users", 34], 32, [[], "ADD COLUMN ", [34, "service", 34], 32, "boolean", [[" DEFAULT ", "false"], []]], []], types: {Postgrex.DefaultTypes, #Reference<0.2121130605.263061505.39872>}}, %Postgrex.Result{columns: nil, command: :alter_table, connection_id: 80873, messages: [], num_rows: 0, rows: nil}}}: ** (FunctionClauseError) no function clause matching in Logger.__should_log__/1
    (logger) lib/logger.ex:663: Logger.__should_log__(true)
    (ecto) lib/ecto/log_entry.ex:57: Ecto.LogEntry.log/2
    (ecto_sql) lib/ecto/adapters/sql.ex:752: Ecto.Adapters.SQL.log/4
    (db_connection) lib/db_connection.ex:1303: DBConnection.log/5
    (postgrex) lib/postgrex.ex:168: Postgrex.query_prepare_execute/4
    (ecto_sql) lib/ecto/adapters/sql.ex:288: Ecto.Adapters.SQL.query!/4
    (elixir) lib/enum.ex:1326: Enum."-map/2-lists^map/1-0-"/2
    (ecto_sql) lib/ecto/adapters/sql.ex:671: Ecto.Adapters.SQL.execute_ddl/4
    (ecto_sql) lib/ecto/migration/runner.ex:272: Ecto.Migration.Runner.log_and_execute_ddl/3
    (ecto_sql) lib/ecto/migration/runner.ex:105: anonymous fn/2 in Ecto.Migration.Runner.flush/0
    (elixir) lib/enum.ex:1936: Enum."-reduce/3-lists^foldl/2-0-"/3
    (ecto_sql) lib/ecto/migration/runner.ex:103: Ecto.Migration.Runner.flush/0
    (stdlib) timer.erl:181: :timer.tc/2
    (ecto_sql) lib/ecto/migration/runner.ex:27: Ecto.Migration.Runner.run/6
    (ecto_sql) lib/ecto/migrator.ex:195: Ecto.Migrator.attempt/6
    (ecto_sql) lib/ecto/migrator.ex:112: anonymous fn/3 in Ecto.Migrator.do_up/4
    (ecto_sql) lib/ecto/adapters/sql.ex:774: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
    (db_connection) lib/db_connection.ex:1341: DBConnection.run_transaction/4
    (ecto_sql) lib/ecto/migrator.ex:183: Ecto.Migrator.do_run_maybe_in_transaction/3
    (elixir) lib/task/supervised.ex:88: Task.Supervised.do_apply/2

[info] execute "ALTER TABLE users ALTER password DROP NOT NULL"
[error] an exception was raised logging %DBConnection.LogEntry{call: :prepare_execute, connection_time: 496000, decode_time: 9000, params: [], pool_time: nil, query: %Postgrex.Query{cache: :reference, columns: nil, name: "", param_formats: nil, param_oids: nil, param_types: nil, ref: nil, result_formats: nil, result_oids: nil, result_types: nil, statement: "ALTER TABLE users ALTER password DROP NOT NULL", types: nil}, result: {:ok, %Postgrex.Query{cache: :reference, columns: nil, name: "", param_formats: [], param_oids: [], param_types: [], ref: #Reference<0.2121130605.262930433.40374>, result_formats: [], result_oids: nil, result_types: nil, statement: "ALTER TABLE users ALTER password DROP NOT NULL", types: {Postgrex.DefaultTypes, #Reference<0.2121130605.263061505.39872>}}, %Postgrex.Result{columns: nil, command: :alter_table, connection_id: 80873, messages: [], num_rows: 0, rows: nil}}}: ** (FunctionClauseError) no function clause matching in Logger.__should_log__/1
    (logger) lib/logger.ex:663: Logger.__should_log__(true)
    (ecto) lib/ecto/log_entry.ex:57: Ecto.LogEntry.log/2
    (ecto_sql) lib/ecto/adapters/sql.ex:752: Ecto.Adapters.SQL.log/4
    (db_connection) lib/db_connection.ex:1303: DBConnection.log/5
    (postgrex) lib/postgrex.ex:168: Postgrex.query_prepare_execute/4
    (ecto_sql) lib/ecto/adapters/sql.ex:288: Ecto.Adapters.SQL.query!/4
    (elixir) lib/enum.ex:1326: Enum."-map/2-lists^map/1-0-"/2
    (ecto_sql) lib/ecto/adapters/sql.ex:671: Ecto.Adapters.SQL.execute_ddl/4
    (ecto_sql) lib/ecto/migration/runner.ex:272: Ecto.Migration.Runner.log_and_execute_ddl/3
    (ecto_sql) lib/ecto/migration/runner.ex:105: anonymous fn/2 in Ecto.Migration.Runner.flush/0
    (elixir) lib/enum.ex:1936: Enum."-reduce/3-lists^foldl/2-0-"/3
    (ecto_sql) lib/ecto/migration/runner.ex:103: Ecto.Migration.Runner.flush/0
    (stdlib) timer.erl:181: :timer.tc/2
    (ecto_sql) lib/ecto/migration/runner.ex:27: Ecto.Migration.Runner.run/6
    (ecto_sql) lib/ecto/migrator.ex:195: Ecto.Migrator.attempt/6
    (ecto_sql) lib/ecto/migrator.ex:112: anonymous fn/3 in Ecto.Migrator.do_up/4
    (ecto_sql) lib/ecto/adapters/sql.ex:774: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
    (db_connection) lib/db_connection.ex:1341: DBConnection.run_transaction/4
    (ecto_sql) lib/ecto/migrator.ex:183: Ecto.Migrator.do_run_maybe_in_transaction/3
    (elixir) lib/task/supervised.ex:88: Task.Supervised.do_apply/2

[info] execute "INSERT INTO users (username, service, inserted_at, updated_at)\nVALUES ('hexdocs', true, now(), now())\n"
[error] an exception was raised logging %DBConnection.LogEntry{call: :prepare_execute, connection_time: 1963000, decode_time: 16000, params: [], pool_time: nil, query: %Postgrex.Query{cache: :reference, columns: nil, name: "", param_formats: nil, param_oids: nil, param_types: nil, ref: nil, result_formats: nil, result_oids: nil, result_types: nil, statement: "INSERT INTO users (username, service, inserted_at, updated_at)\nVALUES ('hexdocs', true, now(), now())\n", types: nil}, result: {:ok, %Postgrex.Query{cache: :reference, columns: nil, name: "", param_formats: [], param_oids: [], param_types: [], ref: #Reference<0.2121130605.262930433.40411>, result_formats: [], result_oids: nil, result_types: nil, statement: "INSERT INTO users (username, service, inserted_at, updated_at)\nVALUES ('hexdocs', true, now(), now())\n", types: {Postgrex.DefaultTypes, #Reference<0.2121130605.263061505.39872>}}, %Postgrex.Result{columns: nil, command: :insert, connection_id: 80873, messages: [], num_rows: 1, rows: nil}}}: ** (FunctionClauseError) no function clause matching in Logger.__should_log__/1
    (logger) lib/logger.ex:663: Logger.__should_log__(true)
    (ecto) lib/ecto/log_entry.ex:57: Ecto.LogEntry.log/2
    (ecto_sql) lib/ecto/adapters/sql.ex:752: Ecto.Adapters.SQL.log/4
    (db_connection) lib/db_connection.ex:1303: DBConnection.log/5
    (postgrex) lib/postgrex.ex:168: Postgrex.query_prepare_execute/4
    (ecto_sql) lib/ecto/adapters/sql.ex:288: Ecto.Adapters.SQL.query!/4
    (elixir) lib/enum.ex:1326: Enum."-map/2-lists^map/1-0-"/2
    (ecto_sql) lib/ecto/adapters/sql.ex:671: Ecto.Adapters.SQL.execute_ddl/4
    (ecto_sql) lib/ecto/migration/runner.ex:272: Ecto.Migration.Runner.log_and_execute_ddl/3
    (ecto_sql) lib/ecto/migration/runner.ex:105: anonymous fn/2 in Ecto.Migration.Runner.flush/0
    (elixir) lib/enum.ex:1936: Enum."-reduce/3-lists^foldl/2-0-"/3
    (ecto_sql) lib/ecto/migration/runner.ex:103: Ecto.Migration.Runner.flush/0
    (stdlib) timer.erl:181: :timer.tc/2
    (ecto_sql) lib/ecto/migration/runner.ex:27: Ecto.Migration.Runner.run/6
    (ecto_sql) lib/ecto/migrator.ex:195: Ecto.Migrator.attempt/6
    (ecto_sql) lib/ecto/migrator.ex:112: anonymous fn/3 in Ecto.Migrator.do_up/4
    (ecto_sql) lib/ecto/adapters/sql.ex:774: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
    (db_connection) lib/db_connection.ex:1341: DBConnection.run_transaction/4
    (ecto_sql) lib/ecto/migrator.ex:183: Ecto.Migrator.do_run_maybe_in_transaction/3
    (elixir) lib/task/supervised.ex:88: Task.Supervised.do_apply/2

[info] == Migrated in 0.0s
hexpm[wm-ecto-3]%

Expected behavior

No error

(RuntimeError) could not find migration runner process for #PID<0.91.0>

Environment

• Elixir version: 1.7.4
• Database and version: postgres 11.1
• Ecto version: 3.0
• Database adapter and version: postgrex 0.14.1
• Operating system: macos 10.13.6

Current behavior

config :avocado, Avocado.Repo,
  migration_default_prefix: "avocadoz"

The database for Avocado.Repo has been created
[info] == Running 20190115121352 Avocado.Repo.Migrations.CreateUsers.change/0 forward
[info] create table avocadoz.users
** (Postgrex.Error) ERROR 3F000 (invalid_schema_name) schema "avocadoz" does not exist
    (ecto_sql) lib/ecto/adapters/sql.ex:620: Ecto.Adapters.SQL.raise_sql_call_error/1
    (elixir) lib/enum.ex:1314: Enum."-map/2-lists^map/1-0-"/2
    (ecto_sql) lib/ecto/adapters/sql.ex:707: Ecto.Adapters.SQL.execute_ddl/4
    (ecto_sql) lib/ecto/migration/runner.ex:301: Ecto.Migration.Runner.log_and_execute_ddl/3
    (ecto_sql) lib/ecto/migration/runner.ex:110: anonymous fn/2 in Ecto.Migration.Runner.flush/0
    (elixir) lib/enum.ex:1925: Enum."-reduce/3-lists^foldl/2-0-"/3
    (ecto_sql) lib/ecto/migration/runner.ex:108: Ecto.Migration.Runner.flush/0
    (stdlib) timer.erl:166: :timer.tc/1
    (ecto_sql) lib/ecto/migration/runner.ex:26: Ecto.Migration.Runner.run/7
    (ecto_sql) lib/ecto/migrator.ex:211: Ecto.Migrator.attempt/7
    (ecto_sql) lib/ecto/migrator.ex:113: anonymous fn/4 in Ecto.Migrator.do_up/4
    (ecto_sql) lib/ecto/migrator.ex:193: anonymous fn/3 in Ecto.Migrator.run_maybe_in_transaction/5
    (ecto_sql) lib/ecto/adapters/sql.ex:814: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
    (db_connection) lib/db_connection.ex:1349: DBConnection.run_transaction/4
    (ecto_sql) lib/ecto/migrator.ex:192: Ecto.Migrator.run_maybe_in_transaction/5
    (elixir) lib/task/supervised.ex:89: Task.Supervised.do_apply/2
    (elixir) lib/task/supervised.ex:38: Task.Supervised.reply/5
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

Adding the prefix to the migration itself yields the same result.

I tried execute "create schema avocadoz;" and got:

** (RuntimeError) could not find migration runner process for #PID<0.91.0>
    (ecto_sql) lib/ecto/migration/runner.ex:288: Ecto.Migration.Runner.runner/0
    (ecto_sql) lib/ecto/migration/runner.ex:122: Ecto.Migration.Runner.execute/1
    priv/repo/migrations/20190115121352_create_users.exs:4: (module)
    (stdlib) erl_eval.erl:680: :erl_eval.do_apply/6
    (elixir) lib/code.ex:715: Code.load_file/2
    (ecto_sql) lib/ecto/migrator.ex:476: Ecto.Migrator.extract_module/2
    (ecto_sql) lib/ecto/migrator.ex:433: anonymous fn/5 in Ecto.Migrator.do_migrate/4
    (elixir) lib/enum.ex:3281: Enumerable.List.reduce/3
    (elixir) lib/enum.ex:1968: Enum.reduce_while/3
    (ecto_sql) lib/ecto/migrator.ex:427: Ecto.Migrator.migrate/4
    (ecto_sql) lib/ecto/adapters/sql.ex:814: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
    (db_connection) lib/db_connection.ex:1349: DBConnection.run_transaction/4

Expected Behavior

obvious

Ecto.Adapters.Postgres.structure_load delegates to `psql`

Present in master.

This means that we need to ship the psql tool to where we build our database, which is less than ideal.

Current behavior

Ecto.Adapters.Postgres.structure_load delegates to psql

Expected behavior

Ecto.Adapters.Postgres.structure_load performs the work without reliance on the psql tool.

Problem with the async migrations

Environment

• Elixir version (elixir -v):

Erlang/OTP 21 [erts-10.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Elixir 1.8.1 (compiled with Erlang/OTP 20)

• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): PostgreSQL 9.6
• Ecto version (mix deps):

* ecto (Hex package) (mix)
  locked at 3.0.1 (ecto) a26605ee
* ecto_sql (Hex package) (mix)
  locked at 3.0.0 (ecto_sql) 8d188337

• Database adapter and version (mix deps):

* postgrex (Hex package) (mix)
  locked at 0.14.0 (postgrex) f3d6ffea

• Operating system: MacOS 10.14.3

Current behavior

So, we are trying to support ecto 3 on triplex (PR ateliware/triplex#61), and we stumbled in a problem: we cannot run a CREATE SCHEMA "a" and the migrations for the prefix "a" inside the same transaction, because now the migrations are ran inside a Task.async, so the db connection will not be the same and then we get a ** (Postgrex.Error) ERROR 3F000 (invalid_schema_name) schema "a" does not exist for the migrations.

A sample code would be:

Repo.transaction fn ->
  Ecto.Adapters.SQL.query!(Repo, "CREATE SCHEMA \"a\"", [])
  Ecto.Migrator.run(Repo, :up, all: true, prefix: "a")
end

And the error you will get on ecto 3 is:

** (Postgrex.Error) ERROR 3F000 (invalid_schema_name) schema "a" does not exist
    (ecto_sql) lib/ecto/adapters/sql.ex:590: Ecto.Adapters.SQL.raise_sql_call_error/1
    (elixir) lib/enum.ex:1327: Enum."-map/2-lists^map/1-0-"/2
    (ecto_sql) lib/ecto/adapters/sql.ex:677: Ecto.Adapters.SQL.execute_ddl/4
    (ecto_sql) lib/ecto/migration/runner.ex:272: Ecto.Migration.Runner.log_and_execute_ddl/3
    (ecto_sql) lib/ecto/migration/runner.ex:105: anonymous fn/2 in Ecto.Migration.Runner.flush/0
    (elixir) lib/enum.ex:1940: Enum."-reduce/3-lists^foldl/2-0-"/3
    (ecto_sql) lib/ecto/migration/runner.ex:103: Ecto.Migration.Runner.flush/0
    (stdlib) timer.erl:181: :timer.tc/2

Expected behavior

So with ecto 2 the sample code runs smoothly, and migrates everything, because all the migrations are run inside of the same process of the transaction.

My suggestion for a solution would be an async option for Ecto.Migrator.up/4, Ecto.Migrator.down/4 and Ecto.Migrator.run/3,4, that could be true by default, but if false, it will not run the migrations on a Task.async.

I can work on that if you want.

** (Postgrex.Error) ERROR 3B001 (invalid_savepoint_specification) no such savepoint

Environment

• Ecto version (mix deps): master
• Database adapter and version (mix deps): postgrex master

Current behavior

I'm getting this error:

15:41:37.483 [debug] QUERY ERROR db=1.1ms queue=1.9ms
INSERT INTO "packages" ("meta","name") VALUES ($1,$2) RETURNING "id" [%{}, "foo"]

15:41:37.484 [error] Postgrex.Protocol (#PID<0.294.0>) disconnected: ** (Postgrex.Error) ERROR 3B001 (invalid_savepoint_specification) no such savepoint


  1) test greets the world (ExampleMasterTest)
     test/example_master_test.exs:10
     ** (Postgrex.Error) ERROR 3B001 (invalid_savepoint_specification) no such savepoint
     code: Repo.insert!(Package.changeset(%Package{}, %{name: "foo", meta: %{}}))
     stacktrace:
       (ecto_sql) lib/ecto/adapters/sql.ex:584: Ecto.Adapters.SQL.raise_sql_call_error/1
       (ecto) lib/ecto/repo/schema.ex:600: Ecto.Repo.Schema.apply/4
       (ecto) lib/ecto/repo/schema.ex:226: anonymous fn/15 in Ecto.Repo.Schema.do_insert/3
       (ecto) lib/ecto/repo/schema.ex:128: Ecto.Repo.Schema.insert!/3
       test/example_master_test.exs:11: (test)

Turns out I forgot to set config :postgrex, json_library: ... which is needed for map field. When I set it, it's fine. It's possible I've somehow misconfigured the repo too.

Repo to reproduce: https://github.com/wojtekmach/example_master

Expected behavior

Perhaps a compile-time error from Ecto.Adapters.Postgres that json library is missing.

Failing migration left db in inconsistent state

Environment

• Elixir version (elixir -v): 1.8.1
• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.):PostgreSQL 9.6
• Ecto version (mix deps): ecto 3.0.7, ecto-sql 3.0.5
• Database adapter and version (mix deps): postgrex 0.14.1
• Operating system: Linux (debian)

Current behavior

On our production system we run migrations as pre startup task. We deployed a new version of our app with several migrations to be applied (Let's call them M1, M2, M3, ..). From what was logged we can see that M1 was applied successfully. M2 started and took too long to complete before our app health check failed and the app was killed. After restart the migrations were supposed to rerun but failed due to inconsistent state in schema_migrations.

Logs of the first run

Running migrations
Loading my_app..
Starting dependencies..
Starting repos..
Running migrations for my_app
11:24:22.407 [info] == Running 20190131122210 MyApp.Repo.Migrations.M1.change/0 forward
11:24:22.408 [info] create table table_1
11:24:22.436 [info] create index table_1_column_1_index
11:24:22.440 [info] alter table table_2
11:24:22.447 [info] execute "DROP MATERIALIZED VIEW view_1;"
11:24:22.456 [info] == Migrated 20190131122210 in 0.0s
11:24:22.523 [info] == Running 20190131122211 MyApp.Repo.Migrations.M2.change/0 forward
11:24:22.524 [info] execute "update table_3 set status = 'status_1' where status = 'status_2';"
(killed by health check policy)

Logs of the second run

Loading my_app..
Starting dependencies..
Starting repos..
Running migrations for my_app
11:28:48.228 [info] == Running 20190131122210 MyApp.Repo.Migrations.M1.change/0 forward
11:28:48.229 [info] create table table_1
[31m▸ Evaluation failed with: ERROR 42P07 (duplicate_table) relation "table_1" already exists[0m
(returned non 0 exit code)

Expected behavior

Migration should be marked as applied even if app terminates during applying of following migrations

[Proposal] Allow Addapter developers to override how Ecto.UUID dumps and loads uuid binary representation

Background

Most database engines that was first developed for *nix operating system stores UUID type binary in order as human readable representation is rendered. As for many things, Microsoft did not respect this standard, so fist part of 4 bytes, bytes in reversed order and next 2 bytes also, after that, other bytes are not changed.

standard order:

<<b1, b2, b3, b4, ?-, b5, b6, ?-, b7, b8, ?-, b9, b10, ?-, b11, b12, b13, b14, b15, b16>>

tds order:

<<b4, b3, b2, b1, ?-, b6, b5, ?-, b8, b7, ?-, b9, b10, ?-, b11, b12, b13, b14, b15, b16>>

Earlier, before ecto 3.x I used custom type Tds.UUID instead of builtin and had to rewrite all support integration tests. Since most of integration test are using UUID this would mean that if I add MsSql adapter to ecto_sql repository, I have to write my own test cases, schemas and migrations in order to cover all cases properly.

Solution

So far I don't have idea how to nicely override Ecto.UUID since this is out of adapter reach, any suggestions that do not involve rewriting all that integration cases are welcome!!!

`:ownership_timeout` setting globally do not work

Environment

• Ecto version (mix deps): "3.0.7", "44dda84ac6b17bbbdeb8ac5dfef08b7da253b37a453c34ab1a98de7f7e5fec7f"
• Database adapter and version (mix deps):
  ecto_sql, "3.0.5", "7e44172b4f7aca4469f38d7f6a3da394dbf43a1bcf0ca975e958cb957becd74e"
  db_connection, "2.0.6", "bde2f85d047969c5b5800cb8f4b3ed6316c8cb11487afedac4aa5f93fd39abfa"

Current behavior

ecto_sql/lib/ecto/adapters/sql/sandbox.ex

Lines 212 to 232 in b9dfe39

	In some situations, you may see error reports similar to the one below:
	09:56:43.081 [error] Postgrex.Protocol (#PID<>) disconnected:
	** (DBConnection.ConnectionError) owner #PID<> timed out
	because it owned the connection for longer than 60000ms
	If you have a long running test (or you're debugging with IEx.pry),
	the timeout for the connection ownership may be too short. You can
	increase the timeout by setting the `:ownership_timeout` options for
	your repo config in `config/config.exs` (or preferably in `config/test.exs`):
	config :my_app, MyApp.Repo,
	ownership_timeout: NEW_TIMEOUT_IN_MILLISECONDS
	The `:ownership_timeout` option is part of `DBConnection.Ownership`
	and defaults to 60000ms. Timeouts are given as integers in milliseconds.
	Alternately, if this is an issue for only a handful of long-running tests,
	you can pass an `:ownership_timeout` option when calling
	`Ecto.Adapters.SQL.Sandbox.checkout/2` instead of setting a longer timeout
	globally in your config.

we drop :ownership_timeout option (ecto_sql/lib/ecto/adapters/sql.ex:450)

ecto_sql/lib/ecto/adapters/sql.ex

Lines 427 to 453 in b9dfe39

	@pool_opts [:timeout, :pool, :pool_size, :migration_lock, :queue_target, :queue_interval]
	@doc false
	def init(connection, driver, config) do
	unless Code.ensure_loaded?(connection) do
	raise """
	could not find #{inspect connection}.
	Please verify you have added #{inspect driver} as a dependency:
	{#{inspect driver}, ">= 0.0.0"}
	And remember to recompile Ecto afterwards by cleaning the current build:
	mix deps.clean --build ecto
	"""
	end
	log = Keyword.get(config, :log, :debug)
	telemetry_prefix = Keyword.fetch!(config, :telemetry_prefix)
	telemetry = {config[:repo], log, telemetry_prefix ++ [:query]}
	config = adapter_config(config)
	opts = Keyword.take(config, @pool_opts)
	meta = %{telemetry: telemetry, sql: connection, opts: opts}
	{:ok, connection.child_spec(config), meta}
	end

then when Ecto.Adapters.SQL.checkout(DB.Repo)
:ownership_timeout options for my repo config in config/test.exs is not available.

ecto_sql/lib/ecto/adapters/sql/sandbox.ex

Lines 485 to 486 in b9dfe39

	defp lookup_meta!(repo) do
	%{opts: opts} = meta = Ecto.Adapter.lookup_meta(repo)

Expected behavior

don't clean up option in ecto_sql/lib/ecto/adapters/sql.ex

:after_connect causing timeout

Precheck

• Do not use the issues tracker for help or support requests (try Elixir Forum, Stack Overflow, IRC or mailing lists, etc).
• For proposing a new feature, please start a discussion on elixir-ecto.
• For bugs, do a quick search and make sure the bug has not yet been reported.
• Finally, be nice and have fun!

Environment

• Elixir version (elixir -v): 1.7.1
• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): 9.5
• Ecto version (mix deps): 3.0
• Database adapter and version (mix deps): postgrex >= 0.0.0
• Operating system: Mac OSX

Current behavior

My Repo config uses the :after_connect option to set search_path for Postgres:

config :my_app, MyApp.Repo,
  after_connect: {Postgrex, :query!, ["SET search_path=\"$user\",public,shared_extensions", []]}

This was working fine previously on Ecto v2.2.10. However, since upgrading to Ecto 3.0 I'm starting to see this error in console:

[error] Postgrex.Protocol (#PID<0.512.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0.685.0> timed out because it checked out the connection for longer than 15000ms

When I removed the after_connect option from config then the error no longer shows up.

Expected behavior

No error in console

Missing condition {:ok, _} in execute/4

Current behavior

The function Ecto.Adapters.Postgres.Connection.execute/4 is not handling the case where Postgrex.execute/4 returns {: ok, Postgrex.Result.t}

execute/4 specification: execute(conn, Postgrex.Query.t, list, Keyword.t) :: {:ok, Postgrex.Result.t} | {:error, Postgrex.Error.t}

Expected behavior

Add condition for {:ok, Postgrex.Result.t}

Example:

        {:ok _} = ok ->
           ok

---

Can such cases occur?

        {:ok, %{ref: ^ref}, result} ->
          {:ok, result}

        {:ok, _, _} = ok ->
          ok

In the Postgrex.execute/4 @spec do not have these cases

Breaking change in migrations when upgrading 3.0.3 -> 3.0.5

Environment

Upgrading

• ecto: 3.0.5 -> 3.0.7
• ecto_sql: 3.0.3 -> 3.0.5

Intro

One of our developers created migrations with different file names but with the same module name inside. These migrations contain different instructions inside.
From logs:

[info] == Running 20180107124803 App.Repo.Migrations.AddCoherenceToUser.change/0 forward
...
[info] == Running 20180123144846 App.Repo.Migrations.AddCoherenceToUser.change/0 forward

Previous behavior

It worked well

Current behavior

The first migration runs twice with different timestamps, and we have conflicts.

Expected behavior

Allow conflicting module names or show warning.

P. S: Is it safe to just rename a module?

Edited: okay, a warning is shown.

Add Repo name to Telemetry metadata

For monitoring solutions, it'd be nice to have the Repo module name in the Telemetry metadata map.

This would enable us to report metrics with the proper namespace in the condition that there are multiple Repos in an application. Without this we couldn't tell the difference between tables with the same name in different databases...

Thanks!

Plug requires telemetry "~> 0.4.0"

There has recently been an update to plug (v1.8.0) and it looks like there is a conflict with the telemetry library for ecto_sql. It looks like everything is up to date in master but a release has not been cut for a while. Is there any way we can get out patch release?

error running migrations

I get an errror running my migrations. I'm running postgrex 14.0 with ecto 3.0

Here is the error I get

00:41:38.291 [error] an exception was raised logging %DBConnection.LogEntry{call: :prepare_execute, connection_time: 344317, decode_time: 8247259, params: [], pool_time: 71427753, query: %Postgrex.Query{cache: :reference, columns: nil, name: "", param_formats: nil, param_oids: nil, param_types: nil, ref: nil, result_formats: nil, result_oids: nil, result_types: nil, statement: ["CREATE TABLE ", "IF NOT EXISTS ", [34, "schema_migrations", 34], 32, 40, [[[], [[34, "version", 34], 32, "bigint", [[], []]], ", "], [34, "inserted_at", 34], 32, ["timestamp", "(0)"], [[], []]], [", ", "PRIMARY KEY (", [[], 34, "version", 34], ")"], 41, []], types: nil}, result: {:ok, %Postgrex.Query{cache: :reference, columns: nil, name: "", param_formats: [], param_oids: [], param_types: [], ref: #Reference<0.2077835036.3294363649.48140>, result_formats: [], result_oids: nil, result_types: nil, statement: ["CREATE TABLE ", "IF NOT EXISTS ", [34, "schema_migrations", 34], 32, 40, [[[], [[34, "version", 34], 32, "bigint", [[], []]], ", "], [34, "inserted_at", 34], 32, ["timestamp", "(0)"], [[], []]], [", ", "PRIMARY KEY (", [[], 34, "version", 34], ")"], 41, []], types: {Postgrex.DefaultTypes, #Reference<0.2077835036.3294494721.48090>}}, %Postgrex.Result{columns: nil, command: :create_table, connection_id: 7250, messages: [%{code: "42P07", file: "parse_utilcmd.c", line: "206", message: "relation \"schema_migrations\" already exists, skipping", routine: "transformCreateStmt", severity: "NOTICE", unknown: "NOTICE"}], num_rows: 0, rows: nil}}}: ** (ArgumentError) argument error
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (stdlib) :ets.lookup(Telemetry.HandlerTable, [:union, :repo, :query])
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (telemetry) lib/telemetry/handler_table.ex:59: Telemetry.HandlerTable.list_for_event/1
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (telemetry) lib/telemetry.ex:76: Telemetry.execute/3
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (ecto_sql) lib/ecto/adapters/sql.ex:754: Ecto.Adapters.SQL.log/4
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (db_connection) lib/db_connection.ex:1303: DBConnection.log/5
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (postgrex) lib/postgrex.ex:188: Postgrex.query_prepare_execute/4
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (ecto_sql) lib/ecto/adapters/sql.ex:288: Ecto.Adapters.SQL.query!/4
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (elixir) lib/enum.ex:1314: Enum."-map/2-lists^map/1-0-"/2
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3: 00:41:38.329 [error] an exception was raised logging %DBConnection.LogEntry{call: :begin, connection_time: 251680, decode_time: nil, params: nil, pool_time: 36295, query: :begin, result: {:ok, %DBConnection{conn_mode: nil, conn_ref: #Reference<0.2077835036.3294363649.48247>, pool_ref: {:pool_ref, #PID<0.133.0>, #Reference<0.2077835036.3294494721.47914>, nil, #Reference<0.2077835036.3294494721.48135>, #Reference<0.2077835036.3294363649.48246>}}, %Postgrex.Result{columns: nil, command: :commit, connection_id: 7250, messages: [], num_rows: nil, rows: nil}}}: ** (ArgumentError) argument error
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (stdlib) :ets.lookup(Telemetry.HandlerTable, [:union, :repo, :query])
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (telemetry) lib/telemetry/handler_table.ex:59: Telemetry.HandlerTable.list_for_event/1
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (telemetry) lib/telemetry.ex:76: Telemetry.execute/3
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (ecto_sql) lib/ecto/adapters/sql.ex:754: Ecto.Adapters.SQL.log/4
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (db_connection) lib/db_connection.ex:1303: DBConnection.log/5
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (db_connection) lib/db_connection.ex:760: DBConnection.transaction/3
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (ecto_sql) lib/ecto/adapters/sql.ex:692: Ecto.Adapters.SQL.lock_for_migrations/4
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (ecto_sql) lib/ecto/migrator.ex:317: Ecto.Migrator.lock_for_migrations/3
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3: 00:41:38.367 [error] an exception was raised logging %DBConnection.LogEntry{call: :prepare_execute, connection_time: 1215295, decode_time: 14453, params: [], pool_time: nil, query: %Postgrex.Query{cache: :reference, columns: nil, name: "ecto_50", param_formats: nil, param_oids: nil, param_types: nil, ref: nil, result_formats: nil, result_oids: nil, result_types: nil, statement: "SELECT s0.\"version\"::bigint FROM \"schema_migrations\" AS s0 FOR UPDATE", types: nil}, result: {:ok, %Postgrex.Query{cache: :reference, columns: ["version"], name: "ecto_50", param_formats: [], param_oids: [], param_types: [], ref: #Reference<0.2077835036.3294363649.48276>, result_formats: [:binary], result_oids: [20], result_types: [Postgrex.Extensions.Int8], statement: "SELECT s0.\"version\"::bigint FROM \"schema_migrations\" AS s0 FOR UPDATE", types: {Postgrex.DefaultTypes, #Reference<0.2077835036.3294494721.48090>}}, %Postgrex.Result{columns: ["version"], command: :select, connection_id: 7250, messages: [], num_rows: 94, rows: [[20170817002627], [20170817003845], [20170817045138], [20170901160212], [20170907160212], [20170908082120], [20170908085748], [20170917070541], [20170917082508], [20170922051950], [20170922192527], [20170923023516], [20170923165044], [20171002105454], [20171007031041], [20171008031041], [20171030105824], [20171212143912], [20180101092243], [20180103215359], [20180105134830], [20180106073049], [20180106103608], [20180106134459], [20180106154245], [20180110072641], [20180115162350], [20180118162023], [20180207012117], [20180211015839], [20180211042058], [20180211123318], [20180218022325], [...], ...]}}}: ** (ArgumentError) argument error
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (stdlib) :ets.lookup(Telemetry.HandlerTable, [:union, :repo, :query])
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (telemetry) lib/telemetry/handler_table.ex:59: Telemetry.HandlerTable.list_for_event/1
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (telemetry) lib/telemetry.ex:76: Telemetry.execute/3
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (ecto_sql) lib/ecto/adapters/sql.ex:754: Ecto.Adapters.SQL.log/4
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (db_connection) lib/db_connection.ex:1303: DBConnection.log/5
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (ecto_sql) lib/ecto/adapters/sql.ex:531: Ecto.Adapters.SQL.execute!/4
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (ecto_sql) lib/ecto/adapters/sql.ex:523: Ecto.Adapters.SQL.execute/5
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (ecto) lib/ecto/repo/queryable.ex:147: Ecto.Repo.Queryable.execute/4
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3: 00:41:38.383 [info] Already up
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3: Success!
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3: 00:41:38.398 [error] an exception was raised logging %DBConnection.LogEntry{call: :commit, connection_time: 3696570, decode_time: nil, params: nil, pool_time: nil, query: :commit, result: {:ok, %Postgrex.Result{columns: nil, command: :commit, connection_id: 7250, messages: [], num_rows: nil, rows: nil}}}: ** (ArgumentError) argument error
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (stdlib) :ets.lookup(Telemetry.HandlerTable, [:union, :repo, :query])
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (telemetry) lib/telemetry/handler_table.ex:59: Telemetry.HandlerTable.list_for_event/1
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (telemetry) lib/telemetry.ex:76: Telemetry.execute/3
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (ecto_sql) lib/ecto/adapters/sql.ex:754: Ecto.Adapters.SQL.log/4
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (db_connection) lib/db_connection.ex:1303: DBConnection.log/5
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (db_connection) lib/db_connection.ex:1363: DBConnection.run_transaction/4
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (ecto_sql) lib/ecto/adapters/sql.ex:692: Ecto.Adapters.SQL.lock_for_migrations/4
Nov 20 09:41:38 1362f30f9ec7 staging_web.1.de5bs80ltl8wxnayh9dh1hqq3:     (ecto_sql) lib/ecto/migrator.ex:317: Ecto.Migrator.lock_for_migrations/3 

Here is my mix.lock

%{
  "absinthe": {:hex, :absinthe, "1.4.13", "81eb2ff41f1b62cd6e992955f62c22c042d1079b7936c27f5f7c2c806b8fc436", [:mix], [{:dataloader, "~> 1.0.0", [hex: :dataloader, repo: "hexpm", optional: true]}, {:decimal, "~> 1.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm"},
  "absinthe_plug": {:hex, :absinthe_plug, "1.4.6", "ac5d2d3d02acf52fda0f151b294017ab06e2ed1c6c15334e06aac82c94e36e08", [:mix], [{:absinthe, "~> 1.4.11", [hex: :absinthe, repo: "hexpm", optional: false]}, {:plug, "~> 1.3.2 or ~> 1.4", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
  "artificery": {:hex, :artificery, "0.2.6", "f602909757263f7897130cbd006b0e40514a541b148d366ad65b89236b93497a", [:mix], [], "hexpm"},
  "base64url": {:hex, :base64url, "0.0.1", "36a90125f5948e3afd7be97662a1504b934dd5dac78451ca6e9abf85a10286be", [:rebar], [], "hexpm"},
  "bcrypt_elixir": {:hex, :bcrypt_elixir, "1.1.1", "6b5560e47a02196ce5f0ab3f1d8265db79a23868c137e973b27afef928ed8006", [:make, :mix], [{:elixir_make, "~> 0.4", [hex: :elixir_make, repo: "hexpm", optional: false]}], "hexpm"},
  "bunt": {:hex, :bunt, "0.2.0", "951c6e801e8b1d2cbe58ebbd3e616a869061ddadcc4863d0a2182541acae9a38", [:mix], [], "hexpm"},
  "certifi": {:hex, :certifi, "2.4.2", "75424ff0f3baaccfd34b1214184b6ef616d89e420b258bb0a5ea7d7bc628f7f0", [:rebar3], [{:parse_trans, "~>3.3", [hex: :parse_trans, repo: "hexpm", optional: false]}], "hexpm"},
  "cloudex": {:hex, :cloudex, "1.2.1", "fd2b115c8519399f11c63d11c13ff98a88d381196c01228a206e0af12dee13dd", [:mix], [{:httpoison, "~> 1.3.0", [hex: :httpoison, repo: "hexpm", optional: false]}, {:poison, "~> 3.1.0", [hex: :poison, repo: "hexpm", optional: false]}, {:timex, "~> 3.4.2", [hex: :timex, repo: "hexpm", optional: false]}, {:tzdata, "~> 0.5.11", [hex: :tzdata, repo: "hexpm", optional: false]}], "hexpm"},
  "combine": {:hex, :combine, "0.10.0", "eff8224eeb56498a2af13011d142c5e7997a80c8f5b97c499f84c841032e429f", [:mix], [], "hexpm"},
  "comeonin": {:hex, :comeonin, "4.1.1", "c7304fc29b45b897b34142a91122bc72757bc0c295e9e824999d5179ffc08416", [:mix], [{:argon2_elixir, "~> 1.2", [hex: :argon2_elixir, repo: "hexpm", optional: true]}, {:bcrypt_elixir, "~> 0.12.1 or ~> 1.0", [hex: :bcrypt_elixir, repo: "hexpm", optional: true]}, {:pbkdf2_elixir, "~> 0.12", [hex: :pbkdf2_elixir, repo: "hexpm", optional: true]}], "hexpm"},
  "connection": {:hex, :connection, "1.0.4", "a1cae72211f0eef17705aaededacac3eb30e6625b04a6117c1b2db6ace7d5976", [:mix], [], "hexpm"},
  "cowboy": {:hex, :cowboy, "2.6.0", "dc1ff5354c89e36a3e3ef8d10433396dcff0dcbb1d4223b58c64c2d51a6d88d9", [:rebar3], [{:cowlib, "~> 2.7.0", [hex: :cowlib, repo: "hexpm", optional: false]}, {:ranch, "~> 1.7.0", [hex: :ranch, repo: "hexpm", optional: false]}], "hexpm"},
  "cowlib": {:hex, :cowlib, "2.7.0", "3ef16e77562f9855a2605900cedb15c1462d76fb1be6a32fc3ae91973ee543d2", [:rebar3], [], "hexpm"},
  "credo": {:hex, :credo, "1.0.0", "aaa40fdd0543a0cf8080e8c5949d8c25f0a24e4fc8c1d83d06c388f5e5e0ea42", [:mix], [{:bunt, "~> 0.2.0", [hex: :bunt, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm"},
  "db_connection": {:hex, :db_connection, "2.0.2", "440c05518b0bdca0469dafaf45403597430448c1281def14ef9ccaa41833ea1e", [:mix], [{:connection, "~> 1.0.2", [hex: :connection, repo: "hexpm", optional: false]}], "hexpm"},
  "decimal": {:hex, :decimal, "1.5.0", "b0433a36d0e2430e3d50291b1c65f53c37d56f83665b43d79963684865beab68", [:mix], [], "hexpm"},
  "dialyxir": {:hex, :dialyxir, "0.5.1", "b331b091720fd93e878137add264bac4f644e1ddae07a70bf7062c7862c4b952", [:mix], [], "hexpm"},
  "distillery": {:hex, :distillery, "2.0.12", "6e78fe042df82610ac3fa50bd7d2d8190ad287d120d3cd1682d83a44e8b34dfb", [:mix], [{:artificery, "~> 0.2", [hex: :artificery, repo: "hexpm", optional: false]}], "hexpm"},
  "earmark": {:hex, :earmark, "1.2.6", "b6da42b3831458d3ecc57314dff3051b080b9b2be88c2e5aa41cd642a5b044ed", [:mix], [], "hexpm"},
  "ecto": {:hex, :ecto, "3.0.2", "9d06ece60e2fc9b7593e74ede23747424ef32390016baca85cea3f4b84081c52", [:mix], [{:decimal, "~> 1.5", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:poison, "~> 2.2 or ~> 3.0", [hex: :poison, repo: "hexpm", optional: true]}], "hexpm"},
  "ecto_enum": {:hex, :ecto_enum, "1.1.0", "d44fe2ce6e1c0e907e7c3b6456a69e0f1d662348d8b4e2a662ba312223d8ff62", [:mix], [{:ecto, ">= 2.0.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:mariaex, ">= 0.0.0", [hex: :mariaex, repo: "hexpm", optional: true]}, {:postgrex, ">= 0.0.0", [hex: :postgrex, repo: "hexpm", optional: true]}], "hexpm"},
  "ecto_sql": {:hex, :ecto_sql, "3.0.1", "7f0e123a3b4d3b8ac67da6ba3014c53dbed63e614c5086f3e99d01cab5defe27", [:mix], [{:db_connection, "~> 2.0", [hex: :db_connection, repo: "hexpm", optional: false]}, {:ecto, "~> 3.0.2", [hex: :ecto, repo: "hexpm", optional: false]}, {:mariaex, "~> 0.9.1", [hex: :mariaex, repo: "hexpm", optional: true]}, {:postgrex, "~> 0.14.0", [hex: :postgrex, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.2.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm"},
  "eflame": {:hex, :eflame, "1.0.1", "0664d287e39eef3c413749254b3af5f4f8b00be71c1af67d325331c4890be0fc", [:mix], [], "hexpm"},
  "elixir_make": {:hex, :elixir_make, "0.4.2", "332c649d08c18bc1ecc73b1befc68c647136de4f340b548844efc796405743bf", [:mix], [], "hexpm"},
  "ex_doc": {:hex, :ex_doc, "0.19.1", "519bb9c19526ca51d326c060cb1778d4a9056b190086a8c6c115828eaccea6cf", [:mix], [{:earmark, "~> 1.1", [hex: :earmark, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.7", [hex: :makeup_elixir, repo: "hexpm", optional: false]}], "hexpm"},
  "excoveralls": {:hex, :excoveralls, "0.10.3", "b090a3fbcb3cfa136f0427d038c92a9051f840953ec11b40ee74d9d4eac04d1e", [:mix], [{:hackney, "~> 1.13", [hex: :hackney, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm"},
  "exprintf": {:hex, :exprintf, "0.2.1", "b7e895dfb00520cfb7fc1671303b63b37dc3897c59be7cbf1ae62f766a8a0314", [:mix], [], "hexpm"},
  "exprof": {:hex, :exprof, "0.2.3", "8d4d657d73fc0c9ef1e30b2f9207b26ccbd2aec2baf1ca43f0b6d244c841c9f8", [:mix], [{:exprintf, "~> 0.2", [hex: :exprintf, repo: "hexpm", optional: false]}], "hexpm"},
  "file_system": {:hex, :file_system, "0.2.6", "fd4dc3af89b9ab1dc8ccbcc214a0e60c41f34be251d9307920748a14bf41f1d3", [:mix], [], "hexpm"},
  "gettext": {:hex, :gettext, "0.16.1", "e2130b25eebcbe02bb343b119a07ae2c7e28bd4b146c4a154da2ffb2b3507af2", [:mix], [], "hexpm"},
  "guardian": {:hex, :guardian, "1.1.1", "be14c4007eaf05268251ae114030cb7237ed9a9631c260022f020164ff4ed733", [:mix], [{:jose, "~> 1.8", [hex: :jose, repo: "hexpm", optional: false]}, {:phoenix, "~> 1.0 or ~> 1.2 or ~> 1.3", [hex: :phoenix, repo: "hexpm", optional: true]}, {:plug, "~> 1.3.3 or ~> 1.4", [hex: :plug, repo: "hexpm", optional: true]}, {:poison, "~> 2.2 or ~> 3.0", [hex: :poison, repo: "hexpm", optional: false]}], "hexpm"},
  "hackney": {:hex, :hackney, "1.14.3", "b5f6f5dcc4f1fba340762738759209e21914516df6be440d85772542d4a5e412", [:rebar3], [{:certifi, "2.4.2", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "6.0.0", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "1.0.1", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "1.0.2", [hex: :mimerl, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "1.1.4", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}], "hexpm"},
  "httpoison": {:hex, :httpoison, "1.3.1", "7ac607311f5f706b44e8b3fab736d0737f2f62a31910ccd9afe7227b43edb7f0", [:mix], [{:hackney, "~> 1.8", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"},
  "idna": {:hex, :idna, "6.0.0", "689c46cbcdf3524c44d5f3dde8001f364cd7608a99556d8fbd8239a5798d4c10", [:rebar3], [{:unicode_util_compat, "0.4.1", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm"},
  "jason": {:hex, :jason, "1.1.2", "b03dedea67a99223a2eaf9f1264ce37154564de899fd3d8b9a21b1a6fd64afe7", [:mix], [{:decimal, "~> 1.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm"},
  "jose": {:hex, :jose, "1.8.4", "7946d1e5c03a76ac9ef42a6e6a20001d35987afd68c2107bcd8f01a84e75aa73", [:mix, :rebar3], [{:base64url, "~> 0.0.1", [hex: :base64url, repo: "hexpm", optional: false]}], "hexpm"},
  "makeup": {:hex, :makeup, "0.5.5", "9e08dfc45280c5684d771ad58159f718a7b5788596099bdfb0284597d368a882", [:mix], [{:nimble_parsec, "~> 0.4", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm"},
  "makeup_elixir": {:hex, :makeup_elixir, "0.10.0", "0f09c2ddf352887a956d84f8f7e702111122ca32fbbc84c2f0569b8b65cbf7fa", [:mix], [{:makeup, "~> 0.5.5", [hex: :makeup, repo: "hexpm", optional: false]}], "hexpm"},
  "metrics": {:hex, :metrics, "1.0.1", "25f094dea2cda98213cecc3aeff09e940299d950904393b2a29d191c346a8486", [:rebar3], [], "hexpm"},
  "mime": {:hex, :mime, "1.3.0", "5e8d45a39e95c650900d03f897fbf99ae04f60ab1daa4a34c7a20a5151b7a5fe", [:mix], [], "hexpm"},
  "mimerl": {:hex, :mimerl, "1.0.2", "993f9b0e084083405ed8252b99460c4f0563e41729ab42d9074fd5e52439be88", [:rebar3], [], "hexpm"},
  "nimble_parsec": {:hex, :nimble_parsec, "0.4.0", "ee261bb53214943679422be70f1658fff573c5d0b0a1ecd0f18738944f818efe", [:mix], [], "hexpm"},
  "number": {:hex, :number, "0.5.7", "da1ee0434f3c7b84360d33b335d0ee2f86f247f45fbe2e7636e6f55680b464dd", [:mix], [{:decimal, "~> 1.2", [hex: :decimal, repo: "hexpm", optional: false]}], "hexpm"},
  "observer_cli": {:hex, :observer_cli, "1.4.1", "8c829bb28e3f316a361216314405fe296d8dd2fd7409dfa53fa4ac1cff3797a0", [:rebar3], [{:recon, "2.3.6", [hex: :recon, repo: "hexpm", optional: false]}], "hexpm"},
  "parent": {:hex, :parent, "0.5.1", "fed79e79402a1f91bffb117e0ef01fd802345dde79fb70df317f8857f2679237", [:mix], [], "hexpm"},
  "parse_trans": {:hex, :parse_trans, "3.3.0", "09765507a3c7590a784615cfd421d101aec25098d50b89d7aa1d66646bc571c1", [:rebar3], [], "hexpm"},
  "phoenix": {:hex, :phoenix, "1.4.0", "56fe9a809e0e735f3e3b9b31c1b749d4b436e466d8da627b8d82f90eaae714d2", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 1.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:plug, "~> 1.7", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 1.0 or ~> 2.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}], "hexpm"},
  "phoenix_ecto": {:hex, :phoenix_ecto, "4.0.0", "c43117a136e7399ea04ecaac73f8f23ee0ffe3e07acfcb8062fe5f4c9f0f6531", [:mix], [{:ecto, "~> 3.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 2.9", [hex: :phoenix_html, repo: "hexpm", optional: true]}, {:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
  "phoenix_html": {:hex, :phoenix_html, "2.12.0", "1fb3c2e48b4b66d75564d8d63df6d53655469216d6b553e7e14ced2b46f97622", [:mix], [{:plug, "~> 1.5", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
  "phoenix_live_reload": {:hex, :phoenix_live_reload, "1.2.0", "3bb31a9fbd40ffe8652e60c8660dffd72dd231efcdf49b744fb75b9ef7db5dd2", [:mix], [{:file_system, "~> 0.2.1 or ~> 0.3", [hex: :file_system, repo: "hexpm", optional: false]}, {:phoenix, "~> 1.4", [hex: :phoenix, repo: "hexpm", optional: false]}], "hexpm"},
  "phoenix_pubsub": {:hex, :phoenix_pubsub, "1.1.1", "6668d787e602981f24f17a5fbb69cc98f8ab085114ebfac6cc36e10a90c8e93c", [:mix], [], "hexpm"},
  "plug": {:hex, :plug, "1.7.1", "8516d565fb84a6a8b2ca722e74e2cd25ca0fc9d64f364ec9dbec09d33eb78ccd", [:mix], [{:mime, "~> 1.0", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_crypto, "~> 1.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}], "hexpm"},
  "plug_cowboy": {:hex, :plug_cowboy, "2.0.0", "ab0c92728f2ba43c544cce85f0f220d8d30fc0c90eaa1e6203683ab039655062", [:mix], [{:cowboy, "~> 2.5", [hex: :cowboy, repo: "hexpm", optional: false]}, {:plug, "~> 1.7", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
  "plug_crypto": {:hex, :plug_crypto, "1.0.0", "18e49317d3fa343f24620ed22795ec29d4a5e602d52d1513ccea0b07d8ea7d4d", [:mix], [], "hexpm"},
  "poison": {:hex, :poison, "3.1.0", "d9eb636610e096f86f25d9a46f35a9facac35609a7591b3be3326e99a0484665", [:mix], [], "hexpm"},
  "postgrex": {:hex, :postgrex, "0.14.0", "f3d6ffea1ca8a156e0633900a5338a3d17b00435227726baed8982718232b694", [:mix], [{:connection, "~> 1.0", [hex: :connection, repo: "hexpm", optional: false]}, {:db_connection, "~> 2.0", [hex: :db_connection, repo: "hexpm", optional: false]}, {:decimal, "~> 1.0", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}], "hexpm"},
  "ranch": {:hex, :ranch, "1.7.0", "9583f47160ca62af7f8d5db11454068eaa32b56eeadf984d4f46e61a076df5f2", [:rebar3], [], "hexpm"},
  "recon": {:hex, :recon, "2.3.6", "2bcad0cf621fb277cabbb6413159cd3aa30265c2dee42c968697988b30108604", [:rebar3], [], "hexpm"},
  "sentry": {:hex, :sentry, "7.0.3", "093fa4b6937760afb9a5fcb0e4a9092a305b6c0ff26a710e977614b201feab75", [:mix], [{:hackney, "~> 1.8 or 1.6.5", [hex: :hackney, repo: "hexpm", optional: false]}, {:jason, "~> 1.1", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix, "~> 1.3", [hex: :phoenix, repo: "hexpm", optional: true]}, {:plug, "~> 1.6", [hex: :plug, repo: "hexpm", optional: true]}, {:plug_cowboy, "~> 1.0 or ~> 2.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}], "hexpm"},
  "site_encrypt": {:git, "https://github.com/sasa1977/site_encrypt.git", "c76576b8948b77d74cb1808b5eedc75c983a545c", []},
  "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.4", "f0eafff810d2041e93f915ef59899c923f4568f4585904d010387ed74988e77b", [:make, :mix, :rebar3], [], "hexpm"},
  "swoosh": {:hex, :swoosh, "0.20.0", "9a6c13822c9815993c03b6f8fccc370fcffb3c158d9754f67b1fdee6b3a5d928", [:mix], [{:cowboy, "~> 1.0.1 or ~> 1.1 or ~> 2.4", [hex: :cowboy, repo: "hexpm", optional: true]}, {:gen_smtp, "~> 0.12", [hex: :gen_smtp, repo: "hexpm", optional: true]}, {:hackney, "~> 1.9", [hex: :hackney, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}, {:mime, "~> 1.1", [hex: :mime, repo: "hexpm", optional: false]}, {:plug, "~> 1.4", [hex: :plug, repo: "hexpm", optional: true]}], "hexpm"},
  "telemetry": {:hex, :telemetry, "0.2.0", "5b40caa3efe4deb30fb12d7cd8ed4f556f6d6bd15c374c2366772161311ce377", [:mix], [], "hexpm"},
  "timex": {:hex, :timex, "3.4.2", "d74649c93ad0e12ce5b17cf5e11fbd1fb1b24a3d114643e86dba194b64439547", [:mix], [{:combine, "~> 0.10", [hex: :combine, repo: "hexpm", optional: false]}, {:gettext, "~> 0.10", [hex: :gettext, repo: "hexpm", optional: false]}, {:tzdata, "~> 0.1.8 or ~> 0.5", [hex: :tzdata, repo: "hexpm", optional: false]}], "hexpm"},
  "turbolinks": {:hex, :turbolinks, "1.0.1", "ee48ddc70884c738ff5a58a004d2565276cf5841f3ef43c22afd884d0404603e", [:mix], [{:phoenix, "~> 1.0", [hex: :phoenix, repo: "hexpm", optional: false]}, {:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
  "tzdata": {:hex, :tzdata, "0.5.19", "7962a3997bf06303b7d1772988ede22260f3dae1bf897408ebdac2b4435f4e6a", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"},
  "unicode_util_compat": {:hex, :unicode_util_compat, "0.4.1", "d869e4c68901dd9531385bb0c8c40444ebf624e60b6962d95952775cac5e90cd", [:rebar3], [], "hexpm"},
  "x509": {:hex, :x509, "0.3.0", "c6f3db66960c6e4f424d1e6cca5c7d730e0a577af8dc115a613f4560ce1df6d3", [:mix], [], "hexpm"},
}

let me know if you need anything more. I you have ideas on where to search.
I am using postgres 11.1, that might be the cause of the error

Unable to override `:migration_lock` parameter when used in connection configuration

I'm working on MSSQL adapter and having trouble to understand how to disable default "FOR UPDATE" :migration_lock value without implementing my custom Ecto.Adapter.SQL.lock_for_migrations/4 function in adapter.

The meta I'm getting looks like this

%{
  cache: #Reference<0.224688069.2605580292.202322>,
  opts: [
    timeout: 15000,
    pool_timeout: 5000,
    pool_size: 5,
    pool: DBConnection.Ownership
  ],
  pid: #PID<0.290.0>,
  sql: Ecto.Adapters.MsSql.Connection,
  telemetry: {:debug, [], [:ecto, :integration, :test_repo, :query]}
}

Where TestRepo is configured like this

Application.put_env(
  :ecto_sql,
  TestRepo,
  migration_lock: nil, 
  hostname: System.get_env("SQL_HOSTNAME"),
  username: System.get_env("SQL_USERNAME"),
  password: System.get_env("SQL_PASSWORD"),
  database: "ecto_test",
  pool: Ecto.Adapters.SQL.Sandbox,
  pool_size: 5,
  set_allow_snapshot_isolation: :on,
  filter_null_on_unique_indexes: true
)

(DBConnection.ConnectionError) in tests

Since upgrading to Ecto 3.0.3 we randomly get this error in tests:

(DBConnection.ConnectionError) connection not available and request was dropped from queue after 254ms

It seems to only happen sometimes, but once it happens all subsequent tests are affected.

index names changed between ecto 2.2 and ecto_sql 3.0.5

I am working on migrating a Postgres back-end from Ecto 2.2 to Ecto 3.0.5 (To coincide with the 1.3 Triplex release). This is for these indices:

From my migration file:

create(unique_index(:account, ["(lower(email))"]))
create(unique_index(:account, ["(lower(username))"]))

In Ecto 2.2, the index names in postgres are created as:

account_lower_username__index
account_lower_email__index

But in Ecto 3.0.5, the index names in postgres are created as:

account__lower_username_index
account__lower_email_index

Please note that the double underscore in the index names moved between the two versions. Because this is a custom index, I must specify the constraint name in my changeset to handle any violations correctly:

    |> unique_constraint(:username, name: :account_lower_username__index)
    |> unique_constraint(:email, name: :account_lower_email__index)

Because this is a multi-tenant application, I will have Ecto 2.2 created indexes and Ecto 3.0.5 created indexes being used by the same changeset.

My workaround is to edit my existing migration documents to include the legacy name when creating those indices. I use:

create(unique_index(:account, ["(lower(email))"], name: :account_lower_email__index))
create(unique_index(:account, ["(lower(username))"], name: :account_lower_username__index))

And the changesets for current tenant schemas continue to work for new tenants.

Other options would have been to either drop and recreate indices in a migration to get the new naming convention (specifying the old name due to the naming convention change), or to add additional unique constraints to the changesets for all possible index names moving forward.

Environment

• Elixir version (elixir -v): 1.8.1 on OTP 21
• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): Postgres 10.5.1 (docker)
• Ecto version (mix deps): ecto 2.2 to ecto_sql 3.0.5
• Database adapter and version (mix deps): postgrex 0.13.5 to 0.14.1
• Operating system: MacOS 10.14.3

Current behavior

Index names for a custom index field changed between versions. Due to multi-tenancy, the unique_constraint in the changeset no longer matches the index names between existing tenants and new tenants.

Expected behavior

The behavior should be consistent, or noted in the changelog.

Add `table_exists?`

We started the discussion about making the ecto.load smarter and skip loaded databases. We'll need a function that can check if a table exists.

In Postgres we can check like this:

SELECT to_regclass('schema_name.table_name')

In MySQL we can check like this:

SELECT table_name
FROM information_schema.tables
WHERE table_schema = 'yourdb' 
    AND table_name = 'testtable'
LIMIT 1;

We thought that we can add this function to Ecto.Adapters.SQL, however, I wonder how we can do it?

Today the Adapters.SQL has these behaviours:

Ecto.Adapter
Ecto.Adapter.Migration
Ecto.Adapter.Queryable
Ecto.Adapter.Schema
Ecto.Adapter.Transaction

Wich one should we add the function table_existence_check_query?

We have also the

Ecto.Adapter.Migration
Ecto.Adapter.Structure 

I'm a little lost where I should put the callback.

Additional links

The discussion started here: https://groups.google.com/forum/#!topic/elixir-ecto/8aw2DL_c9Yc
The idea of the feature came from here: #108

Add cache_statement to insert/update/delete

Migration.references/2 should support :prefix

I would expect the following to create a cross-schema reference:

create table(:comments, prefix: :child_app) do
  add :user_id, references("parent.users")
end

Alternatively:

create table(:comments, prefix: :child_app) do
  add :user_id, references("users", prefix: :parent)
end

Incorrect :timeout handling for Postgres and MyXQL

ecto_sql/lib/ecto/adapters/postgres.ex

Line 40 in 3bad7d2

* `:timeout` - The default timeout to use on queries, defaults to `15000`

ecto_sql/lib/ecto/adapters/myxql.ex

Line 24 in 3bad7d2

* `:timeout` - The default timeout to use on queries, defaults to `15000`

However neither adapters actually supports this option.

Postgrex.start_link takes this option but it does something different:

* `:timeout` - Socket receive timeout when idle in milliseconds (default:
`#{@timeout}`);

https://github.com/elixir-ecto/postgrex/blob/v0.14.1/lib/postgrex.ex#L46:L47

We should either change the code or the docs, thoughts? Note, for Ecto.Adapters.MySQL/Mariaex the docs are the same and the code works as advertised.

Do not pass Ecto.LogEntry as a whole to telemetry

Calling a mysql stored procedure which includes transactions causes a case clause error

Environment

• Elixir version: Elixir 1.7.3 (compiled with Erlang/OTP 21)
• Phoenix version: 1.4.0
• Database and version: MariaDB 10.3.10
• Ecto version: ecto 3.0.1, ecto_sql 3.0.0
• Database adapter and version: mariaex 0.9.9
• Operating system: macOS 10.14

Current behavior

Running the following as part of a phoenix conn test using MyappWeb.ConnCase causes the test to pass, but the exception is printed to the terminal after the test has completed. I believe it could be something to do with the COMMIT inside the stored procedure.

stored procedure:

create table db_test
(
  id    int auto_increment
    primary key,
  `key` varchar(32) default 'NULL' null,
  val   text default 'NULL'        null
);


create procedure db_test(IN inKey varchar(32), IN inVal text)
  BEGIN
    START TRANSACTION;
    INSERT INTO db_test (`key`, val) VALUES (inKey, inVal);
    COMMIT;
  END;

controller:

  def test_db(conn, _params) do
    {:ok, v} = Repo.query("CALL test.db_test('3', 'maybe')")

    json(conn, v.rows)
  end

exception:

10:02:07.200 [error] GenServer #PID<0.456.0> terminating
** (CaseClauseError) no case clause matching: {:idle, %Mariaex.Protocol{binary_as: :field_type_var_string, buffer: "", cache: #Reference<0.2931625945.744620033.246778>, catch_eof: false, connection_id: #PID<0.418.0>, cursors: %{}, datetime: :structs, deprecated_eof: true, json_library: Poison, lru_cache:{100, #Reference<0.2931625945.744620033.246779>}, opts: [socket_options: [], sock_type: :tcp, cache_size: 100, port: 3306, datetime: :structs, repo: Myapp.Repo, telemetry_prefix: [:myapp, :repo], otp_app: :myapp, timeout: 15000, pool_timeout: 5000, pool_size: 10, username: "xxxx", password: "xxxx", database: "myapp_test", hostname: "localhost"], seqnum: 0, sock: {Mariaex.Connection.Tcp, #Port<0.82>}, ssl_conn_state: :not_used, state: :running, state_data: nil, timeout: 15000, transaction_status: :idle}}
    (ecto_sql) lib/ecto/adapters/sql/sandbox.ex:514: Ecto.Adapters.SQL.Sandbox.pre_checkin/4
    (db_connection) lib/db_connection/ownership/proxy.ex:210: DBConnection.Ownership.Proxy.pool_done/5
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: {:DOWN, #Reference<0.2931625945.744488961.246901>, :process, #PID<0.455.0>, :shutdown}
10:02:07.200 [error] Mariaex.Protocol (#PID<0.418.0>) disconnected: ** (DBConnection.ConnectionError)client #PID<0.456.0> exited

Expected behavior

The test will pass and no error will be printed to the console. As happened with Ecto 2 / Phoenix 1.3.4.

Migrations under "test/support" not detected

Environment

• Elixir version (elixir -v):

Erlang/OTP 21 [erts-10.1.1] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [hipe] [dtrace]

Elixir 1.7.4 (compiled with Erlang/OTP 21)

• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): Postgres 11
• Ecto version (mix deps): 3.0.0
• Database adapter and version (mix deps): Postgrex 0.14.0
• Operating system: MacOS 10.14.1

Current behavior

I'm migrating a library from ecto 2.2 to ecto 3.0.
The library contains a test Repo under test/support/repo.ex and associated migrations under test/support/migrations/*.exs.

I've configured the priv: option for the repo accordingly for the test Mix config:

config :my_lib, MyLib.Test.Repo,
  database: "my_lib_test",
  username: "postgres",
  password: "postgres",
  priv: "test/support"

Running MIX_ENV=test mix ecto.migrations produces:

Repo: MyLib.Test.Repo

  Status    Migration ID    Migration Name
--------------------------------------------------

Similarly, MIX_ENV=test mix ecto.migrate does not apply any migrations:

mix ecto.migrate

21:30:12.472 [debug] QUERY OK source="schema_migrations" db=1.0ms
SELECT s0."version"::bigint FROM "schema_migrations" AS s0 FOR UPDATE []

21:30:12.474 [info]  Already up

Expected behavior

The migration scripts in the test/support/migrations directory should be applied.

I believe this behaviour is due to the migrations path being initially validated relative to the mix project: https://github.com/elixir-ecto/ecto_sql/blob/master/lib/mix/ecto_sql.ex#L92

Then later re-evaluated relative to the Application.app_dir: https://github.com/elixir-ecto/ecto_sql/blob/master/lib/ecto/migrator.ex#L36 which evaluates to _build/test/lib/my_lib/test/support/migrations. As this directory doesn't exist, no migrations are found.

If the new behaviour is intended, then any advice on managing a test-only repository and migrations for a library, such that the migrations are not included in the final hex package would be appreciated.

Thanks

Schema prefix should override Repo prefix when preloading associations

(Per IRC discussion monologue)

Given modules:

defmodule Post do
  @schema_prefix "public"
  schema "posts" do
    # ...
    has_many :comments, Comment
  end
end

defmodule Comment do
  @schema_prefix "child"
  schema "comments" do
    # ...
    belongs_to :post, Post
  end
end

defmodule Repo do
  # ...
end

config :app, Repo,
  after_connect: {Postgrex, :query!, ["SET search_path TO child", []]}

Given call:

Comment |> preload(:posts) |> Repo.all()

I would expect that the prefix of the particular schema would override the prefix set on the repo. Actually, the prefixes should be in the following order:

repo.prefix || schema.prefix || query.prefix || nil

Database operations within a task within a transaction in test mode hang indefinitely

Environment

• Elixir version (elixir -v): 1.9.0
• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): PostgreSQL 11
• Ecto version (mix deps): 3.1.7
• Database adapter and version (mix deps): postgrex 0.14.3
• Operating system: Ubuntu 18.04 LTS

Current behavior

The following test case in a blank Phoenix project will cause the inner task to hang indefinitely (until Ecto timeout):

 test "should allow creating an item from a task within a transaction", %{conn: _conn} do
    Repo.transaction(fn ->
      Task.async(fn ->
        %Item{}
        |> Repo.insert!()
      end)
      |> Task.await()
    end)
  end

The error is:

  1) test failing test case (EctoFailCaseWeb.PageControllerTest)
     test/ecto_fail_case_web/controllers/page_controller_test.exs:4
     ** (EXIT from #PID<0.363.0>) an exception was raised:
         ** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 978ms. You can configure how long requests wait in the queue using :queue_target and :queue_interval. See DBConnection.start_link/2 for more information
             (ecto_sql) lib/ecto/adapters/sql.ex:621: Ecto.Adapters.SQL.raise_sql_call_error/1
             (ecto) lib/ecto/repo/schema.ex:649: Ecto.Repo.Schema.apply/4
             (ecto) lib/ecto/repo/schema.ex:262: anonymous fn/15 in Ecto.Repo.Schema.do_insert/4
             (ecto) lib/ecto/repo/schema.ex:164: Ecto.Repo.Schema.insert!/4
             (elixir) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2
             (elixir) lib/task/supervised.ex:35: Task.Supervised.reply/5
             (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

Switching the Ecto sandbox to shared mode and toggling async to false doesn't seem to fix the issue. However, the same code in a non-test environment (e.g. IEx running in dev mode) succeeds without issue.

Full example project (essentially just a mix phx.new with the above test case and a single schema) at: https://github.com/dkulchenko/ecto-fail-case

Calling to_sql on an intersect query right after a union query generates identitical SQL

Environment

• Elixir version (elixir -v): 1.7.2, Erlang/OTP 21
• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): PostgreSQL 10.3
• Ecto version (mix deps): 3.0.4
• Database adapter and version (mix deps): postgrex 0.14.0
• Operating system: MacOS High Sierra

Current behavior

If you create a union query and call Repo.to_sql, then change the query to use intersect and call Repo.to_sql again, you get the exact same generated SQL (the UNION isn't changing to INTERSECT)

I created a failing test case in my fork of ecto_sql which demonstrates the issue (branch is issue/union-then-intersect):

https://github.com/darinwilson/ecto_sql/blob/b70c96104bc98dc655ac91e288f40846c9ab6ee1/integration_test/sql/sql.exs#L55,L65

I ran this with ECTO_ADAPTER=pg - I haven't tried it with MySQL.

Expected behavior

UNION should change to INTERSECT in the second to_sql call.

3.0 release?

Ecto 3.0 was released which references this project as the upgrade path for database users. I see that a release candidate was published a few days ago. Are there any issues preventing the release candidate getting promoted to stable?

Ecto Mix tasks do not read application environment only global configuration

Environment

• Elixir version (elixir -v):

  Erlang/OTP 21 [erts-10.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]
  
  Elixir 1.8.0 (compiled with Erlang/OTP 21)
  

• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): doesn't matter
• Ecto version (mix deps):

  * ecto 3.0.7 (Hex package) (mix)
  * ecto_sql 3.0.5 (Hex package) (mix)
  

• Database adapter and version (mix deps): postgrex 0.14.1 (Hex package) (mix) but doesn't really matter
• Operating system: Darwin 18.2.0 Darwin Kernel Version 18.2.0: Fri Oct 5 19:41:49 PDT 2018; root:xnu-4903.221.2~2/RELEASE_X86_64 x86_64 but doesn't matter as well

Current behavior

In umbrella (haven't tested that in single project repository) when I define my application in mix.exs as:

def application do
  [
    mod: {MyApp.Application, []},
    env: [
      ecto_repos: [MyApp.Repo]
    ]
  ]
end

It states that it cannot find :ecto_repos configuration option in defined applications:

warning: could not find Ecto repos in any of the apps: [:my_app, :my_app_web].

You can avoid this warning by passing the -r flag or by setting the
repositories managed by those applications in your config/config.exs:

    config :my_app, ecto_repos: [...]

Expected behavior

It should load application environment as well as configuration files ("normal" precedence rules should apply).

order_by + distinct builds wrong query

Environment

• Elixir version (elixir -v):

Erlang/OTP 21 [erts-10.1.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] [dtrace]

Elixir 1.7.4 (compiled with Erlang/OTP 21)

• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): PG_VERSION=9.6.6
• Ecto version (mix deps):

* ecto 2.2.11 (Hex package) (mix)
  locked at 2.2.11 (ecto) 4bb8f117

• Database adapter and version (mix deps):

* postgrex 0.13.5 (Hex package) (mix)
  locked at 0.13.5 (postgrex) 3d931aba

• Operating system: OSX 10.14.1

Current behavior

This code:

queryable
|> order_by([desc: :begin_at, desc: :id])
|> distinct([:begin_at, :id])

generates the following request:

SELECT DISTINCT ON (m0."begin_at",
                    m0."id") m0."id",
                   ...)
...
GROUP BY m0."id"
ORDER BY m0."begin_at",
         m0."id",
         m0."begin_at" DESC,
         m0."id" DESC

Expected behavior

SELECT DISTINCT ON (m0."begin_at",
                    m0."id") m0."id",
                   ...)
...
GROUP BY m0."id"
ORDER BY m0."begin_at" DESC,
         m0."id" DESC

It appears there is a difference between these two way of writing the order_by()s.

See elixir-ecto/ecto#504 for initial discussion on distinct_on+order_by.
See elixir-ecto/ecto#1669 for what seems to be a very similar bug report.

Note that I explicitly select distinct order_by fields to avoid

SELECT DISTINCT ON expressions must match initial ORDER BY expressions

Current workaround:

1. take out order_bys
2. convert to_sql
3. generate correct text from order_bys

EDIT: I haven't tried ecto_sql v3 but code in connection.sql looks similiar enough.

mix ecto.migrations fails on Erlang error: :timeout_value

Environment

• Elixir version (elixir -v): Elixir 1.7.3 (compiled with Erlang/OTP 21)
• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): PostgreSQL 10.5
• Ecto version (mix deps): 3.0.0-rc.1
• Database adapter and version (mix deps): postgrex 0.14.0-rc.1
• Operating system: macOS

Current behavior

mix ecto.migrations fails with this error on 3.0.0-rc1:

** (ErlangError) Erlang error: :timeout_value
    (stdlib) proc_lib.erl:933: :proc_lib.stop/3
    (elixir) lib/gen_server.ex:880: GenServer.stop/3
    (ecto_sql) lib/mix/tasks/ecto.migrations.ex:45: anonymous fn/3 in Mix.Tasks.Ecto.Migrations.run/3
    (elixir) lib/enum.ex:1314: Enum."-map/2-lists^map/1-0-"/2
    (ecto_sql) lib/mix/tasks/ecto.migrations.ex:38: Mix.Tasks.Ecto.Migrations.run/3
    (mix) lib/mix/task.ex:316: Mix.Task.run_task/3
    (mix) lib/mix/cli.ex:79: Mix.CLI.run_task/2

Expected behavior

I think the call to repo.stop(pid) should be replaced to repo.stop()

https://github.com/elixir-ecto/ecto_sql/blob/master/lib/mix/tasks/ecto.migrations.ex#L45