elixir-plug / plug_cowboy Goto Github PK
View Code? Open in Web Editor NEWPlug adapter for the Cowboy web server
License: Other
Plug adapter for the Cowboy web server
License: Other
Hi,
It looks like the exception logger from the plug here adds the entire conn
to the logger metadata. The conn can contain sensitive information such as headers with authorization keys, cookies with sensitive values, or maybe even request/response bodies with sensitive data in them that should not be logged. In cases where metadata: :all
is used this leads to those sensitive values being logged out which may be great for debugging, is not great for production environments where those values need to be scrubbed and or other serious implications.
Should this be added to the metadata by default given the high sensitivity of those values and the somewhat common practice to allow all metadata?
Additionally, I do not see any mention of this in the docs for the plug. Maybe it's somewhere in the phoenix or plug docs though and I missed it.
Last but not least, thank you for the awesome project, we really appreciate all the hard work put into this community <3
Elixir 1.10.2 (compiled with Erlang/OTP 21)
Mix 1.10.2 (compiled with Erlang/OTP 21)
Plug 2.1
Hello all,
Should I open a quick pull-request to update the documentation?
When I tried to add the Plug to my application's Supervision tree as specified by the documentation on Hex:
children = [
{Plug.Cowboy, scheme: :http, plug: MyApp, options: [port: 4040]}
]
Supervisor.start_link(children, strategy: :one_for_one)
I got the following error message after running the command mix run --no-halt
:
20:00:38.644 [info] Starting Concise application.
20:00:38.689 [info] Application concise exited: Concise.Application.start(:normal, []) returned an error: shutdown: failed to start child: {:ranch_listener_sup, ConcisePlug.HTTP}
** (EXIT) shutdown: failed to start child: :ranch_acceptors_sup
** (EXIT) :badarg
** (Mix) Could not start application concise: Concise.Application.start(:normal, []) returned an error: shutdown: failed to start child: {:ranch_listener_sup, ConcisePlug.HTTP}
** (EXIT) shutdown: failed to start child: :ranch_acceptors_sup
** (EXIT) :badarg
However, after following the instructions provided by the project's README, it worked out fine at the end.
children = [
Plug.Cowboy.child_spec(scheme: :http, plug: MyRouter, port: 4001)
]
# See https://hexdocs.pm/elixir/Supervisor.html
# for other strategies and supported options
opts = [strategy: :one_for_one, name: MyApp.Supervisor]
Supervisor.start_link(children, opts)
We have a Phoenix application that generates very large API responses (approaching 1Gb in size) in both JSON and CSV formats. To keep memory usage down, we are streaming these large responses back to the client using Plug.Conn.send_chunked
and Plug.Conn.chunk
, and we "consume" the stream using Enum.reduce_while
, as suggested in the documentation for Plug.Conn.chunk
.
While this approach has worked fine for API responses that are not extraordinarily large, it has been causing problems for our very large responses. Specifically, when downloading these large API responses (using curl
for example), the download will start out reasonably fast, but then after a period of time it will slow down to a crawl. By using the observer
tool I was able to pinpoint the problem: the cowboy_clear:connection_process
process gets overloaded with millions of messages in its mailbox and also consumes a large amount of memory. When the system gets into this state is when the chunked HTTP response slows down so much that it has virtually stopped altogether.
(Incidentally, aborting the download on the client-side (such as Ctrl-C
on curl
) will eventually cause the server-side to clean itself up and the memory usage quickly returns to normal levels.)
The root problem appears to be that Plug.Cowboy.Conn.chunk/2
calls :cowboy_req.stream_body/3
, which immediately sends a message to its stream handler process. Since there is no backpressure mechanism, this means that the Stream
that is being chunked back to the client is fully "realized" almost immediately, so memory usage spikes to the hold the entire stream in memory. It also means that the Cowboy stream handler process gets overloaded with messages in its process mailbox, and this in turn causes the extreme slow down mentioned above.
I cannot share our real codebase that demonstrates this issue, but I have put together a demo project that simulates the scenario and reproduces the behavior.
In this demo project, there is a controller with a flood
action that generates a large stream of CSV data and streams it to the client: https://github.com/pro-football-focus/plug_chunk_backpressure_demo/blob/25f93204024e479a965cda61ddb401ff792de8e6/lib/chunk_backpressure_demo_web/controllers/demo_controller.ex#L6
This flood
action demonstrates the behavior described above, where the cowboy_clear:connection_process
mailbox gets overloaded and the streaming response slows to a crawl.
In that same controller, there is another action called backpressure
, which streams the same large CSV response to the client, but in which I've implemented a rudimentary backpressure mechanism: https://github.com/pro-football-focus/plug_chunk_backpressure_demo/blob/25f93204024e479a965cda61ddb401ff792de8e6/lib/chunk_backpressure_demo_web/controllers/demo_controller.ex#L12
The backpressure
action checks the size of the process mailbox for the cowboy stream handler, and if it goes over a certain threshold (arbitrarily set at 500 in the demo), then it waits until the mailbox size goes back down below that threshold before sending the chunk to cowboy. (This is done using the wait_for_it
library, but that is an incidental detail and not important to demonstrate the issue.)
This rudimentary backpressure mechanism not only keeps the memory usage low and prevents the overload of the stream handler process mailbox, but it also speeds up the response dramatically! The CSV data produced in the demo is 300Mb in size, and downloading it with curl
took nearly 6 minutes using the flood
action (with no backpressure) but took only 3.5 seconds with the backpressure
action:
$ time curl -o flood.csv http://localhost:4000/flood
curl -o flood.csv http://localhost:4000/flood 2.88s user 27.72s system 8% cpu 5:50.33 total
$ time curl -o backpressure.csv http://localhost:4000/backpressure
curl -o backpressure.csv http://localhost:4000/backpressure 0.34s user 2.95s system 93% cpu 3.518 total
I think this convincingly demonstrates the need for a good backpressure mechanism in Plug.Cowboy.Conn.chunk/2
.
I don't intend to suggest that inspecting the process mailbox size like I've done in the demo should be the basis for such a backpressure mechanism, though. In fact, it is quite possible that there is something like a quantum "observer effect" happening here: the very act of checking the size of the mailbox slows things down enough to "accidentally" provide backpressure. That said, I'm hoping that someone on the plug_cowboy
team can come up with a production-ready backpressure mechanism that allows for streaming large responses efficiently.
Looks like the release v2.6.1 is cut on non main branch. The changelog at the master is still for v2.6.0
. Some code changes are missing as well.
Latest hex package is v2.6.1
: https://diff.hex.pm/diff/plug_cowboy/2.6.0..2.6.1.
Hello,
I am trying to send a list of values in query parameters using the key-value notation, but I am not getting the expected results.
Key-Value
We can pass query parameters as key-value pairs like this:
?obj[eq]=val
Fetched query parameters for this request would be:
%{ "obj" => %{"eq" => "val"} }
List of value
Similarly we can pass a list using the array notation:
?arr[]=val1&arr[]=val2
This would yield:
%{ "arr" => ["val1", "val2"] }
Key-Value where value is a list
But if we try passing a list of key-value pairs we only get the last value
?obj_arr[in]=val1&obj_arr[in]=val2
This yields:
%{ "obj_arr" => %{"in" => "val2"} }
Expected result
It would be nice, if the request above would result in:
%{ "obj_arr" => %{"in" =>[ "val1", "val2"]} }
I am using:
Erlang/OTP 26
Elixir 1.16.0
plug_cowboy 2.7.0
$ mix test
warning: use Mix.Config is deprecated. Use the Config module instead
config/config.exs:1
* creating test/fixtures/ssl/server_key.pem
* creating test/fixtures/ssl/server_key_enc.pem
* creating test/fixtures/ssl/other_key.pem
* creating test/fixtures/ssl/other_key_enc.pem
* creating test/fixtures/ssl/client_key.pem
* creating test/fixtures/ssl/client_key_enc.pem
* creating test/fixtures/ssl/cacerts.pem
* creating test/fixtures/ssl/alternate_cacerts.pem
* creating test/fixtures/ssl/chain.pem
* creating test/fixtures/ssl/ca_and_chain.pem
* creating test/fixtures/ssl/expired_chain.pem
* creating test/fixtures/ssl/revoked_chain.pem
* creating test/fixtures/ssl/alternate_chain.pem
* creating test/fixtures/ssl/valid.pem
* creating test/fixtures/ssl/wildcard.pem
* creating test/fixtures/ssl/expired.pem
* creating test/fixtures/ssl/revoked.pem
* creating test/fixtures/ssl/selfsigned.pem
* creating test/fixtures/ssl/client.pem
The certificates and keys can be found in test/fixtures/ssl.
WARNING: only use the generated certificates for testing in a closed network
environment, such as running a development server on `localhost`.
For production, staging, or testing servers on the public internet, obtain a
proper certificate, for example from [Let's Encrypt](https://letsencrypt.org).
..................
21:15:34.231 [warning] Description: 'Authenticity is not established by certificate path validation'
Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'
21:15:34.265 [error] :closed
1) test http2 server push without automatic mime type (Plug.Cowboy.ConnTest)
test/plug/cowboy/conn_test.exs:607
** (EXIT from #PID<0.696.0>) bad return value: {:error, :closed}
21:15:34.262 [error] Ranch listener Plug.Cowboy.ConnTest.HTTPS had connection process started with :cowboy_tls:start_link/4 at #PID<0.704.0> exit with reason: {:undef, [{:ssl, :ssl_accept, [{:sslsocket, {:gen_tcp, #Port<0.18>, :tls_connection, [option_tracker: #PID<0.577.0>, session_tickets_tracker: :disabled, session_id_tracker: #PID<0.578.0>]}, [#PID<0.702.0>, #PID<0.701.0>]}, [], 5000], []}, {:ranch_ssl, :handshake, 3, [file: '/tmp/plug_cowboy/deps/ranch/src/ranch_ssl.erl', line: 142]}, {:ranch, :handshake, 2, [file: '/tmp/plug_cowboy/deps/ranch/src/ranch.erl', line: 243]}, {:cowboy_tls, :connection_process, 4, [file: '/tmp/plug_cowboy/deps/cowboy/src/cowboy_tls.erl', line: 43]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}
......
21:15:35.289 [error] Ranch listener Plug.Cowboy.ConnTest.HTTPS had connection process started with :cowboy_tls:start_link/4 at #PID<0.730.0> exit with reason: {:undef, [{:ssl, :ssl_accept, [{:sslsocket, {:gen_tcp, #Port<0.22>, :tls_connection, [option_tracker: #PID<0.577.0>, session_tickets_tracker: :disabled, session_id_tracker: #PID<0.578.0>]}, [#PID<0.729.0>, #PID<0.728.0>]}, [], 5000], []}, {:ranch_ssl, :handshake, 3, [file: '/tmp/plug_cowboy/deps/ranch/src/ranch_ssl.erl', line: 142]}, {:ranch, :handshake, 2, [file: '/tmp/plug_cowboy/deps/ranch/src/ranch.erl', line: 243]}, {:cowboy_tls, :connection_process, 4, [file: '/tmp/plug_cowboy/deps/cowboy/src/cowboy_tls.erl', line: 43]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}
2) test exposes peer data (Plug.Cowboy.ConnTest)
test/plug/cowboy/conn_test.exs:624
match (=) failed
code: assert {:ok, 200, _headers, client} = :hackney.get("https://127.0.0.1:8004/peer_data", [], "", opts)
left: {:ok, 200, _headers, client}
right: {:error, :closed}
stacktrace:
test/plug/cowboy/conn_test.exs:634: (test)
...
21:15:35.332 [warning] Description: 'Authenticity is not established by certificate path validation'
Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'
21:15:35.336 [error] Ranch listener Plug.Cowboy.ConnTest.HTTPS had connection process started with :cowboy_tls:start_link/4 at #PID<0.746.0> exit with reason: {:undef, [{:ssl, :ssl_accept, [{:sslsocket, {:gen_tcp, #Port<0.26>, :tls_connection, [option_tracker: #PID<0.577.0>, session_tickets_tracker: :disabled, session_id_tracker: #PID<0.578.0>]}, [#PID<0.745.0>, #PID<0.743.0>]}, [], 5000], []}, {:ranch_ssl, :handshake, 3, [file: '/tmp/plug_cowboy/deps/ranch/src/ranch_ssl.erl', line: 142]}, {:ranch, :handshake, 2, [file: '/tmp/plug_cowboy/deps/ranch/src/ranch.erl', line: 243]}, {:cowboy_tls, :connection_process, 4, [file: '/tmp/plug_cowboy/deps/cowboy/src/cowboy_tls.erl', line: 43]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}
3) test https (Plug.Cowboy.ConnTest)
test/plug/cowboy/conn_test.exs:543
match (=) failed
code: assert {:ok, 200, _headers, client} = :hackney.get("https://127.0.0.1:8004/https", [], "", opts)
left: {:ok, 200, _headers, client}
right: {:error, :closed}
stacktrace:
test/plug/cowboy/conn_test.exs:553: (test)
......
21:15:35.477 [error] Ranch protocol #PID<0.767.0> of listener Plug.Cowboy.ConnTest.HTTP (connection #PID<0.732.0>, stream id 14) terminated
an exception was raised:
** (UndefinedFunctionError) function :erlang.get_stacktrace/0 is undefined or private
:erlang.get_stacktrace()
(cowboy 2.7.0) /tmp/plug_cowboy/deps/cowboy/src/cowboy_stream_h.erl:314: :cowboy_stream_h.request_process/3
(stdlib 3.17) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
..
21:15:35.480 [warning] Description: 'Authenticity is not established by certificate path validation'
Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'
21:15:35.480 [error] :closed
21:15:35.480 [error] Ranch listener Plug.Cowboy.ConnTest.HTTPS had connection process started with :cowboy_tls:start_link/4 at #PID<0.778.0> exit with reason: {:undef, [{:ssl, :ssl_accept, [{:sslsocket, {:gen_tcp, #Port<0.28>, :tls_connection, [option_tracker: #PID<0.577.0>, session_tickets_tracker: :disabled, session_id_tracker: #PID<0.578.0>]}, [#PID<0.777.0>, #PID<0.775.0>]}, [], 5000], []}, {:ranch_ssl, :handshake, 3, [file: '/tmp/plug_cowboy/deps/ranch/src/ranch_ssl.erl', line: 142]}, {:ranch, :handshake, 2, [file: '/tmp/plug_cowboy/deps/ranch/src/ranch.erl', line: 243]}, {:cowboy_tls, :connection_process, 4, [file: '/tmp/plug_cowboy/deps/cowboy/src/cowboy_tls.erl', line: 43]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}
21:15:35.481 [warning] Description: 'Authenticity is not established by certificate path validation'
Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'
21:15:35.483 [error] :closed
21:15:35.483 [error] Ranch listener Plug.Cowboy.ConnTest.HTTPS had connection process started with :cowboy_tls:start_link/4 at #PID<0.787.0> exit with reason: {:undef, [{:ssl, :ssl_accept, [{:sslsocket, {:gen_tcp, #Port<0.30>, :tls_connection, [option_tracker: #PID<0.577.0>, session_tickets_tracker: :disabled, session_id_tracker: #PID<0.578.0>]}, [#PID<0.786.0>, #PID<0.784.0>]}, [], 5000], []}, {:ranch_ssl, :handshake, 3, [file: '/tmp/plug_cowboy/deps/ranch/src/ranch_ssl.erl', line: 142]}, {:ranch, :handshake, 2, [file: '/tmp/plug_cowboy/deps/ranch/src/ranch.erl', line: 243]}, {:cowboy_tls, :connection_process, 4, [file: '/tmp/plug_cowboy/deps/cowboy/src/cowboy_tls.erl', line: 43]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}
21:15:35.483 [warning] Description: 'Authenticity is not established by certificate path validation'
Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'
21:15:35.484 [error] :closed
21:15:35.484 [error] Ranch listener Plug.Cowboy.ConnTest.HTTPS had connection process started with :cowboy_tls:start_link/4 at #PID<0.796.0> exit with reason: {:undef, [{:ssl, :ssl_accept, [{:sslsocket, {:gen_tcp, #Port<0.32>, :tls_connection, [option_tracker: #PID<0.577.0>, session_tickets_tracker: :disabled, session_id_tracker: #PID<0.578.0>]}, [#PID<0.795.0>, #PID<0.794.0>]}, [], 5000], []}, {:ranch_ssl, :handshake, 3, [file: '/tmp/plug_cowboy/deps/ranch/src/ranch_ssl.erl', line: 142]}, {:ranch, :handshake, 2, [file: '/tmp/plug_cowboy/deps/ranch/src/ranch.erl', line: 243]}, {:cowboy_tls, :connection_process, 4, [file: '/tmp/plug_cowboy/deps/cowboy/src/cowboy_tls.erl', line: 43]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}
4) test http2 response (Plug.Cowboy.ConnTest)
test/plug/cowboy/conn_test.exs:585
** (MatchError) no match of right hand side value: {:error, {:bad_return_value, {:error, :closed}}}
code: {:ok, pid} = Kadabra.open('localhost', :https, @http2_opts)
stacktrace:
test/plug/cowboy/conn_test.exs:586: (test)
5) test http2 server push (Plug.Cowboy.ConnTest)
test/plug/cowboy/conn_test.exs:599
** (MatchError) no match of right hand side value: {:error, {:bad_return_value, {:error, :closed}}}
code: {:ok, pid} = Kadabra.open('localhost', :https, @http2_opts)
stacktrace:
test/plug/cowboy/conn_test.exs:600: (test)
6) test http2 early hints (Plug.Cowboy.ConnTest)
test/plug/cowboy/conn_test.exs:592
** (MatchError) no match of right hand side value: {:error, {:bad_return_value, {:error, :closed}}}
code: {:ok, pid} = Kadabra.open('localhost', :https, @http2_opts)
stacktrace:
test/plug/cowboy/conn_test.exs:593: (test)
.
7) test metadata in ranch/cowboy 500 logs (Plug.Cowboy.TranslatorTest)
test/plug/cowboy/translator_test.exs:106
Assertion with =~ failed
code: assert metadata =~ "conn: %Plug.Conn{"
left: "[crash_reason: :undef, domain: [:cowboy]]"
right: "conn: %Plug.Conn{"
stacktrace:
test/plug/cowboy/translator_test.exs:115: (test)
.
8) test ranch/cowboy 500 logs (Plug.Cowboy.TranslatorTest)
test/plug/cowboy/translator_test.exs:28
Assertion with =~ failed
code: assert output =~ ~r"#PID<0\.\d+\.0> running Plug\.Cowboy\.TranslatorTest \(.*\) terminated"
left: "\n21:15:37.512 [error] Ranch protocol #PID<0.1130.0> of listener Plug.Cowboy.TranslatorTest.HTTP (connection #PID<0.1129.0>, stream id 1) terminated\nan exception was raised:\n ** (UndefinedFunctionError) function :erlang.get_stacktrace/0 is undefined or private\n :erlang.get_stacktrace()\n (cowboy 2.7.0) /tmp/plug_cowboy/deps/cowboy/src/cowboy_stream_h.erl:314: :cowboy_stream_h.request_process/3\n (stdlib 3.17) proc_lib.erl:226: :proc_lib.init_p_do_apply/3\n"
right: ~r/#PID<0\.\d+\.0> running Plug\.Cowboy\.TranslatorTest \(.*\) terminated/
stacktrace:
test/plug/cowboy/translator_test.exs:37: (test)
..
9) test ranch/cowboy logs configured statuses (Plug.Cowboy.TranslatorTest)
test/plug/cowboy/translator_test.exs:59
Assertion with =~ failed
code: assert output =~ ~r"#PID<0\.\d+\.0> running Plug\.Cowboy\.TranslatorTest \(.*\) terminated"
left: "\n21:15:40.537 [error] Ranch protocol #PID<0.1458.0> of listener Plug.Cowboy.TranslatorTest.HTTP (connection #PID<0.1457.0>, stream id 1) terminated\nan exception was raised:\n ** (UndefinedFunctionError) function :erlang.get_stacktrace/0 is undefined or private\n :erlang.get_stacktrace()\n (cowboy 2.7.0) /tmp/plug_cowboy/deps/cowboy/src/cowboy_stream_h.erl:314: :cowboy_stream_h.request_process/3\n (stdlib 3.17) proc_lib.erl:226: :proc_lib.init_p_do_apply/3\n"
right: ~r/#PID<0\.\d+\.0> running Plug\.Cowboy\.TranslatorTest \(.*\) terminated/
stacktrace:
test/plug/cowboy/translator_test.exs:71: (test)
Finished in 7.7 seconds (1.6s async, 6.0s sync)
48 tests, 9 failures
Randomized with seed 788721
I found some trace of it in 040c53f but not on master
?
https://phoenixframework.org/blog/phoenix-1-4-0-released references plug_cowboy version 2.5, is that coming out anytime soon?
Hello! just a friendly request to release version v2.2.2
with the new early_error
events...
There are a category of requests that cowboy
itself will respond to before it invokes the Plug.Cowboy.Handler
, and thus they aren't detectable by any telemetry
instrumentation currently in plug
or plug_cowboy
...
The early_error
cowboy callback is implemented, and a subset of requests are logged:
plug_cowboy/lib/plug/cowboy/stream.ex
Lines 27 to 31 in dc2972f
It would be nice to emit a telemetry
event for these requests so that they aren't lost. They consume real resources, but are currently invisible to any monitoring solution based on the exiting Plug events.
Thoughts @josevalim?
> mix deps.compile
# ...
===> Compiling cowboy
src/cowboy_clear.erl:16:2: Warning: behaviour ranch_protocol undefined
src/cowboy_tls.erl:16:2: Warning: behaviour ranch_protocol undefined
===> Compiling src/cowboy_http.erl failed
src/cowboy_http.erl:{155,14}: can't find include lib "cowlib/include/cow_inline.hrl"; Make sure cowlib is in your app file's 'applications' list
src/cowboy_http.erl:{156,14}: can't find include lib "cowlib/include/cow_parse.hrl"; Make sure cowlib is in your app file's 'applications' list
src/cowboy_http.erl:512:11: undefined macro 'IS_TOKEN/1'
src/cowboy_http.erl:658:74: undefined macro 'IS_WS/1'
src/cowboy_http.erl:492:6: function parse_method/4 undefined
src/cowboy_http.erl:646:4: function parse_hd_name/4 undefined
src/cowboy_http.erl:517:1: Warning: function parse_uri/3 is unused
src/cowboy_http.erl:534:1: Warning: function parse_uri_authority/3 is unused
src/cowboy_http.erl:538:1: Warning: function parse_uri_authority/5 is unused
src/cowboy_http.erl:563:1: Warning: function parse_uri_path/5 is unused
src/cowboy_http.erl:573:1: Warning: function parse_uri_query/6 is unused
src/cowboy_http.erl:582:1: Warning: function skip_uri_fragment/6 is unused
Seems like calling up an "app file's 'applications' list" means I should be using:
https://hexdocs.pm/plug_cowboy/Plug.Cowboy.html#child_spec/1
inside lib/<app>/application.ex
-
defmodule Pain do
def start(_type, _args) do
children = [
# Start the Telemetry supervisor
PainWeb.Telemetry,
# Start the Ecto repository
# Pain.Repo,
# Start the PubSub system
{Phoenix.PubSub, name: Pain.PubSub},
# Start Finch
{Finch, name: Pain.Finch},
# Start the Endpoint (http/https)
{ Plug.Cowboy, scheme: :http, plug: PainWeb.Endpoint },
PainWeb.Endpoint,
# Start a worker by calling: Pain.Worker.start_link(arg)
# {Pain.Worker, arg}
]
# See https://hexdocs.pm/elixir/Supervisor.html
# for other strategies and supported options
opts = [strategy: :one_for_one, name: Pain.Supervisor]
Supervisor.start_link(children, opts)
end
# ...
end
And here's config:
# config/config.exs
config :pain, PainWeb.Endpoint,
url: [host: System.get_env("PHX_HOST") || "0.0.0.0"],
adapter: Phoenix.Endpoint.Cowboy2Adapter,
render_errors: [
formats: [html: PainWeb.ErrorHTML, json: PainWeb.ErrorJSON],
layout: false,
],
pubsub_server: Pain.PubSub,
live_view: [signing_salt: "aaaabbbb"]
# ...
# config/prod.exs
import Config
config :pain, PainWeb.Endpoint,
server: true,
http: [ ip: {0, 0, 0, 0}, port: System.get_env("PORT") ],
cache_static_manifest: "priv/static/cache_manifest.json"
# ...
It took me a while to figure out that in order to use Plug.Cowboy.Drainer
with phoenix endpoint one must use it like this:
# lib/myapp/application.ex
def children do
[
# ...
MyappWeb.Endpoint,
{Plug.Cowboy.Drainer, refs: [MyappWeb.Endpoint.HTTP]},
]
end
Would it make sense to mention this in README or Drainer docs?
In order to handle WebSocket connections (or have any other custom cowboy2 handler) alongside using a Plug.Router to route other http requests, I have to manually configure Cowboy's dispatch using the :dispatch
option.
It's great that you have provided this option! π
However, it is only briefly referred to here and there is no mention of the Plug.Adapters.Cowboy.Handler
module which is necessary to configure Plug.Router manually. This makes me wonder if doing this is supported functionality?
It would be nice to have the example:
dispatch: [
{^hostname, [
{"/to_custom_handler", WebServerApp.CustomHandler, []},
{:_, Plug.Adapters.Cowboy.Handler, {WebServerApp.Router, []}}
]}
]
Thanks πΊ - PS The docs may very well already contain this, if so - my mistake π
See failures on https://travis-ci.org/elixir-plug/plug_cowboy/jobs/544024765
Hi.
I put my plug on post request in router:
post("/upload", Project.Plugs.UploadHooks, :call)
There is a small part of plug:
## UploadHooks module
def call(conn, _) do
conn
|> get_req_header("hook-name")
|> trigger_hook(conn, conn.params)
end
## PRE-CREATE hook clauses
## This event will be triggered before an upload is created.
## At this stage, verification and authorization of upload metadata is performed.
defp trigger_hook(["pre-create"], conn, %{
"Upload" => %{"MetaData" => %{"user_id" => user_id, "filename_token" => token}}
}) do
with {:ok, _} <- validate_user_id(user_id),
{:ok, _} <- Druid.find_id_by_token(token),
%User{} <- Accounts.find_user(user_id),
{:ok, _} <- Resources.get_asset_by_token(token) do
Logger.info("""
#{inspect __MODULE__}[pre-create]:
Upload allowed for asset #{inspect token}\
""")
send_resp(conn, 204, "OK")
else
e ->
Logger.info("""
#{inspect __MODULE__}[pre-create]:
Upload not allowed - #{inspect e}
""")
send_resp(conn, 403, "Forbidden")
end
end
And this is how it works: the hook worked successfully and it looks like it will send 204 response, but after 2ms this exception was raised. The client receives 500 status response
Even if i do this then get the same:
conn
|> send_resp(204, "OK")
|> halt()
The v2.5.0
git tag is missing from this repository so the view source button on hexdocs (https://hexdocs.pm/plug_cowboy/Plug.Cowboy.html) gives a 404 (https://github.com/elixir-plug/plug_cowboy/blob/v2.5.0/lib/plug/cowboy.ex#L1)
Continuing discussion from phoenixframework/phoenix#3869...
Current plug telemetry handlers for :stop
and :exception
are susceptible to being terminated during their execution because the cowboy process can exit at any time after the request is complete.
This may be avoided by leveraging the built-in handler system of cowboy, especially cowboy_metrics_h
which provides a way to execute a callback at the end of each request with stats about its execution.
Wiring up the additional handler is trivial, the thing to figure out is what data from it to expose, and what to name the telemetry event...
https://ninenines.eu/docs/en/cowboy/2.8/manual/cowboy_metrics_h/
Should we all the data directly or transform it so we aren't exposing raw cowboy data structures?
Is [:plug_cowboy, :metrics]
a reasonable name since it's providing summarized metrics (ie: not start and stop for spans)? Or [:plug_cowboy, :terminate]
since it's called in the terminate
phase of the cowboy process
After updating to Cowboy version 2.11.0, compatibility issues have arisen with plug_cowboy version 2.7.0, leading to failures in environments where plug_cowboy requests the latest version of Cowboy. This issue manifests specifically when running mix phx.create, resulting in an application that crashes on initial load with a header handling error related to set-cookie directives.
As identified by a Cowboy maintainer, the root cause of this issue stems from Cowboy's recent update to be more explicit in what is not allowed concerning the set-cookie header. The way Plug sets the set-cookie header does not comply with the HTTP specification, as Cowboy cannot handle it like other headers due to its special nature. Previously, this incompatibility did not surface due to coincidental factors but has now become evident with Cowboy's stricter enforcement.
Can we get a proper fix for it?
Has anyone tried using OpenTelemetry with Cowboy and a Plug? The idea is to have a root span start when cowboy starts a request and a child span for the plug duration. I know I can hook into telemetry events but I need to link the created spans together. What are the ways in which you could pass an OpenTelemetry context up to the plug, or is this simply not possible without forking / using ETS?
Asking for a friend ;) opentelemetry-beam/opentelemetry_phoenix#28
I'm trying to use custom options on Plug.Conn.read_body/2
to prevent the reading to stop before the size I need.
# my method
defp get_body(conn) do
IO.inspect(">>>>>> Starting read conn.body")
opts = [
# :length - sets the maximum number of bytes to read from the body on every call, defaults to 8_000_000 bytes
length: 30_000_000,
# :read_length - sets the amount of bytes to read at one time from the underlying socket to fill the chunk, defaults to 1_000_000 bytes
read_length: 30_000_0000,
# :read_timeout - sets the timeout for each socket read, defaults to 15_000 milliseconds
read_timeout: 300_000
]
case read_body(conn, opts) do
{status, binary, conn} ->
IO.inspect(">>>>>> Ending read conn.body: #{status}")
{:ok, binary, conn}
_ ->
:no_body
end
end
result:
13:40:54.025 request_id=2lnq07rbse5509m5ts0005p1 [info] PATCH /files/bc235362-fe22-11e8-9328-3c15c2d33dce
13:40:54.122 request_id=2lnq07rbse5509m5ts0005p1 [debug] Processing with MyAppWeb.UploadController.patch/2
Parameters: %{"uid" => "bc235362-fe22-11e8-9328-3c15c2d33dce"}
Pipelines: [:api]
13:40:54.122 request_id=2lnq07rbse5509m5ts0005p1 [debug] [upload] path chunk of upload bc235362-fe22-11e8-9328-3c15c2d33dce
">>>>>> Starting read conn.body"
">>>>>> Ending read conn.body: more"
[
min_size: 5242880,
part_size: 1300176,
"file.offset": 0,
"file.size": 1794211,
part_size_min_size: true,
"file.offset_min_size_file.size": true
]
last_part?: false
part_too_small?: true
13:41:09.341 request_id=2lnq07rbse5509m5ts0005p1 [info] Sent 409 in 15315ms
{:plug, "1.7.1"},
{:plug_cowboy, "2.0.0"}
{:cowboy, "2.6.1"}
Some configurations that i've tried:
config :my_app, MyApp.Endpoint,
http: [
port: port,
# 3 min in ms
timeout: 600_0000,
protocol_options: [
chunked: true,
http10_keepalive: true,
# 2 min in ms
idle_timeout: 600_0000,
inactivity_timeout: 600_0000,
linger_timeout: 30_0000,
# 2 min in ms
request_timeout: 600_0000,
# 3 min in ms
# shutdown_timeout: 180_000,
# 10 min in ms
shutdown_timeout: 600_0000,
# 5 MB in bytes
# max_empty_lines (5)
# Maximum number of empty lines before a request.
max_empty_lines: 5000,
# max_header_name_length (64)
# Maximum length of header names.
max_header_name_length: 64000,
# max_header_value_length (4096)
# Maximum length of header values.
max_header_value_length: 4_096_000,
# max_headers (100)
# Maximum number of headers allowed per request.
max_headers: 100_000,
# max_keepalive (100)
# Maximum number of requests allowed per connection.
max_keepalive: 100_000,
# max_method_length (32)
# Maximum length of the method.
max_method_length: 32000,
max_skip_body_length: 20_000_000,
max_request_line_length: 2_000_0000
]
]
I noticed that when compress
is set to true, the order of the stream handlers is: [:cowboy_compress_h | @default_stream_handlers]
, where @default_stream_handlers
is [:cowboy_telemetry_h, :cowboy_stream_h]
.
Does this work as I understand it in that the telemetry does not take into account the compression handler? Is this working as intended or should the compression be taken into account? I can imagine this is a breaking change but we could put it behind a flag or support compress: [metrics: true]
or something like that.
Current implementation of connection draining works in a way that make it stop creating new connections, but that do not prevent existing connections from accepting and processing new requests. That mean that single client that is connected to the node will prevent it from going down (potentially disrupting any other client that tries to connect to the given node).
I do not know the solution (maybe this issue should be forwarded to Ranch or Cowboy) but maybe there already exists some solution in these deps that would allow handling such problem.
Hi there, we are running into a problem where the call to chunk/2
is hanging. I've traced it down to the following. Recently, cowboy2 added a backpressure mechanism for streaming content: ninenines/cowboy#1376
This new mechanism sends ack
messages back to the caller process in order to provide the backpressure. As noted in the comments here (added as part of ninenines/cowboy@eaa0526) , it is no longer possible to send streaming data to cowboy2's handler from multiple processes:
https://github.com/ninenines/cowboy/blob/master/src/cowboy_stream_h.erl#L223
However, when we have a simple HTTP controller in phoenix which streams data it appears that it (unsurprisingly) sends to the same HTTP/2 cowboy handler from multiple request handle PIDs when requests come in concurrently through the same HTTP/2 transport. This breaks the contract, and is resulting in the backpressure mechanism hanging when enough data is pushed at once.
The temporary workaround (which may still fail I think) is to increase the new buffer size to a sufficient amount so all data is ack'ed immediately. I'm not sure how to resolve this properly, it seems like the cowboy plug would need to start multiplexing streamed data somehow through a new process π¬
I'm currently trying to use the latest opentelemetry_phoenix
, which depends on telemetry ~> 1.0
via opentelemetry_telemetry ~> 1.0
.
plug_cowboy
is currently my blocker, since it depends on cowboy_telemetry ~> 0.3
. cowboy_telemetry
version 0.4.0 added support for telemetry ~> 1.0
Howdy,
I just ran a new mix phx.create
today and it is broken out of the box. On initial load you see the following error:
{:response_error, :invalid_header, :"Response cookies must be set using cowboy_req:set_resp_cookie/3,4."}
I went down a bit of a rabbit hole trying to get it to work, but luckily had created a new project a couple of days ago which worked and still works fine.
Seems Cowboy has released an update today (2.11) and that doesn't play nice with plug_cowboy, which asks for the latest cowboy (from what I can see). So todays build running with 2.11 doesn't work, but the build with 2.10 works fine.
Problem is that Cowboy isn't an explicit dependency for a new Phoenix project, plug_cowboy is. So we can't downgrade the Cowboy version to a working one.
I have a problem in my web application: some clients don't wait for HTTP response and disconnect early. As a result, the erlang process executing this request on my server terminates instantly, while I need to do some resource cleanup when it happens.
So I need some callback to be called on my Plug to notify me of client closing connection before termination.
Please advise if it's already possible, thanks!
Hi!
My app has been shutting down seemingly randomly. This only happens on my local machine (not on other devs, not in production environments) so I suspect it has something to do with my set up.
There are no logs when my app shuts down. It doesn't seem to correlate to any particular behavior. This has been happening for at least a couple weeks with regular daily development happening.
I finally got a clue by enabling SASL logging:
[error] Child Plug.Cowboy.Drainer of Supervisor MyApp.Endpoint shut down abnormally
** (exit) killed
Pid: #PID<0.1011.0>
Start Call: Plug.Cowboy.Drainer.start_link([refs: [MyAppWeb.Endpoint.HTTP]])
Restart: :permanent
Shutdown: 5000
Type: :worker
[notice] Application my_app exited: shutdown
Based on looking at what Plug.Cowboy.Drainer
does, this seems like the error could be a red herring? Any ideas how I could get more diagnostic information? I highly suspect this is a memory limit or running out of file descriptors or something else specific to my environment but I can't seem to get any more clues.
I'm using plug_cowboy
2.6.0 with Phoenix 1.7.0-rc.2 on Elixir 1.14.3, OTP 25
cowboy version lower than 2.8.0 is using a deprecated_function erlang, get_stacktrace, making all error log to undef error:
Ranch protocol #PID<0.409.0> of listener Plug.Cowboy.TranslatorTest.HTTP (connection #PID<0.408.0>, stream id 1) terminated\n** (exit) :undef
error reproducing:
plug_cowboy: Release v2.5.2
erlang otp: 24
mix test test/plug/cowboy/translator_test.exs
We are creating a new Cowboy stream handler that will emit telemetry events:
Once it's released, we should adopt it in plug_cowboy
and include it by default, and also adopt it for the early_error
logging.
I'm in the process of upgrading an app relying on plug_cowboy
version 1.0.0
.
The changelog (https://github.com/elixir-plug/plug_cowboy/blob/master/CHANGELOG.md) does not mention anything related to 1.0.0
.
Are there breaking changes between 1.0.0
and the rest of the upgrades?
I have investigated and cannot see much changes (v1.0.0...v2.0.0) but would prefer to be sure!
Thank you (I can create a doc PR with my findings to help others).
I'm currently working on a custom Logger
backend to report crashes to a 3rd party service. For HTTP requests, I'd like to pull some metadata from the Plug.Conn
and attach that to the report.
To get access to the conn
in the Logger
backend, I think it could be added to the metadata
here:
plug_cowboy/lib/plug/cowboy/translator.ex
Lines 40 to 44 in a01f629
Thoughts?
If this change makes sense, I can open a PR.
The docs seem to state that you can pass any option from Plug.SSL.configure/1
(see this)
But in practice, that doesn't seem to work for me. I don't know enough about plug to know which rabbit hole to go down for this. Any help is appreciated.
iex)> opts = [
certfile: "/path/to/my.crt",
keyfile: "/path/to/my.key",
host: "coolhost.dev",
rewrite_on: [:x_forwarded_proto],
port: 443
]
iex)> Plug.Cowboy.https MyApp.Router, [], opts
{:error,
{{:shutdown, {:failed_to_start_child, :ranch_acceptors_sup, :badarg}},
{:child, :undefined, {:ranch_listener_sup, MyApp.Router.HTTPS},
{:ranch_listener_sup, :start_link,
[
MyApp.Router.HTTPS,
:ranch_ssl,
%{
connection_type: :supervisor,
max_connections: 16384,
num_acceptors: 100,
socket_opts: [
next_protocols_advertised: ["h2", "http/1.1"],
alpn_preferred_protocols: ["h2", "http/1.1"],
reuse_sessions: true,
secure_renegotiate: true,
certfile: '/path/to/my.crt',
keyfile: '/path/to/my.key',
port: 443,
host: "coolhost.dev",
rewrite_on: [:x_forwarded_proto]
]
},
:cowboy_tls,
%{
connection_type: :supervisor,
env: %{
dispatch: [
{:_, [],
[{:_, [], Plug.Cowboy.Handler, {MyApp.Router, []}}]}
]
},
stream_handlers: [Plug.Cowboy.Stream]
}
]}, :permanent, :infinity, :supervisor, [:ranch_listener_sup]}}}
Also fails when trying to use child_spec/1
iex)> spec = Plug.Cowboy.child_spec(plug: MyApp.Router, scheme: :https, options: opts)
iex)> DynamicSupervisor.start_child(MyApp.Supervisor, spec)
{:error, {:shutdown, {:failed_to_start_child, :ranch_acceptors_sup, :badarg}}}
The only options I can get to work are :certfile
and :keyfile
. This may be user-error and I might just need help understanding how to specify the SSL opts correctly?
ππΌ Howdy!
In NervesHub we use websockets for connections with devices and utilize mTLS with them. We wanted to the ability to prevent socket connections as low as possible with some rate limiting and were enabled to do this with Thousand Island in a past PR.
However, due to some recently discovered problems with socket disconnects, we've had to switch back to cowboy until that can be further investigated. While switching, I found that :ranch
does a :ranch_transport
behavior that can be implemented, but after doing so we realized there is no easy way to use that with :cowboy
or Plug.Cowboy
via Phoenix.
The end result is a somewhat hack replicating the Phoenix.Endpoint.Cowboy2Adapter
and injecting our ranch_transport
module into the specific child_spec generated.
Is it desirable to have that functionality either added here or in Phoenix.Endpoint.Cowboy2Adapter
? I'd be happy to add, but don't want to do the work if not wanted. We can simply leave this small change in until figuring out how Bandit/Thousand Island can be adjusted for our needs and switch back
Hi everybody,
I hope this is not to offtopic, but is it possible to create websocket connections with plug_cowboy?
If yes, could you point my to any resources that show how to do it?
Thanks in advance
Currently Plug.Cowboy.Translator
only logs exceptions that were raised in the Plug pipeline only if the Plug.Exception.status/1
for the error returns a status code greater than or equal to 500:
plug_cowboy/lib/plug/cowboy/translator.ex
Lines 31 to 33 in 8aa34d8
Would you be open for making this behaviour configurable? My thinking is that while implementing Plug.Exception
is a great way to customize status codes (and in case of Phoenix, rendered responses), it's not always desirable to swallow the exception completely.
I'm on ranch 1.5, so this might not be applicable to you.
I'm using Plug.Cowboy with a Phoenix project and can't confirm if this is caused by Phoenix, Plug.Cowboy, or my own local configuration. Phoenix on master and Plug.Cowboy on master.
Ranch expects the transport opts to be a proplist. They are currently being passed as a map.
After reworking plug_cowboy locally to convert the transport opts %{max_connections: 16384, num_acceptors: 100, socket_opts: [port: 4000]}
to a Keyword list/proplist, I was getting a :badarg
exception. I'm not sure where socket_opts
is coming from (maybe Phoenix?), but I pulled the port config out of socket opts and was able to start without crashing.
Here is the specific exception in question:
[info] Application administration exited: Administration.Application.start(:normal, []) returned an error: shutdown: failed to start child: AdministrationWeb.Endpoint
** (EXIT) an exception was raised:
** (FunctionClauseError) no function clause matching in :proplists.get_value/3
(stdlib) proplists.erl:215: :proplists.get_value(:num_acceptors, %{max_connections: 16384, num_acceptors: 100, socket_opts: [port: 4000]}, 10)
(ranch) /data/repos/turret/deps/ranch/src/ranch.erl:116: :ranch.child_spec/5
(plug_cowboy) lib/plug/cowboy.ex:185: Plug.Cowboy.child_spec/1
(phoenix) lib/phoenix/endpoint/cowboy2_adapter.ex:44: Phoenix.Endpoint.Cowboy2Adapter.child_spec/3
(phoenix) lib/phoenix/endpoint/supervisor.ex:106: anonymous fn/6 in Phoenix.Endpoint.Supervisor.server_children/4
(elixir) lib/enum.ex:1925: Enum."-reduce/3-lists^foldl/2-0-"/3
(phoenix) lib/phoenix/endpoint/supervisor.ex:97: Phoenix.Endpoint.Supervisor.server_children/4
(phoenix) lib/phoenix/endpoint/supervisor.ex:57: Phoenix.Endpoint.Supervisor.init/1
(stdlib) supervisor.erl:295: :supervisor.init/1
(stdlib) gen_server.erl:374: :gen_server.init_it/2
(stdlib) gen_server.erl:342: :gen_server.init_it/6
(stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
I expected config :phoenix, :filter_parameters
to filter sensitive parameters from logs. But exceptions logged by PlugCowboy exposes all parameters.
[error] #PID<0.823.0> running MyAppWeb.Endpoint (connection #PID<0.821.0>, stream id 1) terminated
Server: localhost:80 (http)
Request: GET /signin/callback?sensitive-parameters-here
** (exit) an exception was raised:
** (FunctionClauseError) ...
I thought of two solutions, both less than ideal.
:filtered_parameters
are passed to PlugCowboy during start to configure translator. They show up as [FILTERED]
as expected. Possible duplicate code with Phoenix.Description
I'm not sure it's bug or not, but I observe a strange behavior of server. I face with the task of starting two or more supervised http server with different ports, and I did the following:
defmodule Master do
@moduledoc false
use Application
require Logger
@doc false
def start(_type, _args) do
children = [
{
DynamicSupervisor,
strategy: :one_for_one,
name: Master.DynamicSupervisor
}
]
Supervisor.start_link(children, strategy: :one_for_one, name: __MODULE__)
end
def start_server(port) do
DynamicSupervisor.start_child(Master.DynamicSupervisor,{
Plug.Cowboy,
scheme: :http,
plug: Router,
options: [
port: port
]
})
end
end
This module contain just one custom function, it's Master.start_server/1
, it receive the port as argument, and trying to start PlugCowboy with this port as child of DynamicSupervisors.
defmodule Router do
use Plug.Router
require Logger
plug Plug.Parsers, parsers: [{:multipart, length: 20_000_000}]
plug(:match)
plug(:dispatch)
# Creates basic params for Plug.Conn
defp get_base_connection(conn), do: conn |> put_resp_content_type("application/json")
get "/echo", do: conn |> get_base_connection |> send_resp(200, Jason.encode!(%{result: "Success!"}))
# 404 page error
match _, do: send_resp(conn, 404, Jason.encode!(%{code: 404, message: "Page not found"}))
end
iex -S mix
What I've got
I've got
Interactive Elixir (1.9.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>
So far so good. And now, I'm trying to start my first server:
iex(1)> Master.start_server 7000
{:ok, #PID<0.545.0>}
Still okay, but, if I try to run a second server on another port:
iex(2)> Master.start_server 7001
I got this:
03:49:12.712 [error] Failed to start Ranch listener Router.HTTP in :ranch_tcp:listen([cacerts: :..., key: :..., cert: :..., port: 7000]) for reason :eaddrinuse (address already in use)
{:error,
{:shutdown,
{:failed_to_start_child, :ranch_acceptors_sup,
{:listen_error, Router.HTTP, :eaddrinuse}}}}
What's strange?
iex(3)> Master.start_server 7001
{:ok, #PID<0.668.0>}
All log of my console*
Interactive Elixir (1.9.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Master.start_server 7000
{:ok, #PID<0.558.0>}
iex(2)> Master.start_server 7001
03:49:12.712 [error] Failed to start Ranch listener Router.HTTP in :ranch_tcp:listen([cacerts: :..., key: :..., cert: :..., port: 7000]) for reason :eaddrinuse (address already in use)
{:error,
{:shutdown,
{:failed_to_start_child, :ranch_acceptors_sup,
{:listen_error, Router.HTTP, :eaddrinuse}}}}
iex(3)> Master.start_server 7001
{:ok, #PID<0.668.0>}
iex(4)>
I attempted to use master branch's plug and tried to install plug_cowboy.
However, I can not install it, and I get an error like the one below.
Failed to fetch record for 'hexpm/plug_cowboy' from registry (using cache)
This could be because the package does not exist, it was spelled incorrectly or you don't have permissions to it
** (Mix) No package with name plug_cowboy (from: mix.exs) in registry
Have not you registered to hex yet? Or should I not use it yet?
According to the Cowboy documentation, a metrics_callback function can be passed to the options when starting a new Endpoint as follows:
https://ninenines.eu/docs/en/cowboy/2.4/manual/cowboy_http2/
config :my_app, MyApp.Endpoint,
http: [port: 4000,
metrics_callback: &metrics_callback/1,
stream_handlers: [:cowboy_metrics_h, :cowboy_stream_h]]
but I get this error when I try to start it:
** (Mix) Could not start application free_license_server: MyApp.Application.start(:normal, []) returned an error: shutdown: failed to start child: MyApp.Endpoint
** (EXIT) shutdown: failed to start child: {:ranch_listener_sup, MyApp.Endpoint.HTTP}
** (EXIT) shutdown: failed to start child: :ranch_acceptors_sup
** (EXIT) :badarg
One fix was to to add :metrics_callback
to the list of protocol_options
@protocol_options [:compress, :stream_handlers, :metrics_callback]
in the Plug.Cowboy module.
I hope that this issue makes sense.
There was a recent change to remove the timeout
config option:
If this option is still specified, it's now causing the option to get passed to the underlying transport, where it causes a badarg
error when starting ranch...
I'm wondering if we should do more to protect against this... drop the option or raise an error?
10:20:40 18:20:40.815 [error] Child :ranch_acceptors_sup of Supervisor #PID<0.637.0> (:ranch_listener_sup) failed to start
10:20:40 ** (exit) :badarg
10:20:40 Start Call: :ranch_acceptors_sup.start_link(Main.HttpServer.HTTP, :ranch_tcp)
10:20:40
10:20:40 18:20:40.815 [error] Process #PID<0.639.0> terminating
10:20:40 ** (exit) :badarg
10:20:40 (kernel) inet_tcp.erl:142: :inet_tcp.listen/2
10:20:40 (ranch) /work/src/source.datanerd.us/after/nerd-graph/deps/ranch/src/ranch_acceptors_sup.erl:39: :ranch_acceptors_sup.init/1
10:20:40 (stdlib) supervisor.erl:295: :supervisor.init/1
10:20:40 (stdlib) gen_server.erl:374: :gen_server.init_it/2
10:20:40 (stdlib) gen_server.erl:342: :gen_server.init_it/6
10:20:40 (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
10:20:40 Initial Call: :ranch_acceptors_sup.init/1
10:20:40 Ancestors: [#PID<0.637.0>, Main.Supervisor, #PID<0.634.0>]
10:20:40
10:20:40 18:20:40.823 [error] Child {:ranch_listener_sup, Main.HttpServer.HTTP} of Supervisor Main.Supervisor failed to start
10:20:40 ** (exit) shutdown: failed to start child: :ranch_acceptors_sup
10:20:40 ** (EXIT) :badarg
10:20:40 Start Call: :ranch_listener_sup.start_link(Main.HttpServer.HTTP, :ranch_tcp, %{max_connections: 16384, num_acceptors: 100, socket_opts: [port: 3001, timeout: :infinity]}, :cowboy_clear, %{compress: true, env: %{dispatch: [{:_, [], [{:_, [], Plug.Cowboy.Handler, {Main.HttpServer, []}}]}]}, idle_timeout: 300000, max_header_value_length: 8192, stream_handlers: [:cowboy_compress_h, Plug.Cowboy.Stream]})
10:20:40
10:20:40 18:20:40.823 [error] Process #PID<0.633.0> terminating
10:20:40 ** (exit) exited in: Main.Application.start(:normal, [])
10:20:40 ** (EXIT) shutdown: failed to start child: {:ranch_listener_sup, Main.HttpServer.HTTP}
10:20:40 ** (EXIT) shutdown: failed to start child: :ranch_acceptors_sup
10:20:40 ** (EXIT) :badarg
10:20:40 (kernel) application_master.erl:138: :application_master.init/4
10:20:40 (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
10:20:40 Initial Call: :application_master.init/4
10:20:40 Ancestors: [#PID<0.632.0>]
10:20:40 ** (Mix) Could not start application main: Main.Application.start(:normal, []) returned an error: shutdown: failed to start child: {:ranch_listener_sup, Main.HttpServer.HTTP}
10:20:40 ** (EXIT) shutdown: failed to start child: :ranch_acceptors_sup
10:20:40 ** (EXIT) :badarg
I don't know if this library is the place to implement this, but shutdowns due to idle_timeout
need to be made more obvious. I just lost over an hour of my time understanding that my application was silently failing because of this.
This library depends on cowboy ~> 2.7
which is using the deprecated :erlang.get_stacktrace/0
. It happens that when an unexpected error raises, we could end up with this situation:
Ranch protocol #PID<0.10445.0> of listener Wharever.Endpoint.HTTP (connection #PID<0.10444.0>, stream id 1) terminated
an exception was raised:
** (UndefinedFunctionError) function :erlang.get_stacktrace/0 is undefined or private
:erlang.get_stacktrace()
(cowboy 2.7.0) /builds/whatever/deps/cowboy/src/cowboy_stream_h.erl:314: :cowboy_stream_h.request_process/3
(stdlib 3.17.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
We are in front of an outdated dependency that prevents us from using Phoenix with OTP 24, for example.
I think it's time to upgrade this library, so:
Let me know if it makes sense and I'll submit a PR with the changes.
Hi π,
children = [
Plug.Cowboy.child_spec(
scheme: :http,
plug: WSServer.Router,
options: [
dispatch: dispatch(),
port: port(),
transport_options: [
num_acceptors: 2,
max_connections: 16_384 # => default
],
protocol_options: [
idle_timeout: 1000 # => not getting applied
]
]
)
]
But is getting applied inside SocketHandler init
callback
def init(req, state) do
Logger.info("[SocketHandler] init")
dbg({req, state})
state = %{registry_key: req.path}
{:cowboy_websocket, req, state, %{idle_timeout: :infinity}}
end
Is there anything I'm missing ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.