dashbitco/broadway_dashboard

Keep track of your Broadway pipelines from Phoenix LiveDashboard

broadway_dashboard's Issues

docs and hexpm register to package

I think hexpm and documentation is broken.

[error] Handler {BroadwayDashboard.Telemetry, #PID<0.xxx.0>} has failed and has been detached.

I tried integrate broadway_dashboard to live LiveDashboard but got the error:

[error] Handler {BroadwayDashboard.Telemetry, #PID<0.xxx.0>} has failed and has been detached.
Reason={:badkey, :time, %{system_time: 16274584813510685}}
Stacktrace=[
  {BroadwayDashboard.Telemetry, :handle_event, 4,
   [file: 'lib/broadway_dashboard/telemetry.ex', line: 47]},
  {:telemetry, :"-execute/3-fun-0-", 4, ...

My deps

defp deps do
    [
      {:phoenix, github: "phoenixframework/phoenix", override: true},
      {:phoenix_html, github: "phoenixframework/phoenix_html", override: true},
      {:phoenix_live_reload, "~> 1.2", only: :dev},
      {:phoenix_live_view, github: "phoenixframework/phoenix_live_view", override: true},
      {:floki, ">= 0.30.0", only: :test},
      {:phoenix_live_dashboard, github: "phoenixframework/phoenix_live_dashboard", override: true},
      {:esbuild, "~> 0.1", runtime: Mix.env() == :dev},
      {:swoosh, "~> 1.3"},
      {:telemetry_metrics, "~> 0.4"},
      {:telemetry_poller, "~> 0.4"},
      {:jason, "~> 1.0"},
      {:plug_cowboy, "~> 2.0"},
      {:broadway, github: "dashbitco/broadway", override: true},
      {:broadway_rabbitmq, github: "dashbitco/broadway_rabbitmq", override: true},
      {:broadway_dashboard, github: "dashbitco/broadway_dashboard", override: true}
    ]
  end

Please help to clarify this issue.

Add the ability to control zoom in and out to the view of pipeline

When we have a large number of concurrency in batches or processors the view of the whole pipeline becomes very difficult.

I was wondering if we can add the ability to add some zoom control and navigation to this graph?
Something like this: https://codepen.io/osublake/pen/oGoyYb
Or this: https://www.petercollingridge.co.uk/tutorials/svg/interactive/pan-and-zoom/

I've looked to the source code of this project, but I'vent found where is the SVG generation or the place where I can implement this feature with JS.

Yes, I know that I can see the whole pipeline inspecting the element and setting the svg to use max-width as 100% and they will be more responsive. But I think this would be nice if we don't have to do this every time.

I believe this will improve the DX.

Add support for Phoenix Live Dashboard 0.7.0

Replaces #20 PR as changes needed require decisions around support of earlier versions of Elixir, Erlang, etc.

plan to support phoenix_live_dashboard v8 ?

Didn't support :via tuples as broadway names

The new version of Broadway is allowed :via tuples as broadway names.

But the code of Dashboard are not support for it current:

broadway_dashboard/lib/broadway_dashboard.ex

Line 56 in 5bddcee

case Enum.filter(pipelines, &is_atom/1) do

I hope this feature will be supported soon, Many thanks.

Q: How to change the topology or pause/resume the producer in realtime

Marlus presentation had a dashboard that could pause/start and change the graph topology in realtime.

How's it done? Broadway.stop followed by Broadway.start_link ?

Thanks!

Broadway pipelines tab fails to load

Current versions i am using:

broadway 1.0.3
broadway_dashboard 0.2.2
broadway_kafka 0.3.4
phoenix_live_dashboard 0.6.4
phoenix_live_view 0.17.7

router.ex

live_dashboard "/dashboard",
        metrics: TelemetrySupervisor,
        additional_pages: [
          broadway: BroadwayDashboard
        ]
    end

Issue:
Whenever i load up my /dashboard i am unable to access the /dashboard/broadway tab. It will load for a while and says the Dashboard is not connected.

Only meaningful log i can find

iex([email protected])1> 17:24:34.953 request_id=0558abf4-4a16-42a1-8232-e47e5a950a18 [info] GET /ingest/dashboard/broadway
iex([email protected])1> 17:24:34.970 request_id=0558abf4-4a16-42a1-8232-e47e5a950a18 [info] Sent 200 in 17ms
iex([email protected])1> 17:24:35.867 [info] CONNECTED TO Phoenix.LiveView.Socket in 95µs
  Transport: :websocket
  Serializer: Phoenix.Socket.V2.JSONSerializer
  Parameters: %{"_csrf_token" => "JXMuGARSIggZMRUFBjNjL3QrcCosUzYQHEK5wgQyKAahTj6BGJ6Oofcx", "vsn" => "2.0.0"}
iex([email protected])1> 17:24:35.961 [info] Function passed as a handler with ID {BroadwayDashboard.Telemetry, #PID<0.1495.0>} is local function.
This mean that it is either anonymous function or capture of function without module specified. That may cause performance penalty when calling such handler. For more details see note in `telemetry:attach/4` documentation.

"Throughput" and "All time" counters history

Hi

I have implemented a history module for the live dashboard, and it works for Phoenix and my custom metrics but does not work for BroadwayDashboard.

live_dashboard "/dashboard",
      metrics: MyMegaApp.Telemetry,
      additional_pages: [
        broadway: BroadwayDashboard
      ],
      metrics_history: {MyMegaApp.Telemetry.History, :metrics_history, []}

The MyMegaApp.Telemetry.History.metrics_history/1 contains a logger, so I can see that BroadwayDashboard does not call for history.

How can counters history be implemented in BroadwayDashboard?

Package versions:

%{
  "broadway": "1.0.0",
  "broadway_dashboard": "0.2.1",
  "broadway_rabbitmq": "0.7.0",
  "phoenix_live_dashboard": "0.5.3",
  "telemetry": "0.4.3",
  "telemetry_metrics": "0.6.0",
  "telemetry_poller": "0.5.0"
}