Unable to implement Kafka queuing with MVC application (Web appliaction)

[anshul-rastogi]

Unable to implement Kafka queuing with MVC application (Web appliaction)

[Jroland]

The consumer and producers both keep consistent connections to the kafka servers at all times. This means you will have to store them statically and then broker requests from the website through these static objects. You don't specify what you are trying to do with your MVC project, but there are some other REST based versions of the driver out there that you can reference.

[programmer-pragmatic]

Hi Jroland,

I am facing similar problem. If I try to copy code from program.cs (that calls SendMessageAsync on producer object) from console app to 'any web app (be it asp.net or mvc)', it does not work. How can i make it work?

Could you please elaborate the proposed solution 'storing producer/consumers in static objects' in web context.

Thanks In Advance
Balesh Kumar

[LoonaA]

Hi,
I had same problem using library in web-app but managed to get it done.
here is extract from link http://msdn.microsoft.com/en-us/magazine/jj991977.aspx

The root cause of this deadlock is due to the way await handles contexts. By default, when an incomplete Task is awaited, the current “context” is captured and used to resume the method when the Task completes. This “context” is the current SynchronizationContext unless it’s null, in which case it’s the current TaskScheduler. GUI and ASP.NET applications have a SynchronizationContext that permits only one chunk of code to run at a time. When the await completes, it attempts to execute the remainder of the async method within the captured context. But that context already has a thread in it, which is (synchronously) waiting for the async method to complete. They’re each waiting for the other, causing a deadlock.

Note that console applications don’t cause this deadlock. They have a thread pool SynchronizationContext instead of a one-chunk-at-a-time SynchronizationContext, so when the await completes, it schedules the remainder of the async method on a thread pool thread. The method is able to complete, which completes its returned task, and there’s no deadlock. This difference in behavior can be confusing when programmers write a test console program, observe the partially async code work as expected, and then move the same code into a GUI or ASP.NET application, where it deadlocks.

Basically in public async Task<List> SendMessageAsync(......) function
change the context inside this function using following call.

await Task.FromResult(1).ConfigureAwait(continueOnCapturedContext: false);
function run asynchronously, this get me working.

Thanks
Atul.

[Jroland]

What LoonaA wrote sounds great, and sounds like a solution to a specific async threading issue with a web project. However I want to reiterate my comment above. There is significant overhead in connecting to the kafka server and establishing a connection. You should not create a new producer and consumer for each web call. A static (singleton) producer and consumer should be created and then all calls from the web should be brokered through this object.

[Nagendra82]

Hi Jroland,

I am facing similar problem. If I try to copy code from program.cs how can i use in asp.net context.
Could you please elaborate the proposed solution 'storing producer/consumers in web context.

Thanks & Regards
Nagendra Singh