Comments (4)
How should I write the data back to eventhubs? When I read it in from the input eventhubs it's in binary format, so should i write it back to the output eventhubs in binary format as well?
Tagging @jcocchi or please let me know who else I should tag too!
from streaming-at-scale.
Data in EventHub is in binary format, but you should sent to it just in plain text (if you're sending JSON). It will stored in the binary format automatically.
from streaming-at-scale.
You need to put the body in a column called 'body'
To generate JSON from a struct:
https://spark.apache.org/docs/latest/api/java/org/apache/spark/sql/functions.html#to_json-org.apache.spark.sql.Column-
from streaming-at-scale.
I have an example of reading from Event Hub and then writing back to an Event Hub using Structured Streaming here: https://github.com/mpfishe2/az-databricks-realtime-alert-system/blob/master/Real-Time%20Alerting.ipynb. Its a simple example.
from streaming-at-scale.
Related Issues (20)
- akskafka-databricks-cosmosdb fails with helm3
- eventhubskafka-flink-eventhubskafka fails with missing Zookeeper incubator chart
- hdinsightkafka-flink-hdinsightkafka fails to create cluster
- akskafka-databricks-cosmosdb fails to deploy Kafka
- [QUERY] Not able to read AVRO file (following the tutorial for event-hubs + apache drill) HOT 1
- Eventhuub-Function-SQL Deployment issues in functions HOT 1
- Databricks provisioning fails when creating the solution HOT 2
- ERROR: Service principal 'http://bie3i8adx-reader' doesn't exist HOT 3
- Curious why no Python samples here? HOT 3
- More info on sample
- CQRS Link Returns 404
- Duplicate data is being generated HOT 2
- Document deduplication strategies
- Add deduplication in eventhubs-databricks-delta
- Adding scenarios that each sample fits best
- Evaluate use of blobs as storage HOT 1
- Misspelling of the word function HOT 2
- Comparison of cost vs performance across all the options
- Add CosmosDB via MongoDB API sample
- Deployment of eventhubs-streamanalytics-azuresql fails HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from streaming-at-scale.