Follow the steps given below:
- If you don't already have one, create a new 'Resource Group' under 'Resource groups'.
- Under 'Create a resource', click on 'Internet of Things' and then on 'Event Hubs'.
- It will first ask you to 'Create Namespace'. Use 'Standard' under 'Pricing Tier'. Use the same 'Location' as your 'Resource Group'.
- Click on 'Create' & wait till the Namespace is created.
- In the 'Dashboard' click on the newly created Namespace.
- On the top menu bar click on the 'plus' sign next to 'Event Hub'.
- Under 'Create Event Hub' enter 'Name'. Leave default values for rest, click on 'Create' & wait for event hub to be created.
Follow the steps given below:
- Click on this link: https://tinyurl.com/y7azvm2y
- Click on 'Save' at the bottom of the window & then on 'Run'.
- It takes about a minute to finish 'Running'. Wait till it finishes running.
- Click on 'Deploy Web Service'.
- Click on 'Test'. Enter 'I love my life' & click on 'Check mark'. It will show the result in the bottom.
- Under 'REQUEST/RESPONSE' click on 'Excel 2010 or earlier workbook' to save the workbook locally. You will need these values to create a 'Function' in Stream Analytics Job.
Follow the steps given below:
- If you don't already have one, create a new 'Cosmos DB Account'. Under 'Create a resource', click on 'Cosmos DB'. Enter unique 'Account Name'. Use 'SQL' under 'API'. Use same 'Location' as your 'Resource Group'.
- Under 'Create a resource', click on 'Analytics' & then on 'Stream Analytics Job'.
- Create a new job under same 'Resource Group' & 'Location' & wait for it to get created.
- In the 'Dashboard' click on the newly created 'Stream Analytics Job'.
- Click on 'Inputs' under 'Job topology' & then on 'Add stream input'.
- Select 'Event Hub' in the drop-down box.
- Give a name to the Input under 'Input alias'.
- It will pre-populate rest of the values. Confirm them & click on 'Save'.
- It will test connection & create the input.
- Click on 'Outputs'under 'Job Topology' & then on 'Add'.
- In the drop-down box select 'Cosmos DB'.
- Give a name to the Output under 'Output alias'.
- Select your 'Account id' if it's not selected. Under 'Database' select 'Create new' & enter name of database.
- Enter value under 'Collection name pattern'
- IMPORTANT: Under 'Document id', enter 'id'. This is 'name of the field in output events used to specify document id'. In our case, every tweet has a field called 'id' which is a unique identifier.
- Click on 'Save' to create the output.
- Click on 'Functions' & then on 'Add'.
- In the drop-down, select 'Azure ML'.
- Enter 'sentiment' as 'Function alias'. Select 'Provide Azure ML manually'.
- From the workbook saved in step #6 in the previous step, copy 'Web Service URL' value and enter it here under 'URL'.
- From the same workbook, copy 'Access Key' value and enter it under 'Key'.
- Click on 'Save'.
- Click on 'Query' and replace the default query with:
WITH sentiment AS ( SELECT id, "user".screenName, text, sentiment(text) as result, retweetedStatus.retweetCount FROM <Your Input Alias> ) SELECT id, screenName, text, result.[Sentiment], result.[Score], retweetCount INTO <Your Output Alias> FROM sentiment
- Click on 'Save' to save the query.
Follow the steps given below:
- Create an App at: https://apps.twitter.com/
- Twitter will then provide you values for the following 4 properties which you can use to get a 'Sample' of Tweets in real time:
authConsumerKey
authConsumerSecret
authAccessToken
authAccessTokenSecret
- Clone this directory on your machine:
git clone https://github.com/ajaychitre/azure-tools.git
- Open [application.properties](./azure-tools/src/main/resources/application.properties & set the following properties:
authConsumerKey=<from step 2 above>
authConsumerSecret=<from step 2 above>
authAccessToken=<from step 2 above>
authAccessTokenSecret=<from step 2 above>
namespace=<name of namespace created above>
eventHubName=<name of event hub>
sasKeyName=RootManageSharedAccessKey
sasKey=<primary key of the Namespace under Shared Access Policy>
- Compile & Run Restserver
cd azure-tools
mvn clean package
java -jar ./target/azure-tools-1.0-SNAPSHOT.jar
This will start sending tweets to the Event Hub. Keep this job running.
Follow the steps given below:
- On the 'Dashboard' click on the newly created Stream Analytics Job.
- Under 'Overview', click on 'Start'. Leave the 'Job output start time' to 'Now' & click on 'Start' button at the bottom.
- It takes about a minute to start the job.
Follow the steps given below:
- Wait till the Stream Analytics Job is in 'Running' state.
- Click on 'Azure Cosmos DB' in left menu bar.
- Click on the 'Account' you created above.
- Click on the 'Collection' created as 'Output' in your Stream Processing Job.
- Click on 'New SQL Query' tab. You can now enter queries such as this:
SELECT * FROM c where c.sentiment = 'negative' and c.retweetcount > 1000
Follow the steps given below:
- IMPORTANT: Remember that you get charged by compute resources used so stop & release the resources that you no longer need.
- Stop the job that sends tweets to the Event Hub.
- Stop the Stream Analytics Job.
- If you're simply experimenting, you can 'Delete' the resource group which will delete all resources under it.