Depot contains various common sink implementations and publishes them as a library. This library will be used in firehose, daggers or any other application which wants to send data to destinations.
Create HttpClient and related classes to include all the HTTP auth and HTTP methods.
Implement a bare minimum HttpSink which sends data to the HTTP server.
Add support for parsing of list data type in getFieldByName() method in ParsedOdpfMessage.
getFieldByName() should return ParsedMessage instead of Object. ParsedMessage should contain more methods like getType(), isReapated(), isStruct(), isPrimitive() etc etc.
Add ToJson() method to convert struct into JSON string in ParsedOdpfMessage class.
Create HeaderBuilder class
Create URI builder and QueryParam class
Create Raw Payload builder
Create Json Payload builder
Create Message Payload Builder
Create String Payload builder
Create JsonTemplate Payload Builder
Create RequestBuilder build() method for Single request
Create RequestBuilder build() method for Batch request
Create HttpRecordConverter
Implement HttpRequestRecord and HttpResponseRecord classes to contain methods like logging based on configs.
Add methods to parse the HttpResponseRecord to set correct ErrorTypes.
There's issue on starting firehose when proto schema is not matching with BQ table schema. Sample error looks like this: Provided Schema does not match Table xxx. Field yyy is missing in new schema