zoinertejada / azure-data-guide Goto Github PK
View Code? Open in Web Editor NEWAzure Data Architecture Guide
Azure Data Architecture Guide
For people whom are new to data analytics/processing it's sometimes hard to determine when a SQL database is good enough, if they need a data warehouse instead or need a mixture of both.
A scenario for this is where you have your API & backend processing using a database. Later on you need to provide reporting on a subset of that data. Do you run these on top of that database (pref. a replica for performance) or do you move that subset to a SQL data warehouse and run reports on top of that?
Interesting topics:
It might be interesting to provide a high-level overview of all services that are discussed and what purpose they accomplish, to a certain degree.
Example from this blog post:
Provide bread crumbs to articles so that you can see what this section is about.
Proposal:
Common Architectures > Relational > Transactional Data
Remarks:
This is probably done via the Azure documentation header but wanted to make sure that it's there as this can be interesting to see where you are.
Guidance on resource & cost optimizations.
The following should be mentioned at least:
Relates to #9
Would it be worth mentioning the scenarios on how a data pipeline can be triggered?
I'm thinking of the following:
I can contribute to this if you'd like.
In some scenarios there's a need to start storing potential interesting information, without knowing how it will be analyzed or processed further.
Maybe this is an interesting topic that we can tackle somewhere to provide guidance on.
Do we just dump it in Azure Data Lake Store or are there any other relevant locations?
Guidance on manageability of Azure services.
Not sure if that is the best word but mainly looking at
Does it make sense to extend "Options for Pipeline Orchestration, Control Flow, and Data Movement" now that Azure Data Factory is in preview and allows you to run SSIS in the cloud & on premises as part of a pipeline?
More information here.
Any plans on providing guidance on disaster recovery?
Guidance on design patterns such as sharding, caching, etc or having a link to the existing material at least and how to apply them.
Although this might already be covered.
Guidance on scalability in terms of auto scaling based on time or workload.
Scenarios:
Any plans on introducing Azure TIme Series Insights?
Any plans on adding Azure Logic Apps as an orchestrator for "Pipeline Orchestration, Control Flow and Data Movement"?
This works very well with other orchestrators such as Azure Data Factory where one can trigger the other.
I'm working on a blog post on this topic (aiming for beginning Dec), feel free to let me know if you're interested in this topic.
Any plans on adding guidance for data integration scenarios?
As an example - Every hour we need to fetch data with an external data source and process this.
Maybe this can be part of "Data Integration" section or the existing "Pipeline Orchestration, Control Flow and Data Movement" section.
Guidance on operational experience
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.