Template to deploy a Data Product for analytics and data science use-cases into a Data Landing Zone of the Data Management & Analytics Scenario (former Enterprise-Scale Analytics). The Data Product template can be used by cross-functional teams to create insights and products for external users.
License: MIT License
Describe the bug
If the user deploys this template, the CPU Cluster and GPU Cluster deployments fails, with the following error message:
{
"status": "Failed",
"error": {
"code": "UserError",
"message": "{\n \"error\": {\n \"code\": \"UserError\",\n \"severity\": null,\n \"message\": \"Provided EnableNodePublicIp 'False' requires workspace private endpoint.\",\n \"messageFormat\": null,\n \"messageParameters\": null,\n \"referenceCode\": null,\n \"detailsUri\": null,\n \"target\": null,\n \"details\": [],\n \"innerError\": null,\n \"debugInfo\": null,\n \"additionalInfo\": null\n },\n \"correlation\": {\n \"operation\": \"\",\n \"request\": \"\"\n },\n \"environment\": \"eastus2\",\n \"location\": \"eastus2\",\n \"time\": \"2021-08-02T12:17:49.9687945+00:00\",\n \"componentName\": \"machinelearningcompute\"\n}",
"details": []
}
}Steps to reproduce
Screenshots
Describe the solution you'd like
Automate the creation of managed private endpoints on the Data Factory managed vnet. This will allow private communication with Purview to collect Lineage Data.
There doesn't seem to be a way to submit a change in the 'about' section. See below changes.
Template to deploy a Data Product for analytics and data science use-cases into a Data Landing Zone of the Cloud-Scale Analytics Scenario, (former Enterprise-Scale Analytics, Data Management & Analytics Scenario). The Data Product template can be used by cross-functional teams to create insights and products for external users.
Describe the solution you'd like
Make SQL Pool deployment optional when choosing Synapse as deployment option.
Please change all instances of "Enterprise-scale analytics and ai" to "Data Management and Analytics Scenario" to align to marketing ask. This should happen for one-clicks and all documentation.
Describe the solution you'd like
Add new Azure Cognitive Service for Language to the list of supported Cognitive Services.
Also add potential support for this service: https://azure.microsoft.com/en-us/services/openai-service/#overview
Describe the bug
The bicep templates reference the wrong name for the container build compute. This needs to be fixed to be able to build images in the customer environment.
Steps to reproduce
Screenshots
No Screenshots
Describe the solution you'd like
In the prereqs we need more clarity on enterprise landing zone for data being optional to deploy the asset and also need clarity on the min number of assets that needs to be deployed for deploying AML. For instance we just need data factory to deploy along with AML and do not need synapse, cog services and so on..
Hi Marvin,
Please outline what are the features that are getting deployed in terms of security, network and baseline features for the workload esp AML.
Thanks
Sudha
Describe the solution you'd like
Marvin it will be great if we have a multi choice deployment for cog services given that some customers may use more than one.
Thanks Sudha
Describe the solution you'd like
Please add to the docs as well as the Portal experience information about the service endpoint requirements of Azure Machine Learning. Azure Machine Learning and the computes require the Storage Service Endpoint to be enabled on the subnet that is used.
Describe the bug
When deploying this solution with the processingService parameter set to "dataFactory", you receive a deployment error:
Deployment 'synapse001' could not be found.
Steps to reproduce
processingService to "dataFactory"Screenshots
(Pull request to fix this is forthcoming.)
These two env variables in the updateParameters.yml are not covered in the documentation:
ML_COMPUTE_INSTANCE_USER_OBJECT_ID: '{mlComputeInstanceObjectId}'
DATABRICKS_WORKSPACE_ID: '{databricksWorkspaceId}'
It would be nice to list them in the parameter update process and also the details on where we can find these two.
Deploy To Azure
(None) ResourceNotFound: Deployment failed due to timeout while waiting for Environment Image to become available. Check Environment Build Log in Workspace or Workspace storage for potential failures. Environment info: Name: CliV2AnonymousEnvironment, Version: f53727d1df8137354bc0f2c69d2619ee, Build log may be available at (log Please see troubleshooting guide, available here: https://aka.ms/oe-tsg#error-resourcenotfound Code: NoneNo response
There are open compliance tasks that need to be reviewed for your data-product-analytics repo.
To bring this repository to the standard required for 2021, we require administrators of this and all Microsoft GitHub repositories to complete a small set of tasks within the next 60 days. This is critical work to ensure the compliance and security of your Azure GitHub organization.
Please take a few minutes to complete the tasks at: https://repos.opensource.microsoft.com/orgs/Azure/repos/data-product-analytics/compliance
You can close this work item once you have completed the compliance tasks, or it will automatically close within a day of taking action.
If you no longer need this repository, it might be quickest to delete the repo, too.
More information about GitHub inside Microsoft and the new GitHub AE product can be found at https://aka.ms/gim or by contacting [email protected]
FYI: current admins at Microsoft include @marvinbuss, @daltondhcp, @esbran
Azure DevOps
Optional resources deployed causing error as the references to optional resources is present in the deployment template
resource not foundNo response
As part of the provisioning of the Azure ML workspace, we should set up diagnostic logging into a central log analytics workspace.
Azure DevOps
When deployed the new logging elements, for example Synapse pipeline errors did not trigger any alerts.
In the script, the 'static threshold' operator is "GreaterThan 1" when that likely should be "GreaterThanOrEqual 1".
AFAIK, this exists in other products' Alerts.bicep files as well.
No response
No response
Describe the bug
The AML Compute Cluster and Compute Instance deployments fail, when deploying to a region, where the 'no public IP' feature is not available. Hence, we have to change the configuration depending on the selected region of the deployment.
Steps to reproduce
Screenshots
New features need to be added to be in line with latest features:
Describe the solution you'd like
Required for private base images.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.