Giter Site home page Giter Site logo

plagueho / cosmosdb Goto Github PK

View Code? Open in Web Editor NEW
147.0 8.0 47.0 2.06 MB

PowerShell Module for working with Azure Cosmos DB databases, collections, documents, attachments, offers, users, permissions, triggers, stored procedures and user defined functions.

Home Page: http://dscottraynsford.com

License: MIT License

PowerShell 99.56% C# 0.44%
powershell azure cosmosdb restapi powershell-module documentdb

cosmosdb's Introduction

License Documentation PowerShell Gallery PowerShell Gallery Minimum Supported Windows PowerShell Version Minimum Supported PowerShell Core Version Minimum Supported PowerShell Version

CosmosDB PowerShell Module

Module Build Status

Branch Azure Pipelines Automated Tests Code Quality
main ap-image-main ts-image-main cq-image-main

Table of Contents

Introduction

This PowerShell module provides cmdlets for working with Azure Cosmos DB.

The CosmosDB PowerShell module enables management of:

The module uses the Cosmos DB (DocumentDB) Rest APIs.

For more information on the Cosmos DB Rest APIs, see this link.

Requirements

This module requires the following:

  • Windows PowerShell 5.x or PowerShell 6.x:
    • Az.Profile and Az.Resources PowerShell modules are required if using New-CosmosDbContext -ResourceGroupName $resourceGroup or *-CosmosDbAccount functions.

Note: As of 3.0.0.0 of the CosmosDB module, support for AzureRm and AzureRm.NetCore PowerShell modules has been deprecated due to being superceeded by the Az modules. If it is a requirement that AzureRm or AzureRm.NetCore modules are used then you will need to remain on CosmosDB module 2.x.

Recommended Knowledge

It is recommended that before using this module it is important to understand the fundamental concepts of Cosmos DB. This will ensure you have an optimal experience by adopting design patterns that align to Cosmos DB best practice.

Users new to Cosmos DB should familiarize themselves with the following concepts:

It is also recommended to watch this Ignite video on data modelling and partitioning in Cosmos DB.

Installation

To install the module from PowerShell Gallery, use the PowerShell Cmdlet:

Install-Module -Name CosmosDB

Getting Started

The easiest way to use this module is to first create a context object using the New-CosmosDbContext cmdlet which you can then use to pass to the other Cosmos DB cmdlets in the module.

To create the context object you will either need access to the primary primary or secondary keys from your Cosmos DB account or allow the CosmosDB Powershell module to retrieve the keys directly from the Azure management portal for you.

Working with Contexts

Create a Context specifying the Key Manually

First convert your key into a secure string:

$primaryKey = ConvertTo-SecureString -String 'GFJqJesi2Rq910E0G7P4WoZkzowzbj23Sm9DUWFX0l0P8o16mYyuaZBN00Nbtj9F1QQnumzZKSGZwknXGERrlA==' -AsPlainText -Force

Use the key secure string, Azure Cosmos DB account name and database to create a context variable:

$cosmosDbContext = New-CosmosDbContext -Account MyAzureCosmosDB -Database MyDatabase -Key $primaryKey

Create a Context for a Cosmos DB in Azure US Government Cloud

Use the key secure string, Azure Cosmos DB account name and database to create a context variable and set the Environment parameter to AzureUSGovernment:

$cosmosDbContext = New-CosmosDbContext -Account MyAzureCosmosDB -Database MyDatabase -Key $primaryKey -Environment AzureUSGovernment

Create a Context for a Cosmos DB in Azure China Cloud (Mooncake)

Use the key secure string, Azure Cosmos DB account name and database to create a context variable and set the Environment parameter to AzureChinaCloud:

$cosmosDbContext = New-CosmosDbContext -Account MyAzureCosmosDB -Database MyDatabase -Key $primaryKey -Environment AzureChinaCloud

Create a Context for a Cosmos DB with a Custom Endpoint

Use the key secure string, Azure Cosmos DB account name, database and Cosmos DB custom endpoint hostname:

$cosmosDbContext = New-CosmosDbContext -Account MyAzureCosmosDB -Database MyDatabase -Key $primaryKey -EndpointHostname documents.eassov.com

Use CosmosDB Module to Retrieve Key from Azure Management Portal

To create a context object so that the CosmosDB PowerShell module retrieves the primary or secondary key from the Azure Management Portal, use the following command:

$cosmosDbContext = New-CosmosDbContext -Account MyAzureCosmosDB -Database MyDatabase -ResourceGroupName MyCosmosDbResourceGroup -MasterKeyType SecondaryMasterKey

Note: if PowerShell is not connected to Azure then an interactive Azure login will be initiated. If PowerShell is already connected to an account that doesn't contain the Cosmos DB you wish to connect to then you will first need to connect to the correct account using the Connect-AzAccount cmdlet.

Create a Context for a Cosmos DB Emulator

Microsoft provides a Cosmos DB emulator that you can run locally to enable testing and debugging scenarios. To create a context for a Cosmos DB emulator installed on the localhost use the following command:

$cosmosDbContext = New-CosmosDbContext -Emulator -Database MyDatabase

You can also provide a custom URI if the emulator is hosted on another machine or an alternate port as well as specifying an alternate Key to use:

$primaryKey = ConvertTo-SecureString -String 'GFJqJesi2Rq910E0G7P4WoZkzowzbj23Sm9DUWFX0l0P8o16mYyuaZBN00Nbtj9F1QQnumzZKSGZwknXGERrlA==' -AsPlainText -Force
$cosmosDbContext = New-CosmosDbContext -Emulator -Database MyDatabase -Uri https://cosmosdbemulator.contoso.com:9081 -Key $primaryKey

Create a Context from Resource Authorization Tokens

See the section Using Resource Authorization Tokens for instructions on how to create a Context object containing one or more Resource Authorization Tokens.

Working with Accounts

You can create, retrieve, update and remove Azure Cosmos DB accounts using this module. To use these features you will need to ensure the Az.Profile and Az.Resources modules installed - See Requirements above.

Note: You must have first logged PowerShell into Azure using the Connect-AzAccount function before you can use these functions.

Create a new Cosmos DB account in Azure:

New-CosmosDbAccount -Name MyAzureCosmosDB -ResourceGroupName MyCosmosDbResourceGroup -Location WestUS

Get the properties of an existing Cosmos DB account in Azure:

Get-CosmosDbAccount -Name MyAzureCosmosDB -ResourceGroupName MyCosmosDbResourceGroup

Get a Secure String containing the Primary Master Key for an account in Azure:

$key = Get-CosmosDbAccountMasterKey -Name MyAzureCosmosDB -ResourceGroupName MyCosmosDbResourceGroup

Get a Secure String containing the Secondary Readonly Master Key for an account in Azure:

$key = Get-CosmosDbAccountMasterKey -Name MyAzureCosmosDB -ResourceGroupName MyCosmosDbResourceGroup -MasterKeyType SecondaryReadonlyMasterKey

Regenerate the Primary Readonly Master Key for an account in Azure:

New-CosmosDbAccountMasterKey -Name MyAzureCosmosDB -ResourceGroupName MyCosmosDbResourceGroup -MasterKeyType PrimaryReadonlyMasterKey

Get the connection strings used to connect to an existing Cosmos DB account in Azure:

Note: This function is not currently working due to an issue in the Microsoft/DocumentDB Provider. See this issue for more information.

Get-CosmosDbAccountConnectionString -Name MyAzureCosmosDB -ResourceGroupName MyCosmosDbResourceGroup

Update an existing Cosmos DB account in Azure:

Set-CosmosDbAccount -Name MyAzureCosmosDB -ResourceGroupName MyCosmosDbResourceGroup -Location WestUS -DefaultConsistencyLevel Strong

Delete an existing Cosmos DB account in Azure:

Remove-CosmosDbAccount -Name MyAzureCosmosDB -ResourceGroupName MyCosmosDbResourceGroup

Working with Databases

Create a new database in the Cosmos DB account with database throughput provisioned at 1200 RU/s:

New-CosmosDbDatabase -Context $cosmosDbContext -Id MyDatabase -OfferThroughput 1200

Create a new database in the Cosmos DB account with autoscaling throughput with a maximum of 40,000 RU/s down to a minimum of 4,000 RU/s:

New-CosmosDbDatabase -Context $cosmosDbContext -Id MyDatabase -AutoscaleThroughput 40000

Create a new database in the Cosmos DB account that will have throughput provisioned at the collection rather than the database:

New-CosmosDbDatabase -Context $cosmosDbContext -Id DatabaseWithCollectionThroughput

Get a list of databases in the Cosmos DB account:

Get-CosmosDbDatabase -Context $cosmosDbContext

Get the specified database from the Cosmos DB account:

Get-CosmosDbDatabase -Context $cosmosDbContext -Id MyDatabase

Working with Offers

Get a list of offers in the Cosmos DB account:

Get-CosmosDbOffer -Context $cosmosDbContext

Query the offers in the Cosmos DB account:

Get-CosmosDbOffer -Context $cosmosDbContext -Query 'SELECT * FROM root WHERE (root["id"] = "lyiu")'

Update an existing V2 offer to set a different throughput:

Get-CosmosDbOffer -Context $cosmosDbContext -Id lyiu |
    Set-CosmosDbOffer -Context $cosmosDbContext -OfferThroughput 1000 -OfferIsRUPerMinuteThroughputEnabled $true

Update all existing V2 offers to set a different throughput:

Get-CosmosDbOffer -Context $cosmosDbContext -Query 'SELECT * FROM root WHERE (root["offerVersion"] = "V2")' |
    Set-CosmosDbOffer -Context $cosmosDbContext -OfferThroughput 10000 -OfferIsRUPerMinuteThroughputEnabled $false

Working with Collections

Get a list of collections in a database:

Get-CosmosDbCollection -Context $cosmosDbContext

Create a collection in the database with the partition key 'id' and the offer throughput of 50,000 RU/s:

New-CosmosDbCollection -Context $cosmosDbContext -Id MyNewCollection -PartitionKey id -OfferThroughput 50000

Create a collection in the database with the partition key 'id' using autoscaling with the maximum throughput of 40,000 RU/s and a mimimum of 4,000 RU/s:

New-CosmosDbCollection -Context $cosmosDbContext -Id MyNewCollection -PartitionKey id -AutoscaleThroughput 40000

Get a specified collection from a database:

Get-CosmosDbCollection -Context $cosmosDbContext -Id MyNewCollection

Get the first 5 collections from a database with a continuation token to allow retrieval of further collections:

$ResponseHeader = $null
$collections = Get-CosmosDbCollection -Context $cosmosDbContext -MaxItemCount 5 -ResponseHeader ([ref] $ResponseHeader)
$continuationToken = Get-CosmosDbContinuationToken -ResponseHeader $ResponseHeader

Get the next 5 collections from a database using a continuation token:

$collections = Get-CosmosDbCollection -Context $cosmosDbContext -MaxItemCount 5 -ContinuationToken $continuationToken

Delete a collection from the database:

Remove-CosmosDbCollection -Context $cosmosDbContext -Id MyNewCollection

Creating a Collection with a custom Indexing Policy

You can create a collection with a custom indexing policy by assembling an Indexing Policy object using the functions:

  • New-CosmosDbCollectionCompositeIndexElement
  • New-CosmosDbCollectionIncludedPathIndex
  • New-CosmosDbCollectionIncludedPath
  • New-CosmosDbCollectionExcludedPath
  • New-CosmosDbCollectionIndexingPolicy

For example, to create a string range, a number range index and a point spatial index on the '/*' path using consistent indexing mode with no excluded paths:

$indexStringRange = New-CosmosDbCollectionIncludedPathIndex -Kind Range -DataType String
$indexNumberRange = New-CosmosDbCollectionIncludedPathIndex -Kind Range -DataType Number
$indexPointSpatial = New-CosmosDbCollectionIncludedPathIndex -Kind Spatial -DataType Point
$indexIncludedPath = New-CosmosDbCollectionIncludedPath -Path '/*' -Index $indexStringRange, $indexNumberRange, $indexPointSpatial
$indexingPolicy = New-CosmosDbCollectionIndexingPolicy -Automatic $true -IndexingMode Consistent -IncludedPath $indexIncludedPath
New-CosmosDbCollection -Context $cosmosDbContext -Id MyNewCollection -PartitionKey id -IndexingPolicy $indexingPolicy

Important Index Notes

The Hash index Kind is no longer supported by Cosmos DB. A warning will be displayed if the Hash index Kind is used. The Hash index Kind will be removed in a future BREAKING release of the Cosmos DB module. See this page for more information.

The Precision parameter is no longer supported by Cosmos DB and will be ignored. The maximum precision of -1 will always be used for Range indexes. A warning will be displayed if the Precision parameter is passed. The Precision parameter will be removed in a future BREAKING release of the Cosmos DB module. See this page for more information.

It is recommended to remove the use of the Hash index Kind and any instances of the Precision parameter and any automation or scripts to avoid being affected by future BREAKING CHANGES.

For more information on how Cosmos DB indexes documents, see this page.

Creating a Collection with a custom Indexing Policy including Composite Indexes

To create a custom indexing policy that automatically indexes all paths but also includes two composite indexes, each consisting of two paths:

$compositeIndex = @(
    @(
        (New-CosmosDbCollectionCompositeIndexElement -Path /name -Order Ascending ),
        (New-CosmosDbCollectionCompositeIndexElement -Path /age -Order Ascending )
    ),
    @(
        (New-CosmosDbCollectionCompositeIndexElement -Path /name -Order Ascending ),
        (New-CosmosDbCollectionCompositeIndexElement -Path /age -Order Descending )
    )
)
$indexIncludedPath = New-CosmosDbCollectionIncludedPath -Path '/*'
$indexingPolicy = New-CosmosDbCollectionIndexingPolicy -Automatic $true -IndexingMode Consistent -IncludedPath $indexIncludedPath -CompositeIndex $compositeIndex
New-CosmosDbCollection -Context $cosmosDbContext -Id MyNewCollection -PartitionKey id -IndexingPolicy $indexingPolicy

Update an existing Collection with a new Indexing Policy

You can update an existing collection with a custom indexing policy by assembling an Indexing Policy using the method in the previous section and then applying it using the Set-CosmosDbCollection function:

$indexStringRange = New-CosmosDbCollectionIncludedPathIndex -Kind Range -DataType String
$indexIncludedPath = New-CosmosDbCollectionIncludedPath -Path '/*' -Index $indexStringRange
$indexingPolicy = New-CosmosDbCollectionIndexingPolicy -Automatic $true -IndexingMode Consistent -IncludedPath $indexIncludedPath
Set-CosmosDbCollection -Context $cosmosDbContext -Id MyExistingCollection -IndexingPolicy $indexingPolicy

After updating a collection with an indexing policy it will take some time to transform the index. To retrieve the progress of the index transformation, call the Get-CosmosDbCollection function and then evaluate the value of the

PS C:\> $ResponseHeader = $null
PS C:\> $collections = Get-CosmosDbCollection -Context $cosmosDbContext -Id MyExistingCollection -ResponseHeader ([ref] $ResponseHeader)
PS C:\> $indexUpdateProgress = Get-CosmosDbResponseHeaderAttribute -ResponseHeader $ResponseHeader -HeaderName x-ms-documentdb-collection-index-transformation-progress

Creating a Collection with a custom Indexing Policy using JSON

If the New-CosmosDbCollection* functions don't enable you to build the index policy to your requirements, you can also pass the raw index policy JSON to the function using the IndexingPolicyJson parameter:

$indexingPolicyJson = @'
{
    "automatic":true,
    "indexingMode":"Consistent",
    "includedPaths":[
        {
            "path":"/*"
        }
    ],
    "excludedPaths":[],
    "compositeIndexes":[
        [
            {
                "path":"/name",
                "order":"ascending"
            },
            {
                "path":"/age",
                "order":"descending"
            }
        ]
    ]
}
'@
New-CosmosDbCollection -Context $cosmosDbContext -Id MyNewCollection -PartitionKey id -IndexingPolicyJson $indexingPolicyJson

Creating a Collection with a custom Unique Key Policy

You can create a collection with a custom unique key policy by assembling a Unique Key Policy object using the functions:

  • New-CosmosDbCollectionUniqueKey
  • New-CosmosDbCollectionUniqueKeyPolicy

For example, to create a unique key policy that contains two unique keys, with the first unique key combining '/name' and '/address' and the second unique key is set to '/email'.

$uniqueKeyNameAddress = New-CosmosDbCollectionUniqueKey -Path /name, /address
$uniqueKeyEmail = New-CosmosDbCollectionUniqueKey -Path /email
$uniqueKeyPolicy = New-CosmosDbCollectionUniqueKeyPolicy -UniqueKey $uniqueKeyNameAddress, $uniqueKeyEmail
New-CosmosDbCollection -Context $cosmosDbContext -Id MyNewCollection -PartitionKey id -UniqueKeyPolicy $uniqueKeyPolicy

For more information on how Cosmos DB indexes documents, see this page.

Update an existing Collection with a new Unique Key Policy

You can update an existing collection with a custom unique key policy by assembling a Unique Key Policy using the method in the previous section and then applying it using the Set-CosmosDbCollection function:

$uniqueKeyNameAddress = New-CosmosDbCollectionUniqueKey -Path /name, /address
$uniqueKeyEmail = New-CosmosDbCollectionUniqueKey -Path /email
$uniqueKeyPolicy = New-CosmosDbCollectionUniqueKeyPolicy -UniqueKey $uniqueKeyNameAddress, $uniqueKeyEmail
Set-CosmosDbCollection -Context $cosmosDbContext -Id MyExistingCollection -IndexingPolicy $indexingPolicy

Creating a Collection without a Partition Key

Warning: It is not recommended to create a collection without a partition key. It may result in reduced performance and increased cost. This functionality is included for backwards compatibility only.

It is only possible to create non-partitioned collection in a database that has not got provisioned throughput at the database level enabled.

Create a collection in the database with the offer throughput of 2500 RU/s and without a partition key:

New-CosmosDbCollection -Context $cosmosDbContext -Id NonPartitionedCollection -OfferThroughput 2500

Working with Documents

Create 10 new documents in a collection in the database using the id as the partition key:

0..9 | Foreach-Object {
    $id = $([Guid]::NewGuid().ToString())
    $document = @{
        id      = $id
        content = "Some string"
        more    = "Some other string"
    } | ConvertTo-Json
    New-CosmosDbDocument -Context $cosmosDbContext -CollectionId MyNewCollection -DocumentBody $document -PartitionKey $id
}

Create a new document containing non-ASCII characters in a collection in the database using the id as the partition key:

$id = $([Guid]::NewGuid().ToString())
$document = @{
    id      = $id
    content = "ๆ‰ๆœฌ ๅธ"
} | ConvertTo-Json
New-CosmosDbDocument -Context $cosmosDbContext -CollectionId MyNewCollection -DocumentBody $document -Encoding 'UTF-8' -PartitionKey $id

Return a document with a specific Id from a collection in the database using the document ID as the partition key:

Get-CosmosDbDocument -Context $cosmosDbContext -CollectionId MyNewCollection -Id $documents[0].id -PartitionKey $documents[0].id

Note: Because this is a partitioned collection, if you don't specify a partition key you will receive a (400) Bad Request exception.

Get the first 5 documents from the collection in the database:

$ResponseHeader = $null
$documents = Get-CosmosDbDocument -Context $cosmosDbContext -CollectionId MyNewCollection -MaxItemCount 5 -ResponseHeader ([ref] $ResponseHeader)
$continuationToken = Get-CosmosDbContinuationToken -ResponseHeader $ResponseHeader

Note: You don't need to specify the partition key here because you are just getting the first 5 documents in whatever order they are available so going to a specific partition is not required.

Get the next 5 documents from a collection in the database using the continuation token found in the headers from the previous request:

$documents = Get-CosmosDbDocument -Context $cosmosDbContext -CollectionId MyNewCollection -MaxItemCount 5 -ContinuationToken $continuationToken

Replace the content of a document in a collection in the database:

$newDocument = @{
    id      = $documents[0].id
    content = "New string"
    more    = "Another new string"
} | ConvertTo-Json
Set-CosmosDbDocument -Context $cosmosDbContext -CollectionId MyNewCollection -Id $documents[0].id -DocumentBody $newDocument -PartitionKey $documents[0].id

Querying a collection in a database:

$query = "SELECT * FROM customers c WHERE (c.id = '[email protected]')"
Get-CosmosDbDocument -Context $cosmosDbContext -CollectionId MyNewCollection -Query $query

Querying a collection in a database using a parameterized query:

$query = "SELECT * FROM customers c WHERE (c.id = @id)"
$queryParameters = @(
    @{
        name  = "@id"
        value = "[email protected]"
    }
)
Get-CosmosDbDocument -Context $cosmosDbContext -CollectionId MyNewCollection -Query $query -QueryParameters $queryParameters

Delete a document from a collection in the database:

Remove-CosmosDbDocument -Context $cosmosDbContext -CollectionId MyNewCollection -Id $documents[0].id -PartitionKey $documents[0].id

Note: Because this is a partitioned collection, if you don't specify a partition key you will receive a (400) Bad Request exception.

Using a While Loop to get all Documents in a Collection

The Cosmos DB REST APIs have a maximum response size of 4MB. Therefore, to get a set of documents from a collection that will be larger than 4MB the request will need to be broken down into blocks using continuation tokens.

The following is an example of how that could be implemented.

$documentsPerRequest = 20
$continuationToken = $null
$documents = $null

do {
    $responseHeader = $null
    $getCosmosDbDocumentParameters = @{
        Context = $cosmosDbContext
        CollectionId = 'MyNewCollection'
        MaxItemCount = $documentsPerRequest
        ResponseHeader = ([ref] $responseHeader)
    }

    if ($continuationToken) {
        $getCosmosDbDocumentParameters.ContinuationToken = $continuationToken
    }

    $documents += Get-CosmosDbDocument @getCosmosDbDocumentParameters
    $continuationToken = Get-CosmosDbContinuationToken -ResponseHeader $responseHeader
} while (-not [System.String]::IsNullOrEmpty($continuationToken))

Working with Documents in a non-partitioned Collection

Warning: It is not recommended to use a collection without a partition key. It may result in reduced performance and increased cost. This functionality is included for backwards compatibility only.

Creating a document in a collection that has a Partition Key requires the PartitionKey parameter to be specified for the document:

$document = @{
    id      = "en-us"
    locale  = "English (US)"
} | ConvertTo-Json
New-CosmosDbDocument -Context $cosmosDbContext -CollectionId NonPartitionedCollection -DocumentBody $document

Get a document from a partitioned collection with a specific Id:

Get-CosmosDbDocument -Context $cosmosDbContext -CollectionId NonPartitionedCollection -Id 'en-us'

Delete a document from a partitioned collection in the database:

Remove-CosmosDbDocument -Context $cosmosDbContext -CollectionId NonPartitionedCollection -Id 'en-us'

Working with Attachments

Create an attachment on a document in a collection:

New-CosmosDbAttachment -Context $cosmosDbContext -CollectionId MyNewCollection -DocumentId $documents[0].id -PartitionKey $documents[0].id -Id image_1 -ContentType 'image/jpg' -Media www.bing.com

Get all attachments for a document in a collection:

Get-CosmosDbAttachment -Context $cosmosDbContext -CollectionId MyNewCollection -DocumentId $documents[0].id -PartitionKey $documents[0].id

Get an attachment by Id for a document in a collection:

Get-CosmosDbAttachment -Context $cosmosDbContext -CollectionId MyNewCollection -DocumentId $documents[0].id -PartitionKey $documents[0].id -Id image_1

Rename an attachment for a document in a collection:

Set-CosmosDbAttachment -Context $cosmosDbContext -CollectionId MyNewCollection -DocumentId $documents[0].id -PartitionKey $documents[0].id -Id image_1 -NewId image_2

Delete an attachment from a document in collection:

Remove-CosmosDbAttachment -Context $cosmosDbContext -CollectionId MyNewCollection -DocumentId $documents[0].id -PartitionKey $documents[0].id -Id image_2

Working with Users

Get a list of users in the database:

Get-CosmosDbUser -Context $cosmosDbContext

Create a user in the database:

New-CosmosDbUser -Context $cosmosDbContext -Id dscottraynsford@contoso.com

Delete a user from the database:

Remove-CosmosDbUser -Context $cosmosDbContext -Id dscottraynsford@contoso.com

Working with Permissions

Get a list of permissions for a user in the database:

Get-CosmosDbPermission -Context $cosmosDbContext -UserId dscottraynsford@contoso.com

Create a permission for a user in the database with read access to a collection:

$collectionId = Get-CosmosDbCollectionResourcePath -Database MyDatabase -Id MyNewCollection
New-CosmosDbPermission -Context $cosmosDbContext -UserId dscottraynsford@contoso.com -Id r_mynewcollection -Resource $collectionId -PermissionMode Read

Remove a permission for a user from the database:

Remove-CosmosDbPermission -Context $cosmosDbContext -UserId dscottraynsford@contoso.com -Id r_mynewcollection

Using Resource Authorization Tokens

Cosmos DB supports using resource authorization tokens to grant access to individual resources (eg. documents, collections, triggers) to a specific user. A user in this context can also be used to represent an application that needs access to specific data. This can be used to reduce the need to provide access to master keys to end users.

To use a resource authorization token, first a permission must be assigned to the user for the resource using the New-CosmosDbPermission. A user can be created using the New-CosmosDbUser function.

Note: By default, Resource Authorization Tokens expire after an hour. This can be extended to a maximum of 5 hours or reduced to minimum of 10 minutes. Use the TokenExpiry parameter to control the length of time that the resource authorization tokens will be valid for.

The typical pattern for using resource authorization tokens is to have a token broker app that provides some form of user authentication and then returns the resource authorization tokens assigned to that user. This removes the requirement for the user to be given access to the master key for the Cosmos DB database.

For more information on using resource authorization tokens or the token broker app pattern, please see this document.

The following is an example showing how to create a resource context object that contains a resource authorization token granting access to read the collection MyNewCollection. It is assumed that the permission for the user [email protected] has been created as per the previous section. The resource context object is then used to retrieve the MyNewCollection.

The resource authorization token is stored in the context object with an expiration date/time matching what was returned in the permission so that the validity of a token can be validated and reported on without making a request to the Cosmos DB server.

$collectionId = Get-CosmosDbCollectionResourcePath -Database MyDatabase -Id MyNewCollection
$permission = Get-CosmosDbPermission -Context $cosmosDbContext -UserId dscottraynsford@contoso.com -Id r_mynewcollection -Resource $collectionId -TokenExpiry 7200
# Future features planned to make creation of a resource context token from a permission easier
$tokenParams = @{
    Resource    = $collectionId
    TimeStamp   = $permission[0].Timestamp
    TokenExpiry = 7200
    Token       = (ConvertTo-SecureString -String $permission[0].Token -AsPlainText -Force)
}
$contextToken = New-CosmosDbContextToken @tokenParams
$resourceParams = @{
    Account     = $cosmosDBContext.Account
    Database    = MyDatabase
    Token       = $contextToken
}
$resourceContext = New-CosmosDbContext @resourceParams
Get-CosmosDbCollection -Context $resourceContext -Id MyNewCollection

Working with Triggers

Get a list of triggers for a collection in the database:

Get-CosmosDbTrigger -Context $cosmosDbContext -CollectionId MyNewCollection

Create a trigger for a collection in the database that executes after all operations:

$body = @'
function updateMetadata() {
    var context = getContext();
    var collection = context.getCollection();
    var response = context.getResponse();
    var createdDocument = response.getBody();

    // query for metadata document
    var filterQuery = 'SELECT * FROM root r WHERE r.id = "_metadata"';
    var accept = collection.queryDocuments(collection.getSelfLink(), filterQuery, updateMetadataCallback);
    if(!accept) throw "Unable to update metadata, abort";

    function updateMetadataCallback(err, documents, responseOptions) {
        if(err) throw new Error("Error" + err.message);

        if(documents.length != 1) throw 'Unable to find metadata document';
        var metadataDocument = documents[0];

        // update metadata
        metadataDocument.createdDocuments += 1;
        metadataDocument.createdNames += " " + createdDocument.id;

        var accept = collection.replaceDocument(metadataDocument._self, metadataDocument, function(err, docReplaced) {
            if(err) throw "Unable to update metadata, abort";
        });

        if(!accept) throw "Unable to update metadata, abort";
        return;
    }
}
'@
New-CosmosDbTrigger -Context $cosmosDbContext -CollectionId MyNewCollection -Id MyTrigger -TriggerBody $body -TriggerOperation All -TriggerType Post

Update an existing trigger for a collection in the database to execute before all operations:

$body = @'
function updateMetadata() {
    var context = getContext();
    var collection = context.getCollection();
    var response = context.getResponse();
    var createdDocument = response.getBody();

    // query for metadata document
    var filterQuery = 'SELECT * FROM root r WHERE r.id = "_metadata"';
    var accept = collection.queryDocuments(collection.getSelfLink(), filterQuery, updateMetadataCallback);
    if(!accept) throw "Unable to update metadata, abort";

    function updateMetadataCallback(err, documents, responseOptions) {
        if(err) throw new Error("Error" + err.message);

        if(documents.length != 1) throw 'Unable to find metadata document';
        var metadataDocument = documents[0];

        // update metadata
        metadataDocument.createdDocuments += 1;
        metadataDocument.createdNames += " " + createdDocument.id;

        var accept = collection.replaceDocument(metadataDocument._self, metadataDocument, function(err, docReplaced) {
            if(err) throw "Unable to update metadata, abort";
        });

        if(!accept) throw "Unable to update metadata, abort";
        return;
    }
}
'@
Set-CosmosDbTrigger -Context $cosmosDbContext -CollectionId MyNewCollection -Id MyTrigger -Body $body -TriggerOperation All -TriggerType Pre

Remove a trigger for a collection from the database:

Remove-CosmosDbTrigger -Context $cosmosDbContext -CollectionId MyNewCollection -Id MyTrigger

Working with Stored Procedures

Get a list of stored procedures for a collection in the database:

Get-CosmosDbStoredProcedure -Context $cosmosDbContext -CollectionId MyNewCollection

Create a stored procedure for a collection in the database:

$body = @'
function () {
    var context = getContext();
    var response = context.getResponse();

    response.setBody("Hello, World");
}
'@
New-CosmosDbStoredProcedure -Context $cosmosDbContext -CollectionId MyNewCollection -Id spHelloWorld -StoredProcedureBody $body

Update an existing stored procedure for a collection in the database:

$body = @'
function (personToGreet) {
    var context = getContext();
    var response = context.getResponse();

    response.setBody("Hello, " + personToGreet);
}
'@
Set-CosmosDbStoredProcedure -Context $cosmosDbContext -CollectionId MyNewCollection -Id spHelloWorld -StoredProcedureBody $body

Execute a stored procedure for a collection from the database:

Invoke-CosmosDbStoredProcedure -Context $cosmosDbContext -CollectionId MyNewCollection -Id spHelloWorld -StoredProcedureParameters @('PowerShell')

Remove a stored procedure for a collection from the database:

Remove-CosmosDbStoredProcedure -Context $cosmosDbContext -CollectionId MyNewCollection -Id spHelloWorld

Working with User Defined Functions

Get a list of user defined functions for a collection in the database:

Get-CosmosDbUserDefinedFunction -Context $cosmosDbContext -CollectionId MyNewCollection

Create a user defined function for a collection in the database:

$body = @'
function tax(income) {
    if(income == undefined) throw 'no input';
    if (income < 1000)
        return income * 0.1;
    else if (income < 10000)
        return income * 0.2;
    else
        return income * 0.4;
}
'@
New-CosmosDbUserDefinedFunction -Context $cosmosDbContext -CollectionId MyNewCollection -Id udfTax -UserDefinedFunctionBody $body

Update an existing user defined function for a collection in the database:

$body = @'
function tax(income) {
    if(income == undefined) throw 'no input';
    if (income < 1000)
        return income * 0.2;
    else if (income < 10000)
        return income * 0.3;
    else
        return income * 0.4;
}
'@
Set-CosmosDbUserDefinedFunction -Context $cosmosDbContext -CollectionId MyNewCollection -Id udfTax -Body $body

Remove a user defined function for a collection from the database:

Remove-CosmosDbUserDefinedFunction -Context $cosmosDbContext -CollectionId MyNewCollection -Id udfTax

How to Handle Exceeding Provisioned Throughput

When using Azure Cosmos DB it is quite common to exceed the throughput that has been provisioned against a collection (or across multiple collections). See this page for more information on request units and throughput provisioning.

When this happens requests will return a Too Many Request (error code 429). Usually just waiting a small amount of time and trying again will result in the request succeeding. However, the Cosmos DB PowerShell module provides a mechanism for configuring an automatic back-off and retry policy.

This is configured within the Context object that is usually passed to each Cosmos DB module function.

To configure a Back-off Policy, use the New-CosmosDbBackoffPolicy function:

$backoffPolicy = New-CosmosDbBackoffPolicy -MaxRetries 5
$cosmosDbContext = New-CosmosDbContext -Account MyAzureCosmosDB -Database MyDatabase -Key $primaryKey -BackoffPolicy $backoffPolicy

This will cause any functions that use the Context to automatically retry up to 5 times if a 429 response code is returned. Any other type of response code will throw an exception. The number of milliseconds to delay before retrying will be determined automatically by using the x-ms-retry-after-ms header returned by Cosmos DB.

Additional Back-off Policy options can be set to override or extend the value returned in the x-ms-retry-after-ms header.

Note: if the delay calculated by the policy is less than the value returned in the x-ms-retry-after-ms header, then the x-ms-retry-after-ms value will always be used.

The available Back-off Methods are:

  • Default
  • Additive
  • Linear
  • Exponential
  • Random

The following show examples of alternative policy back-off types that can implemented:

Default

$backoffPolicy = New-CosmosDbBackoffPolicy -MaxRetries 10 -Method Default -Delay 100

The delay of 100ms will always be used unless it is less than x-ms-retry-after-ms. The delay can be set to 0 and will cause the x-ms-retry-after-ms to always be used. It is the default Back-off Policy behavior.

Additive

$backoffPolicy = New-CosmosDbBackoffPolicy -MaxRetries 10 -Method Additive -Delay 1000

This will create a policy that will retry 10 times with a delay equaling the value of the returned x-ms-retry-after-ms header plus 1000ms.

Linear

$backoffPolicy = New-CosmosDbBackoffPolicy -MaxRetries 3 -Method Linear -Delay 500

This will create a policy that will wait for 500ms on the first retry, 1000ms on the second retry, 1500ms on final retry.

Exponential

$backoffPolicy = New-CosmosDbBackoffPolicy -MaxRetries 4 -Method Exponential -Delay 1000

This will create a policy that will wait for 1000ms on the first retry, 4000ms on the second retry, 9000ms on the 3rd retry and 16000ms on the final retry.

Random

$backoffPolicy = New-CosmosDbBackoffPolicy -MaxRetries 3 -Method Random -Delay 1000

A policy that adds or subtracts up to 50% of the delay period to the base delay each time can also be applied. For example, the first delay might be 850ms, with the second delay being 1424ms and final delay being 983ms.

Compatibility and Testing

This PowerShell module is automatically tested and validated to run on the following systems:

This module is no longer tested on PowerShell Core 6.x as PowerShell 7.x should be used. It should still work, but will no longer be verified. Issues with this module that only exist on PowerShell Core 6.x but not PowerShell 7.x will not be fixed.

This module should function correctly on other systems and configurations but is not automatically tested with them in every change.

Contributing

If you wish to contribute to this project, please read the Contributing.md document first. We would be very grateful of any contributions.

Cmdlets

A list of Cmdlets in the CosmosDB PowerShell module can be found by running the following PowerShell commands:

Import-Module -Name CosmosDB
Get-Command -Module CosmosDB

Help on individual Cmdlets can be found in the built-in Cmdlet help:

Get-Help -Name Get-CosmosDBUser

The details of the cmdlets contained in this module can also be found in the wiki.

Change Log

For a list of changes to versions, see the CHANGELOG.md file.

Links

cosmosdb's People

Contributors

armaanmcleod avatar azure-pipelines[bot] avatar bak-t avatar chrisjantzen avatar jasonchester avatar mikeybronowski avatar minorai avatar mwl88 avatar plagueho avatar potatoqualitee avatar smneal avatar splaxi avatar watersjohn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cosmosdb's Issues

Add support for Cosmos DB emulator

I tried 'localhost', 'localhost:8081' and other patterns. As per Tweet chat with Dan, this is not yet supported.
Dan: "This doesnโ€™t work because internally the URI to the CosmosDB is built by combining the $account with โ€˜.documents.azure.comโ€™. However, there should be a simple work around you could use (until I add built in support): After creating the $connection variable, set the BaseUri property to โ€˜http://localhost:8081โ€™. E.g. $connection.BaseUri = [uri] โ€˜http://localhost:8081โ€™. Iโ€™m assuming here the emulator uses HTTP rather than HTTPS. What Iโ€™ll do is add a way to set the base URI through a parameter when creating the connection object."
Thanks, great tool (a must IMHO).

Convert "Connection" Functions and Parameters to Context

Convert "Connection" Functions and Parameters to Context. This is to align better with how the AzureRM modules are named (e.g. New-AzureRmApiManagementContext).

Aliases should be used for the old function names and parameter names to ensure the change isn't breaking.

Unable to create Spatial index

When I try to create an included path which contains a spatial index the request fails because the precision property is always returned from New-CosmosDbCollectionIncludedPathIndex whether I specify it or not. The error returned is "The 'Spatial' index type should not have a precision specified."

Stored Procedure Invoke Enhancements

While testing out a performing complicated aggregations in cosmosdb using stored procs (https://github.com/lmaccherone/documentdb-lumenize) I ran smack into some limitations of the stored proc Implementation.

In order to address the limitations Invoke-CosmosDbStoredProcedure should

  • Support for specifying -PartitionKey
    -- this is required when executing a stored procedure in a partitioned collection
  • Change StoredProcedureParameters from string[] to object[]
    -- Should be able to handle most cases with ConvertTo-Json
  • Add -EnableScriptDebugging parameter when executing a stored procedure
    -- This appears to be missing from rest api documentation
    -- Relevant SO https://stackoverflow.com/questions/47855094/how-to-debug-azure-cosmos-db-stored-procedures
    -- Consider adding this to the cosmos request automatically when Powershell -Debug is used.

Planning to put together a PR for these if time allows.

Enhancement: Return the size of a collection

Following the contributions.md guidance, I'm writing an issue prior to submitting an enhancement.

When you use non-partitioned collections, Microsoft imposes a hard size limit of 10 GB. Microsoft allows you to set up alerting based on size, but it is the aggregate account size, not the collection size, rendering it moot.

The CosmosDB PowerShell module contains a lot of the foundation plumbing required to get this information. I've written a small enhancement to return size information about a collection. https://docs.microsoft.com/en-us/azure/cosmos-db/monitor-accounts indicates that this info can be retrieved by doing a GET on the collection, and the response contains usage information for the collection in the x-ms-resource-usage header.

The design introduces a new function in lib\collections.ps1 named Get-CosmosDBCollectionSize which returns information for a given collection, and leaves the existing Get-CosmosDBCollection function untouched. The params for Get-CosmosDBCollectionSize are identical to Get-CosmosDBCollection.

The response contains a hashtable containing the values of the following properties:
triggers, collectionSize, documentsCount, functions, storedProcedures, documentsSize, documentSize. Unfortunately, these do not appear to be documented; some discussion about the potential values are here: https://stackoverflow.com/questions/35274760/understanding-the-x-ms-resource-usage-in-documentdb-response-header.

With this enhancement, I've been able to set up basic alerting based on collection size. If this is of interest, I'm happy to submit a pull request.

Add a Function for creating a new Cosmos DB Account

There is no simple cmdlet available for creating a new CosmosDB Account in the AzureRM cmdlets. There is a command in the Azure CLI.

It would be useful to have a simple function that could use an ARM Template to easily create a Cosmos DB Account.

Suppress secondary error creating connection when not logged into Azure

$cosmosDbConnection = New-CosmosDbConnection -Account 'cosmosdb-bmd' -Database 'TestDatabase' -ResourceGroup 'cosmosdb-bmd-rgp' -MasterKeyType 'SecondaryMasterKey'
Invoke-AzureRmResourceAction : Run Login-AzureRmAccount to login.
At C:\Program Files\WindowsPowerShell\Modules\CosmosDB\2.0.0.152\lib\utils.ps1:82 char:21
+         $resource = Invoke-AzureRmResourceAction `
+                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], PSInvalidOperationException
    + FullyQualifiedErrorId : InvalidOperation

ConvertTo-SecureString : Cannot bind argument to parameter 'String' because it is null.
At C:\Program Files\WindowsPowerShell\Modules\CosmosDB\2.0.0.152\lib\utils.ps1:90 char:47
+ ... y = ConvertTo-SecureString -String ($resource.$MasterKeyType) -AsPlai ...
+                                        ~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [ConvertTo-SecureString], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.ConvertToSecureS
   tringCommand

After creating Store procedure and UDF i am not able see them on Azure Portal

Before submitting your issue for the CosmosDB project, please take a moment to provide the following details:

  • Version of PowerShell you're using
  • PowerShell host you're using (eg. Console Host, ISE, Visual Studio)
  • Operating system you're running
  • Version of CosmosDB you're using (use Get-Module -Name CosmosDB)

If you have a general question or suggestion, feel free to post on the CosmosDB Gitter Chat at https://gitter.im/PlagueHO/CosmosDB. This is also a great place to just say Hi, ask any questions you might have or get help.

Thanks for contributing your feedback and support! You can optionally submit a Pull Request against this project, if you have a fix you'd like to share.

Return Custom CosmosDB Exception on Error

A custom CosmosDB exception object should be created and returned when the CosmosDB server returns an error. This object should contain all the additional information that is returned by the CosmosDB server.

Allow IndexingMode = 'None'

When I try to provision an index with the indexing mode set to none, the New-CosmosDbCollectionIndexingPolicy function returns an error as this value does not exist in the ValidateSet.

Mapping of offers to collections

This may be more of a cosmosDB concepts issue rather than a CosmosDB Powershell issue, but thought I'd ask - feel free to close if out of scope.

I have a context for a given database, and have obtained a list of offers

$cosmosDbContext = New-CosmosDbContext -Account $account -Database $dd -ResourceGroup $rg -MasterKeyType 'SecondaryMasterKey'; $offers = Get-CosmosDbOffer -Context $cosmosDbContext; $colls = Get-CosmosDbCollection -Context $cosmosDbContext;

Then I have code that iterates through the offers, and eventually tries to match up the offer to a collection as follows:

foreach ($offer in $offers) { $collName = ($colls | where-object {$_._rid -eq $rid} ).Id }

However, it looks like those two objects really don't match up as noted by the different resource IDs below:

Offers:
Id: h-5E OfferVersion: V2 OfferType: invalid Etag: 00007801-0000-0000-0000-5acbb2610000 ResourceId: h-5E Timestamp: 4/9/2018 2:35:13 PM Uri: offers/h-5E/

Collections
id : Foo indexingPolicy : @{indexingMode=consistent; automatic=True; includedPaths=System.Object[]; excludedPaths=System.Object[]} _conflicts : conflicts/ _docs : docs/ _etag : "00000700-0000-0000-0000-5acd17070000" _rid : Q4QdAMy0QwE= _sprocs : sprocs/ _triggers : triggers/ _ts : 1523390215 _udfs : udfs/

Any thoughts on how i could match these two up?

Connect CosmoSDB with the New user

Hello Team ,

I have successfully created the new user for the COsmos DB .Could you please let me know how to use the use to connect to the Cosmos DB using 3T or any way to connect via this user

Thanks
Atul Gupta

New-CosmosDbDocument fails on partitioned collections

Version of PowerShell: 5.1.1
Running in: VSTS, ISE

New-CosmosDbDocument fails with a 400 error when the collection is partitioned. This looks to be down to the x-ms-documentdb-partitionkey header missing.

Example code:

    $document = @"
{
    `"id`": `"$([Guid]::NewGuid().ToString())`",
    `"content`": `"Some string`",
    `"more`": `"Some other string`"
}
"@

    $cosmosDbConnection = New-CosmosDbConnection -Emulator -Database 'Database'
    
    New-CosmosDbDatabase -Connection $cosmosDbConnection -Id 'Database'

    New-CosmosDbCollection -Connection $cosmosDbConnection -Id 'Collection' -OfferThroughput 2500 -PartitionKey "id"

    New-CosmosDbDocument -Connection $cosmosDbConnection -CollectionId 'Collection' -DocumentBody $document

If you remove the -PartitionKey "id" from the above it works fine. I have tested a local change with PartitionKey as a parameter to New-CosmosDbDocument, doing something like:

	if ($PSBoundParameters.ContainsKey('PartitionKey'))
    {
        $headers += @{
            'x-ms-documentdb-partitionkey' = '["' + $PartitionKey + '"]'
        }
        $null = $PSBoundParameters.Remove('PartitionKey')
    }

And that all seems OK, I haven't put in a PR as I'm unsure what approach you'd prefer :)

Error 401 When Creating Collections in Databases

I have issues when the database name is capital
image

If I run that script above it will correctly query the collection
If I make a 2nd database with Example it will 401 error

image

image

The creation Error is below :

image

Code for that is :
image

PSVersion 5.1.16299.251 , I get same error on both 2.08.350 and 2.0.7.288

Set-CosmosDbOffer fails with "Unauthorized"

The Set-CosmosDbOffer cmdlet is currently failing with the following output:

Invoke-RestMethod : {"code":"Unauthorized","message":"The input authorization   
token can't serve the request. Please check that the expected payload is built 
as per the protocol, and check the key being used. Server used the following 
payload to sign: 'put\noffers\n{removed}\nmon, 23 apr 2018 15:47:13 
gmt\n\n'\r\nActivityId: {removed}, 
Microsoft.Azure.Documents.Common/1.21.0.0"}
At /usr/local/microsoft/powershell/6.0.0-beta.7/Modules/CosmosDB/2.0.9.364/lib/
utils.ps1:402 char:23
+         $restResult = Invoke-RestMethod @invokeRestMethodParameters
+                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (Method: PUT, Re...ication/jso 
   n
}:HttpRequestMessage) [Invoke-RestMethod], HttpResponseException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShe 
   ll.Commands.InvokeRestMethodCommand

The fix seems to be relatively simple and straightforward. There just needs to be a .ToLowerInvariant() added to the end of Line 327 in the /src/lib/utils.ps1 file.

My local environment details:

$PSVersionTable | Format-List               


Name  : PSVersion
Value : 6.0.0-beta

Name  : PSEdition
Value : Core

Name  : GitCommitId
Value : v6.0.0-beta.7

Name  : OS
Value : Darwin 17.5.0 Darwin Kernel Version 17.5.0: Mon Mar  5 22:24:32 PST 
        2018; root:xnu-4570.51.1~1/RELEASE_X86_64

Name  : Platform
Value : Unix

Name  : PSCompatibleVersions
Value : {1.0, 2.0, 3.0, 4.0...}

Name  : PSRemotingProtocolVersion
Value : 2.3

Name  : SerializationVersion
Value : 1.1.0.1

Name  : WSManStackVersion
Value : 3.0
Get-Module -Name CosmosDB | Format-List     


Name              : CosmosDB
Path              : /usr/local/microsoft/powershell/6.0.0-beta.7/Modules/Cosmos
                    DB/2.0.9.364/CosmosDB.psm1
Description       : This module provides cmdlets for working with Azure Cosmos 
                    DB databases, collections, documents, attachments, offers, 
                    users, permissions, triggers, stored procedures and user 
                    defined functions.
ModuleType        : Script
Version           : 2.0.9.364
NestedModules     : {}
ExportedFunctions : {Get-CosmosDbAttachment, 
                    Get-CosmosDbAttachmentResourcePath, 
                    Get-CosmosDbCollection, 
                    Get-CosmosDbCollectionResourcePath...}
ExportedCmdlets   : 
ExportedVariables : 
ExportedAliases   : New-CosmosDbConnection

400 Bad Request when using -Query

  • Version of PowerShell you're using: 5.1.14393.1770
  • PowerShell host you're using: ISE
  • Operating system you're running: Windows 10
  • Version of CosmosDB: 1.0.10.108

Issuing the following command;

$params = @(@{name = "@in_newrelic";value="false";})
$documents = Get-CosmosDbDocument -Database devops -Connection $Connection -CollectionId 'inventory' -Query "SELECT * FROM inventory WHERE (inventory.in_newrelic = @in_newrelic)" -QueryParameters $params

returns the following response;

Invoke-WebRequest : The remote server returned an error: (400) Bad Request.
At C:\Program Files\WindowsPowerShell\Modules\CosmosDB\1.0.10.108\lib\utils.ps1:477 char:23
+ ... estResult = Invoke-WebRequest -UseBasicParsing @invokeRestMethodParam ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
 
ConvertFrom-Json : Cannot bind argument to parameter 'InputObject' because it is null.
At C:\Program Files\WindowsPowerShell\Modules\CosmosDB\1.0.10.108\lib\documents.ps1:294 char:54
+         $tempObject = (ConvertFrom-JSON -InputObject $result.Content)
+                                                      ~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [ConvertFrom-Json], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.ConvertFromJsonCommand

When I run the following,

$Query = "select * from inventory where (in_newrelic = @in_newrelic)"
$QueryParameters = @(@{name = "@in_newrelic";value="false";}) # @(@{name = "@in_newrelic";value = "false"})

**$bodyObject = @{ query = $Query }
if (-not [String]::IsNullOrEmpty($QueryParameters))
{
    $bodyObject += @{ parameters = $QueryParameters }
}
ConvertTo-Json -InputObject $bodyObject**

the output JSON object appears to match the desired format according to the CosmosDB RestAPI documentation.

{
    "query":  "select * from inventory where (in_newrelic = @in_newrelic)",
    "parameters":  [
                       {
                           "value":  "false",
                           "name":  "@in_newrelic"
                       }
                   ]
}

The bold portion of the script was taken from the source code. I wanted to ensure that the resulting body passed to the RestAPI was in the desired format, which it is. I also checked and the source code seems to be setting the "x-ms-documentdb-isquery" header, so I cannot tell what else could be wrong here.
I can return documents from the table using the following command;
$documents = Get-CosmosDbDocument -Database devops -Connection $Connection -CollectionId 'inventory' -MaxItemCount 5

Incorrect Handling of PartitionKey in Get-CosomosDBDocument with -Query parameter.

PartitionKey is handled when -Id is passed

@ https://github.com/PlagueHO/CosmosDB/blob/dev/src/lib/documents.ps1#L156-L159

    if (-not [String]::IsNullOrEmpty($Id))
    {
        # A document Id has been specified
        if ($PSBoundParameters.ContainsKey('PartitionKey'))

But is not handled in the else block. @ https://github.com/PlagueHO/CosmosDB/blob/dev/src/lib/documents.ps1#L173-L259

Error Produced:

PS /Users/jason.chester/_dev/LDM/scripts/generic> Measure-CosmosDbPartitionDocuments -PartitionKey "bmi"
Invoke-CosmosDbRequest : A parameter cannot be found that matches parameter name 'PartitionKey'.
At /usr/local/share/powershell/Modules/CosmosDB/2.0.10.388/lib/documents.ps1:242 char:42
+         $result = Invoke-CosmosDbRequest @PSBoundParameters `
+                                          ~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidArgument: (:) [Invoke-CosmosDbRequest], ParameterBindingException
+ FullyQualifiedErrorId : NamedParameterNotFound,Invoke-CosmosDbRequest

Attatched file contains edits to documents.ps1 which resolved my specific issue.
documents.ps1.txt

  • Version of PowerShell you're using
Name                           Value
PSVersion                      6.0.0
PSEdition                      Core
GitCommitId                    v6.0.0
OS                             Darwin 17.6.0 Darwin Kernel Version 17.6.0: Fri Apr 13 19:57:44 PDT 2018; root:xnu-4570.60.17.0.1~3/RELEASE_X86_64
Platform                       Unix
  • PowerShell host you're using (eg. Console Host, ISE, Visual Studio)
    pwsh in terminal & VS Code Host

  • Operating system you're running
    macOS High Sierra
    version 10.13.5 Beta (17F45c)

  • Version of CosmosDB you're using (use Get-Module -Name CosmosDB)
    CosmosDB 2.0.10.388

How does one obtain error details?

  • [5.1.16299.431] Version of PowerShell you're using
  • [Console] PowerShell host you're using
  • [Win10] Operating system you're running
  • [2.0.12.418 ] Version of CosmosDB you're using

How does one get the error details when CosmosDB returns an error? When using New-CosmosDbDocument, for example, I occasionally receive this:

Invoke-RestMethod : The remote server returned an error: (500) Internal Server Error.
At C:\Program Files\WindowsPowerShell\Modules\CosmosDB\2.0.12.418\lib\utils.ps1:512 char:23
+         $restResult = Invoke-RestMethod @invokeRestMethodParameters
+                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebExc
   eption
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

I can't see any way to get more information. Sometimes, the error is 400 Invalid Request, which is often missing or malformed data, but I have to figure out what's missing by trial and error.

Authorization token is incorrect for database names with uppercase letters

Function New-CosmosDbAuthorizationToken incorrectly converts the ResourceLink (ResourceId variable) to lowercase here.

The MS documentation states

The ResourceLink portion of the string is the identity property of the resource that the request is directed at. ResourceLink must maintain its case for the id of the resource. Example, for a collection it will look like: "dbs/MyDatabase/colls/MyCollection"

  • [5.1] Version of PowerShell you're using
  • [Console host and VS Code] PowerShell host you're using (eg. Console Host, ISE, Visual Studio)
  • [Win10 x64] Operating system you're running
  • [2.0.7.342] Version of CosmosDB you're using (use Get-Module -Name CosmosDB)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.