insomniacc/pssnow

A powershell module for interacting with the ServiceNow REST API

pssnow's Issues

Batch Request Robustness

When a batch request reaches a size or processing limit, the system cancels the transaction and returns unprocessed requests in the unserviced JSON array in the response.

as per https://docs.servicenow.com/bundle/sandiego-application-development/page/integrate/inbound-rest/concept/batch-api.html

While currently this is not pressing, the user can adjust to what is necessary based on the batch size, it might be a good feature to check default limits and have adjustable limits (or maybe they should be pulled from the instance).
Any returned/cancelled requests will need to be pushed back to be processed in another batch.

Complete basic TABLE API framework

Basic CRUD operations with generic template functions need to be added.
They should have comprehensive coverage of parameters from:
https://docs.servicenow.com/bundle/tokyo-application-development/page/integrate/inbound-rest/concept/c_TableAPI.html

Built in a way that all the interaction is abstracted away in private functions.
Top level public functions should use the parameters from the generic template functions for each crud operation, allowing much faster additions for other table functions, but also better accessibility for modifications to those with less coding knowledge.

Readme.md

Update readme.md with general information around the repo, how to contribute, design general how to.
Also add user guide information, basic how-to for common functions.

Function Documentation

Write help text for all functions.
Generate PlatyPS Docs and add to the build script.

Set-SNOWUserPhoto does not replace an existing photo

If a photo is already set, the attachment is just added along side the existing one.
I'm not sure if I can modify an existing attachment, if it's possible I'll need to add a new Update-SNOWAttachment function, otherwise removing and re-adding the hidden photo will do the job.

New-SNOWApprover - Blank fields

If using New-SNOWApprover as shown:
New-SNOWApprover -sysapproval "d2cdb552db252200a6a2b31be0b8f5ee" -document_id "d2cdb552db252200a6a2b31be0b8f5ee" -approver "1b3f2d6047292110d3e5fa8bd36d4351" -state "Requested" -due_date "2023-09-16 13:15:03" -source_table "change_request" -verbose -passthru

There's an object created, due date, state & source table are set correctly, however the reference fields for document_id and approver are blank.

Same behaviour here noticed in this thread:
https://www.servicenow.com/community/developer-forum/adding-approvers-to-an-open-change-request-via-rest-api/m-p/1398754

I'll leave this out for now, typically approvals will be generated via workflows internally anyway rather than externally via rest, the main function here is the Set-SNOWApprover which will allow external approvals to be updated.

Better rest error handling for 5.1

If using core you'll get a relevant error from a bad rest request including the error message.
example:

Invoke-RestMethod: 
Line |
2265 |              $Response = Invoke-RestMethod @RestSplat
     |                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | {"result":{"errCode":"42202","errMsg":"This item cannot be submitted for multiple people as it is not configured with Requested For"}}

On 5.1 errors are basic and messages not displayed:
example:

Invoke-RestMethod : The remote server returned an error: (400) Bad Request.
+ ... $Response = Invoke-RestMethod -Method POST -Uri $URI -Headers $Header ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

Enhancing the error handling and output would be a better user experience on 5.1

Add support for OAuth2 authentication

Pester testing

Create initial pester testing for all functions
Add to build script

Pester Testing for Assert-SNOWAuth

Any command that uses invoke-webrequest or invoke-restmethod must also invoke Assert-SNOWAuth.

Oauth is not refreshing the token after expiry

Title says it all!

Assert-SNOWAuth is the problem

Check instance aliveness on Set-SNOWAuth

Problem:
Currently if using a developer instance which has not been woken up, you will be able to set auth without error and will get an error on core similar to this if trying to make any subsequent calls:

Conversion from JSON failed with error:

There's likley a similar error on 5.1.

Solution:
While using Set-SNOWAuth an aliveness check should be made to the instance to determine if it responds and also if it's 'awake', if not it should return a relevant error message.

Table dynamic params do not pull default values

I'll explain with a possible use case:
Get-SNOWObject is used as a template function for all other Get-SNOW* (table) commands.
It's parameters are pulled through to form the framework of those other functions (into the dynamic param block)
Default values are not pulled through as it's not part of the 'Get-Command' dataset.

Now lets say that by default, we wanted all Get-SNOW* functions to default the -DisplayValue to 'true'.
I could set this in the Get-SNOWObject command, but it would not apply to all others templating from this.

Possible Solution:
Ast parsing

This is not currently presenting an issue, although might be a nice to have for the use case specified above.

Unable to query request item variables

Describe the bug
Unable to query request item variables.

To Reproduce
Steps to reproduce the behavior:

1. Import-Module
2. Set-SNOWAuth -Instance "dev123456" -Credential (get-credential)
3. Get-SNOWRITMVariableSet -number "RITM0631749" -Verbose
4. See error as per below:

VERBOSE: GET https://removed.service-now.com/api/now/v2/table/sc_req_item?sysparm_query=number=RITM0631749&sysparm_fields=sys_id&sysparm_limit=1000&sysparm_offset=0 with 0-byte
payload
VERBOSE: received -1-byte response of content type application/json;charset=UTF-8
VERBOSE: GET
https://removed.service-now.com/api/now/v2/table/sc_item_option_mtom?sysparm_query=request_item=3b42f52a1ba4f450e49c43bdcc4bcbdc&sysparm_display_value=True&sysparm_exclude_reference_l
ink=True&sysparm_fields=sc_item_option.item_option_new.name,sc_item_option.item_option_new,sc_item_option.value&sysparm_limit=1000&sysparm_offset=0 with 0-byte payload
VERBOSE: received -1-byte response of content type application/json;charset=UTF-8

Get-SNOWRITMVariableSet : Cannot bind argument to parameter 'Name' because it is null.
At line:1 char:1
+ Get-SNOWRITMVariableSet -number "RITM0631749" -Verbose
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Get-SNOWRITMVariableSet], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Get-SNOWRITMVariableSet

Expected behavior
To see the output of the associated request Item variables from the dotwalked field in the sc_item_option junction table.

Environment (please complete the following information):

• Module Version: 1.1.0
• Powershell Version: 5.1
• OS: Windows 10
• Servicenow Version: [e.g. Tokyo] Sandiego

Additional context
Firstly, very cool work on the module. More than likely I will become an active user (maybe a contributor too at some point, if you're open to PRs), given I am tasked with automating a lot of our workload at the moment.

When I manually query the junction table via the Table API, there is no sc_item_option column included in the response. I see the following columns returned:
sys_id, sys_updated_by, sys_created_on, sys_mod_count, requested_item (ref field), sys_updated_on, sys_tags, sys_created_by

When I add a filter to explicitly include the field, it is empty. This seems to be the case for all items I've queried so far. Interestingly, when I query a request item via the JSONv2 web service rather than the Table API, all the appropriate variables are listed.

I am starting to suspect that the issue I am seeing does not really relate to the module itself and is more centered around our ServiceNow instance, maybe the ACLs/API configuration. Unfortunately I am not a SN expert, nor am I a SN admin in our environment, so my ability to tinker and test is very limited.

Wondering if you have any advice here.

Thanks,
f0oster

Create Import-SNOWObject

Create a generic Import-SNOWObject function for interaction with the Import API
Add to Invoke-SNOWBatch

https://docs.servicenow.com/bundle/tokyo-application-development/page/integrate/inbound-rest/concept/c_ImportSetAPI.html

Changelog

Create a changelog for the module

Add more common table functions

ApprovalGroups, CMDBCI, Department, Location, SCItem, OrderGuides, UserGroups

Set-SnowAuth Invoke-WebRequest should -UseBasicParsing

Title says it all, it should have use basic parsing enabled as default for non interactive use on PS 5.1

Document adding custom variables endpoint

Dependant on #8
Add markdown for adding custom endpoint with helper function

Invoke-SNOWRestMethod

A generic Invoke-SNOWRestMethod would be useful, it might be a handy way to interact with other servicenow API's until this module supports them.
It should assert-snowauth so that users can leverage the automatic oauth token renewal and also basic auth.

FunctionBuilder - Additional metadata: Mandatory & Labels

FunctionBuilder can be enhanced with creating SET functions by pulling the mandatory metadata and applying this to the parameters.
Labels (display values) could also be pushed as aliases (spaces will need replacing with _).

RITM Variables custom endpoint

Getting RITM Variables is quite slow.
I cant remember the exact process of the top of my head but I think it goes something like this:
RITM > Variable Values > MTOM Pointer record > Variable Definitions (questions and text)
It's quite API intensive and if you're doing it for lots of RITMS, given how many calls there are, it can be slow.

A more efficient way is to use a custom endpoint in snow to let it do the work and return the variables directly from an input RITM number and/or sysid.

This needs to be documented (#12) in the repo as it's not out of the box, possibly even with a helper function to create the feature in it's own update set for ease of adding this.

Write functions to interact with the service catalog

Get functions for catalog, category, item etc.
Functions to add items to cart and checkout, thus having the ability to mimic a user logging a request or incident, or end to end test integrations that may have been implemented with this module.

https://developer.servicenow.com/dev.do#!/reference/api/tokyo/rest/c_ServiceCatalogAPI

Potential alternative user photo attach method

Once #10 is complete, test if it's possible to add a user photo with the new attachment command instead of the existing method.
I originally added user photos in New-SNOWUserPhoto using the ecc_queue AttachmentCreator method, you can pass a base64 string and it winds up as an attachment in the ixx format on the record, but there might be a cleaner way.

Document functionBuilder

Create docs / demo for functionBuilder

Attachment API

Write functions to interact with the Attachment API

https://developer.servicenow.com/dev.do#!/reference/api/tokyo/rest/c_AttachmentAPI

SNOWAuth Enhancements

Set-SNOWAuth could do with some error handling around the OAuth token call, if there's no token retrieved or a non 200 code, write an error / throw.
Also Get-SNOWAuth may be a worthwhile addition to return the $script:SNOWAuth variable from module scope for user debugging purposes.

Additional common table functions

Once #7 is completed, the most common table functions should be added for basic use.
Incident
Request
RequestedItem
Task
Approvals
Change

Support for additional variable types in Get-SNOWRITMVariableSet

Is your feature request related to a problem? Please describe.
Some request item variable types, such as multi row variable sets (there are likely others that are similar - I'm not sure), do not appear to store their value within the sc_item_option table, and as such the dotwalked property for the value queried by Get-SNOWRITMVariableSet is empty.

I'm not sure where this data is stored table wise, or what the schema looks like.

Describe the solution you'd like
Update the Get-SNOWRITMVariableSet function to detect the request items variable types, and query any additional tables needed to fetch the values.

With respect to the above, if variable type detection is possible/practical, it might be a good idea to add some constraints on changes made via Set-SNOWRITMVariableSet based on variable types, or to add dynamic update logic based on the type. I'm not sure what would happen if I tried to set a value on a MRVS, but I'm not too keen to test it and find out in case it causes any issues.

New-SNOWAttachment does not work for cases

As pointed out here:
https://www.reddit.com/r/PowerShell/comments/123jlls/comment/jdv4ect/?context=3

I'll have to check the CSM plugin to see how cases are handled and if it uses the same attachment table or not.

Document New-SNOWUserPhoto Batching Support

There's a caveat to using -photo and -AsBatchRequest at the same time on New-SNOWUser.
The workaround is to create users in one batch, parse the output and pass the sys_id's and photos into another batch using New-SNOWUserPhoto.

Email API Support

https://docs.servicenow.com/bundle/sandiego-application-development/page/integrate/inbound-rest/concept/email-api.html

Doesnt work behind corporate firewall.

Describe the bug
Doesnt work behind corporate firewall

To Reproduce
Steps to reproduce the behavior:

1. Import-Module
2. Set-SNOWAuth -Instance "dev123456" -Credential (get-credential)

Can basically fix it by adding a check for corporate firewall then using default credentials / input credentials.
https://bycode.dev/2019/08/22/how-to-use-powershell-invoke-webrequest-behind-corporate-proxy/

Example:
`

if([String]::IsNullOrWhiteSpace($Instance)){
    Write-Error "Instance cannot be an empty string" -ErrorAction stop
}

if($instance -like "https://*"){
    $instance = $instance.replace('https://','')
}

if($instance -like "*.service-now.com*"){
    $instance = $instance.split('.') | Select-Object -first 1
}

$dest = "https://registry.npmjs.org/express"
$proxy = ([System.Net.WebRequest]::GetSystemWebproxy()).GetProxy($dest)

#? Aliveness/Hibernation check for developer instances
$response = Invoke-WebRequest -Uri "https://$Instance.service-now.com/stats.do" -ErrorAction Stop -Verbose:$false -UseBasicParsing -Proxy $proxy -ProxyUseDefaultCredentials
if($response -and $response.content -like "*Instance Hibernating page*"){
    Throw "This servicenow instance is hibernating. Please wake the instance up and use $($PSCmdlet.MyInvocation.MyCommand.Name) again."
}

`

Add Invoke-SNOWBatch

Write Invoke-SNOWBatch to handle submissions to the batch API as per:
https://docs.servicenow.com/bundle/sandiego-application-development/page/integrate/inbound-rest/concept/batch-api.html

All table functions should have a hidden? 'AsBatchRequest' parameter that allows the return of a formatted payload suitable to be passed into the Invoke-SNOWBatch function for bulk submission.

Invoke-SNOWBatch should also have a -scriptblock param that can be used as a wrapper around code to effectively do the same as the previous statement, but automatically recognise and apply the -AsBatchRequest parameter and collect the output, making it more user friendly.

Invoke-SNOWBatch should leverage Invoke-Parallel for 5.1 usage.

Set/New-SNOW table commands 'Number' param does not work

The -Number parameter on set/new commands has been inadvertently added by the function builder.
It cannot be used for lookup in the present configuration and I think that adhering to using the GET commands for this purpose and relying on sys_id for SET is a more sensible option currently.
It's a value that cannot be changed but does not show as read only in the sys_dictionary record.

Build & Release script

Create & finish build script.

It should be able to:

• bump the version number
• build to an output folder
• generate the manifest
• generate documentation with platyps
• run PSScriptAnalyzer
• run pester testing
• update the changelog
• publish to the gallery

Revise & refactor functionBuilder

Since every SNOW instance is developed differently for each org, things change.
That's really the primary issue that led me to build this module.
I've seen multiple instances of SNOW in different orgs with completely different config and settings, which makes taking a single module and using it across them all quite difficult. What works for one may not for another.

The framework with the table API was designed to make modifying table functions easier and save time, but it still takes time to check attributes, rename parameters, add validation scripts, test, etc.

The logic behind functionBuilder is to auto generate Table API functions based on the current config.
I'm still yet unsure how I want to fully implement, it could potentially be a 'one time initialization' that generates all the functions of the module against an instance of snow, or just a tool that users can implement themselves.
For v0.0.1 I'll leave it as the latter and provide some basic OOB style table functions that should work for most.

The existing functionBuilder script needs revising, requirements cleanly listing and then rewriting in a much cleaner way.

Invoke-SNOWBatch should support core native -paralllel

-Parallel currently uses invoke-parallel for both 5.1 and core but might be best if the native core method is utilized where it can be.

Can't seem to update RITM Custom Variable field

I have a Custom Variable named "subject". I am able to read the variable in the REST API (both Postman & API Explorer) using the name "variables.subject". When I try to update it through the REST API, it never updates.

I can successfully update native RITM variables ("short_description" & "work_notes") but not any Custom Variables.

Does anybody have a successful example of being able to update Custom Variables using REST API?

I also tried updating them using the Set-SNOWSCRequestedItem function in PSSnow using the "-Properties" parameter but no luck. Any ideas?

Thanks,

Invoke-SNOWStats

Add Invoke-SNOWStats

This will interact with the aggregate API:
https://developer.servicenow.com/dev.do#!/reference/api/rome/rest/c_AggregateAPI

Handle rate limiting

A useful enhancement would be to auto handle rate limiting errors, retry x times with the backoff provided in the X-RateLimit-Reset or Retry-After header, if the request still cannot be sent it should throw an exception.

https://docs.servicenow.com/bundle/sandiego-application-development/page/integrate/inbound-rest/concept/inbound-REST-API-rate-limiting.html

Only literal values are accepted in reference field lookups

While developing a builder function for #9
I noticed that param based url query strings in existing functions do not take display values, only literal values.
Meaning, if you have a reference field you can only provide the SysID. It's not that user friendly.
I've seen the behaviour I want by omitting sysparm_query from the call, and only using key value pairs instead.

Table Datetime conversions

I'm currently in two minds to implement this or not.
Currently dates/times are all considered strings, they are excluded from the get parameters, but can be used in query strings which expects the 'snow' datetime formats anyway.
Set command parameters also expect the snow format as a string.

It's possible to change the behaviour of the set commands and apply datetime conversions to the parameters so they can accept PowerShell standard datetimes but I'm not sure if I should leave this alone for consistency or implement for usability.

If once this is released I get any feedback I'll take another look.

New-SNOWUserPhoto

The -photo param was added to New-SNOWUser as a quality of life feature, but because adding a user photo is technically a separate call to another endpoint / table that requires the newly created user's sys_id, this approach does not work with the batch API.
A warning has been added if -photo is applied with -AsBatchRequest to New-SNOWUser.

As an alternative, if this does need to be batched New-SNOWUserPhoto should be added as a function to split this out.
Once complete it should be retrofitted into New-SNOWUser to clean up any duplicate code.

Batching can be done with this new function by first creating the users in batches, parsing the output and sending a separate series of requests to the batch API to create the photos with the new user sys_id's.

Feedback ---- Thank you for this!

Just wanted to say thank you for this! I'm working on some in-house tools for our team and this is a HUGE timesaver for me!

You're awesome!

Invoke-SNOWBatch v2 API

Strangely if I leave the default URI for the batch API, or specify v2 in the URL my calls always seem to fail with a 400 bad request response.
There's nothing standing out in the documentation to indicate why this may be, or that I'm doing anything wrong:
https://docs.servicenow.com/bundle/tokyo-application-development/page/integrate/inbound-rest/concept/batch-api.html

For now it works on v1 and doesn't seem to pose any limitations or issues.

Linux support

Having written and tested this in a windows only environment, I'm fairly certain there will probably be issues when trying to run in other OS's.
Ideally this module needs to be cross compatible.

FunctionBuilder is causing parameter conflicts

I've noticed that Set-SNOWSCRequestedItem & New-SNOWSCRequestedItem do not work in batch requests.
It seems this is because the Get-Command output's parameter property is null.

It seems both these commands have a parameter called $configuration_item, but also another one called $cmdb_ci with the alias of configuration_item. Which is obviously causing conflict.

I'm not sure if this aliased on the SNOW side and it's actually the same field, I'll need to test that. If it is I'll need to identify and ignore those aliases, otherwise if it's not, I'll need to validate any aliases to make sure there's no duplicates against param names, if there is they should be skipped.

-fields params should have validation

-fields param should be count validated so that an empty array cannot be passed in

Create functionBuilder

Since the module has a framework for the table API it would be beneficial to create a function builder with input for tablename and function name. It should be able to generate a standard CRUD function and output the ps1 file into the src folder along with adding it to the manifest.

It will need to lookup the target table in snow, pull any attributes for parameters, including from any extended tables recursivly.