Giter Site home page Giter Site logo

cisagov / sparrow Goto Github PK

View Code? Open in Web Editor NEW
1.4K 108.0 182.0 57 KB

Sparrow.ps1 was created by CISA's Cloud Forensics team to help detect possible compromised accounts and applications in the Azure/m365 environment.

License: Creative Commons Zero v1.0 Universal

PowerShell 100.00%

sparrow's Introduction

This repo is archived as of 27 Dec 2022 - and is no longer being maintained.

Aviary

Aviary is a new dashboard that CISA and partners developed to help visualize and analyze outputs from its Sparrow detection tool released in December 2020. Sparrow helps network defenders detect possible compromised accounts and applications in Azure/Microsoft O365 environments. CISA created Sparrow to support hunts for threat activity following the SolarWinds compromise. Aviary--a Splunk-base dashboard--facilitates analysis of Sparrow data outputs.

To download Aviary, visit releases.

Sparrow.ps1

Sparrow.ps1 was created by CISA's Cloud Forensics team to help detect possible compromised accounts and applications in the Azure/m365 environment. The tool is intended for use by incident responders, and focuses on the narrow scope of user and application activity endemic to identity and authentication based attacks seen recently in multiple sectors. It is neither comprehensive nor exhaustive of available data, and is intended to narrow a larger set of available investigation modules and telemetry to those specific to recent attacks on federated identity sources and applications.

Sparrow.ps1 will check and install the required PowerShell modules on the analysis machine, check the unified audit log in Azure/M365 for certain indicators of compromise (IoC's), list Azure AD domains, and check Azure service principals and their Microsoft Graph API permissions to identify potential malicious activity. The tool then outputs the data into multiple CSV files that are located in the user's default home directory in a folder called 'ExportDir' (ie: Desktop/ExportDir).

For more guidance on how to use Sparrow and Aviary, please see: https://us-cert.cisa.gov/ncas/alerts/aa21-008a

Requirements

The following AzureAD/m365 permissions are required to run Sparrow.ps1, and provide it read-only access to the Tenant.

  • Azure Active Directory:
    • Security Reader
  • Security and Compliance Center:
    • Compliance Administrator
  • Exchange Online Admin Center: Utilize a custom group for these specific permissions:
    • Mail Recipients
    • Security Group Creation and Membership
    • User options
    • View-Only Audit log
    • View-Only Configuration
    • View-Only Recipients

To check for the MailItemsAccessed Operation, your tenant organization requires an Office 365 or Microsoft 365 E5/G5 license.

Unified Audit Logs will need to be enabled.

Installation

Sparrow.ps1 does not require any extra steps for installation once the permissions detailed in Requirements are satisfied.

The function, Check-PSModules, will check to see if the three required PowerShell modules are installed on the system and if not, it will use the default PowerShell repository on the system to reach out and install. If the modules are present but not imported, the script will also import the missing modules so that they are ready for use.

The required PowerShell modules:

To install the PowerShell modules, run the following commands:

Install-Module ExchangeOnlineManagement
Install-Module AzureAD
Install-Module MSOnline 

Usage

To download and run Sparrow.ps1, type the following command into a PowerShell window (assuming file is in your working directory):

Invoke-WebRequest 'https://github.com/cisagov/Sparrow/raw/develop/Sparrow.ps1' -OutFile 'Sparrow.ps1' -UseBasicParsing; .\Sparrow.ps1

Using Behind A Proxy

If you are executing the script from behind a proxy, you may need to run the following commands, substituting your proxy server prior to execution:

[System.Net.WebRequest]::DefaultWebProxy = New-Object System.Net.WebProxy('http://proxyname:port')
[System.Net.WebRequest]::DefaultWebProxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials

Issues

If you have issues using the code, open an issue on the repository!

You can do this by clicking "Issues" at the top and clicking "New Issue" on the following page.

Contributing

We welcome contributions! Please see here for details.

License

This project is in the worldwide public domain.

This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication.

All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.

Legal Disclaimer

NOTICE

This software package (“software” or “code”) was created by the United States Government and is not subject to copyright. You may use, modify, or redistribute the code in any manner. However, you may not subsequently copyright the code as it is distributed. The United States Government makes no claim of copyright on the changes you effect, nor will it restrict your distribution of bona fide changes to the software. If you decide to update or redistribute the code, please include this notice with the code. Where relevant, we ask that you credit the Cybersecurity and Infrastructure Security Agency with the following statement: “Original code developed by the Cybersecurity and Infrastructure Security Agency (CISA), U.S. Department of Homeland Security.”

USE THIS SOFTWARE AT YOUR OWN RISK. THIS SOFTWARE COMES WITH NO WARRANTY, EITHER EXPRESS OR IMPLIED. THE UNITED STATES GOVERNMENT ASSUMES NO LIABILITY FOR THE USE OR MISUSE OF THIS SOFTWARE OR ITS DERIVATIVES.

THIS SOFTWARE IS OFFERED “AS-IS.” THE UNITED STATES GOVERNMENT WILL NOT INSTALL, REMOVE, OPERATE OR SUPPORT THIS SOFTWARE AT YOUR REQUEST. IF YOU ARE UNSURE OF HOW THIS SOFTWARE WILL INTERACT WITH YOUR SYSTEM, DO NOT USE IT.

sparrow's People

Contributors

akennerly avatar crash0ver1d3 avatar deemonsecurity avatar genericdevname avatar itsreallynick avatar kennethmldk avatar nomorefood avatar surbo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sparrow's Issues

Cheatsheet

Is there a cheat sheet that outlines what one should be looking for in the excel output?

detection to gather all azure app ids, and process via loop for sharepoint and exo investigation is missing

💡 Summary

What is the work, as a high-level summary?
Add detection to collect AzureapplicationIds to an object. Perform the SuspectAppID investigations on all available azuread application ids.

Motivation

Why does this work belong in this project?
DFIR needs to have repeatable, automatable actions. The current state of the script, allows an individual to perform investigation on one app id at a time, entered manually. This work will allow an individual to choose to investigate all app ids, and do the work to perform the UAL analysis, and then organize the data.
This would be useful because...

Implementation notes

Please provide details for implementation, such as:

  • an example for how this would be used
  • what this would look like
  • how this would act
  • any related work, including links to related issues

Acceptance criteria

How do we know when this work is done?
I have posted this pull request with the fix to the solution.
d2af7e0

  • Criterion

Warning; Excel not found - skipping combined file.

🐛 Bug Report

Getting this error "Warning; Excel not found - skipping combined file." once the script is finished running

To Reproduce

Steps to reproduce the behavior:

  • Just run the script using defaults

Expected behavior

I suppose all the separate .CSV files would be merged. I have Excel so I'm not sure why it's saying not found.

Files in ExportDir

After finish the script and creating the files, all these files was empty. Can be user permissions? The user are Global Admin. Thanks.

Choose your Azure Environment [AzureCloud]

I have multiple subscription and having difficulty in selecting the right subscription.

What should i feed as input ? subscription id ? tenant id ? for the below option

Choose your Azure Environment [AzureCloud]

AppId's in PSLogin_Operations_Export query not in results

🐛 Summary

We have the E3 license for M365.

We’re looking at the output of the Sparrow tool reports. In PSLogin_Operations_Export.csv, we should be seeing the results of the three queries:

-FreeText "a0c73c16-a7e3-4564-9a95-2bdf47383716"
-FreeText "1b730954-1685-4b74-9bfd-dac224a7b894"
-FreeText "WinRM"

So in the single csv that all of those are put into, we see some rows containing the string WinRM. There are a lot of other rows that don’t have any of those three FreeText strings. If a row in the output does not contain WinRM, shouldn’t it contain one of the first two strings? None of them do.

Thanks for your assistance.

To reproduce

We ran the Sparrow tool unmodified

Expected behavior

In the PSLogin_Operations_Export, any rows that don't have the string WinRM should have one of the other two AppID strings that make up the PSLogin_Operations_Export, but those AppIDs (for Exchange Online PowerShell and PowerShell) are nowhere in the output.

Any helpful log output or screenshots

Paste the results here:

Add any screenshots of the problem here.

Zero KB CSV files after running script. Good or bad?

I ran this script successfully. It created some files on the desktop but they are ALL 0KB.

What does this mean when 0KB is the result?

SAMLToken_Operations_Export
PSMailbox_Operations_Export
Domain_Operations_Export
AppUpdate_Operations_Export
ServicePrincipal_Operations_Export
AppRoleAssignment_Operations_Export
Consent_Operations_Export

Is this a good outcome?

Also when it asked for a specific application to scan I said NO,
If i hit yes, it asks for input, but I do not know what parameters are valid.
Does this have any affect when selecting NO for that question?

Sparrow script doesn't work under a E3 sub, correct?

I know the documentation states that a E5 subscription is required in order to successfully run the script. Is there any other way to run it or is there less concern with organizations that have a subscription below E5?

What's the expected result we could have from this script?

For this script, we can't choose which Azure / M365 components to check, and we may always get errors while we don't have Exchange Online access to run the Search-UnifiedAuditLog command. And also we found that this command could not run in Azure China Cloud, so at this moment, we could only get errors after running this script.

image

In such case, please could you provide some sample to let us know what's the expected results we could have from below output csv, which could help us better determine how to leverage this script internally. Thanks.

image

The HTTP error (12175) is: A security error occurred .

For more information, see the about_Remote_Troubleshooting Help topic..

Have troubleshooted but unable to identify the issue - permissions are set and able to run other commands. gets to around 400 get-requests then errors

Import-PSSession : Running the Get-Command command in a remote session reported the following error: Processing data
for a remote command failed with the following error message: WS-Management cannot process the request. The operation
failed because of an HTTP error. The HTTP error (12175) is: A security error occurred . For more information, see the
about_Remote_Troubleshooting Help topic..
At C:\Program Files\WindowsPowerShell\Modules\ExchangeOnlineManagement\2.0.3\ExchangeOnlineManagement.psm1:435 char:40

  • ... oduleInfo = Import-PSSession $PSSession -AllowClobber -DisableNameChe ...
  •             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : InvalidResult: (:) [Import-PSSession], RuntimeException
    • FullyQualifiedErrorId : ErrorFromRemoteCommand,Microsoft.PowerShell.Commands.ImportPSSessionCommand

New-ExoPSSession : Access is denied

🐛 Summary

What's wrong? Please be specific.
New-ExoPSSession : Connecting to remote server outlook.office365.com failed with the following error message : Access
is denied. For more information, see the about_Remote_Troubleshooting Help topic.

To reproduce

Steps to reproduce the behavior:

  1. Log into Office 365 in another window, make sure your password works.
  2. Run script:
    PS C:\Users\winny> Invoke-WebRequest 'https://github.com/cisagov/Sparrow/raw/develop/Sparrow.ps1' -OutFile 'Sparrow.ps1' -UseBasicParsing; .\Sparrow.ps1 VERBOSE: Populating RepositorySourceLocation property for module ExchangeOnlineManagement.
    ...
    Choose your Exchange Environment [O365Default]:

The module allows access to all existing remote PowerShell (V1) cmdlets in addition to the 9 new, faster, and more reliable cmdlets.

|--------------------------------------------------------------------------|
| Old Cmdlets | New/Reliable/Faster Cmdlets |
|--------------------------------------------------------------------------|
| Get-CASMailbox | Get-EXOCASMailbox |
| Get-Mailbox | Get-EXOMailbox |
| Get-MailboxFolderPermission | Get-EXOMailboxFolderPermission |
| Get-MailboxFolderStatistics | Get-EXOMailboxFolderStatistics |
| Get-MailboxPermission | Get-EXOMailboxPermission |
| Get-MailboxStatistics | Get-EXOMailboxStatistics |
| Get-MobileDeviceStatistics | Get-EXOMobileDeviceStatistics |
| Get-Recipient | Get-EXORecipient |
| Get-RecipientPermission | Get-EXORecipientPermission |
|--------------------------------------------------------------------------|

To get additional information, run: Get-Help Connect-ExchangeOnline or check https://aka.ms/exops-docs

Send your product improvement suggestions and feedback to [email protected]. For issues related to the module, contact Microsoft support. Don't use the feedback alias for problems or support issues.

New-ExoPSSession : Connecting to remote server outlook.office365.com failed with the following error message : Access
is denied. For more information, see the about_Remote_Troubleshooting Help topic.
At C:\Program Files\WindowsPowerShell\Modules\ExchangeOnlineManagement\2.0.3\ExchangeOnlineManagement.psm1:426 char:30

  • ... PSSession = New-ExoPSSession -ExchangeEnvironmentName $ExchangeEnviro ...
  •             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : ResourceUnavailable: (:) [New-ExoPSSession], PSRemotingTransportException
    • FullyQualifiedErrorId : System.Management.Automation.Remoting.PSRemotingDataStructureException,Microsoft.Exchang
      e.Management.ExoPowershellSnapin.NewExoPSSession

PS C:\Users\winny>

Expected behavior

1 - return code success

Add any screenshots of the problem here.
image

Script returns nothing

Do you have to have E5 license in order for this script to work? We have E1 and the script does not create an output file. Nothing happens when finished.

Error when running the Script.

🐛 Summary

Hello,
Thank you for the script saving me a lot of time looking thru our environment.
I have the proper roles assigned and Auditing is turned on under security and compliance.

When I run the script I'm getting errors under these sections.
VERBOSE: Searching for 'Set domain authentication' and 'Set federation settings on domain' operations in the UAL.

The property 'Count' cannot be found on this object. Verify that the property exists.
At C:\Users\xxxxxx\Documents\Ps_scripts\solarwindsv1.ps1:331 char:13

  •     If ($UALInput.Count -eq 5000)
    
  •         ~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : NotSpecified: (:) [], PropertyNotFoundException
    • FullyQualifiedErrorId : PropertyNotFoundStrict

VERBOSE: Searching for 'Update application' and 'Update application ? Certificates and secrets management' in the UAL.
The property 'Organization' cannot be found on this object. Verify that the property exists.
At C:\Users\xxxxx\Documents\Ps_scripts\solarwindsv1.ps1:339 char:17

  •             $DataProps = [ordered]@{
    
  •             ~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : NotSpecified: (:) [], PropertyNotFoundException
    • FullyQualifiedErrorId : PropertyNotFoundStrict

The variable '$DataProps' cannot be retrieved because it has not been set.
At C:\Users\xxxxx\Documents\Ps_scripts\solarwindsv1.ps1:366 char:68

  • ... $DataObj = New-Object -TypeName PSObject -Property $DataProps

VERBOSE: Searching for 'Update service principal' and 'Add service principal credentials' in the UAL.
The property 'Organization' cannot be found on this object. Verify that the property exists.
At C:\Users\xxxxx\Documents\Ps_scripts\solarwindsv1.ps1:339 char:17

  •             $DataProps = [ordered]@{
    
  •             ~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : NotSpecified: (:) [], PropertyNotFoundException
    • FullyQualifiedErrorId : PropertyNotFoundStrict

The variable '$DataProps' cannot be retrieved because it has not been set.
At C:\Users\xxxxx\Documents\Ps_scripts\solarwindsv1.ps1:366 char:68

  • ... $DataObj = New-Object -TypeName PSObject -Property $DataProps

VERBOSE: Searching for 'Add OAuth2PermissionGrant' and 'Consent to application' in the UAL.
The property 'Organization' cannot be found on this object. Verify that the property exists.
At C:\Users\xxxxx\Documents\Ps_scripts\solarwindsv1.ps1:339 char:17

  •             $DataProps = [ordered]@{
    
  •             ~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : NotSpecified: (:) [], PropertyNotFoundException
    • FullyQualifiedErrorId : PropertyNotFoundStrict

The variable '$DataProps' cannot be retrieved because it has not been set.
At C:\Users\xxxxx\Documents\Ps_scripts\solarwindsv1.ps1:366 char:68

  • ... $DataObj = New-Object -TypeName PSObject -Property $DataProps

VERBOSE: Searching for PowerShell logins using known PS application ids in the UAL.
The property 'Organization' cannot be found on this object. Verify that the property exists.
At C:\Users\xxxxx\Documents\Ps_scripts\solarwindsv1.ps1:339 char:17

  •             $DataProps = [ordered]@{
    
  •             ~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : NotSpecified: (:) [], PropertyNotFoundException
    • FullyQualifiedErrorId : PropertyNotFoundStrict

The variable '$DataProps' cannot be retrieved because it has not been set.
At C:\Users\xxxxx\Documents\Ps_scripts\solarwindsv1.ps1:366 char:68

  • ... $DataObj = New-Object -TypeName PSObject -Property $DataProps

It creates 3 CSV files and the rest of the CSV files have zero bytes.

"An error occurred" Generic error when running this script.

Thank you for your work on this tool. I was pointed this direction from E-ISAC. I had no issues running the crowdstrike reporting tool, but sparrow is giving me trouble.

🐛 Summary

What's wrong? Please be specific.
Running sparrow.ps1 - I get prompted for azure cloud instance and exchange instance, I get prompted for username/password and MFA, it accepts and moves on listing the modules and asking if I have an E5 license, I respond yes. Would I like to investigate a certain application, no. It starts listing its verbose steps, getting to: "VERBOSE: Searching for PowerShell logins using known PS application ID's in the UAL. Then 'Warning, result set may have been truncated; narrow start/end date.'.

It prompts me for my credentials again, my MFA, I enter them and get this error:
"
An error occurred

An error occurred. Contact your administrator for more information.

Error details
•Activity ID: 987bf970-392b-4293-a91c-f2c891eb61a1
•Error time: Fri, 08 Jan 2021 18:11:08 GMT
•Cookie: enabled
•User agent string: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 10.0; Win64; x64; Trident/7.0; .NET4.0C; .NET4.0E)
"

To reproduce

Steps to reproduce the behavior:

  1. Listed in detail above

Expected behavior

I am expecting it to pass authentication like it does after the first request.

Any helpful log output or screenshots

Paste the results here:

An error occurred 

An error occurred. Contact your administrator for more information. 



Error details
•Activity ID: 987bf970-392b-4293-a91c-f2c891eb61a1
•Error time: Fri, 08 Jan 2021 18:11:08 GMT
•Cookie: enabled
•User agent string: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 10.0; Win64; x64; Trident/7.0; .NET4.0C; .NET4.0E)

Add any screenshots of the problem here.

Report Entries should be de-dupped and sorted

🚀 Feature Proposal

I noticed that there were duplicate entries per Unified log event in SAMLToken_Operations_Export.csv. These entries should be de-duplicated and sorted with | Sort-Object -Unique . If other reports contain duplicate values, then they should similarly be sorted and de-duplicated.

Motivation

It's overwhelming to look at a mess of data in rows of a .csv file. Let's try to make this as clean as possible.

Switch to ExchangeOnlineManagement

Please switch to using the ExchangeOnlineManagement module instead of CloudConnect for connecting to Exchange Online.

The ExchangeOnlineManagement module is the officially supported way to connect to Exchange Online. The actual implemention is just a few small code updates

Can Sparrow run without having office365 ?

We have Azure AD but no office 365 for now. Can we still run Sparrow as it keep error out when I sign into Azure as we do not have Exchange/Office 365 online enabled.

Audit Log search error

🐛 Summary

I receive the following error:
Audit Log search arguement startDate should be after 1/16 12:00 (current date and time)...

Could not load type 'Syste.Security.Cryptography.SHA256Cng'

Running sparrow command on Server 2012 R2 domain joined server from an acount with full Azure admin rights; Powershell 7.1, all needed modules installed, and getting this error:

PSerr2

Could not load type 'Syste.Security.Cryptography.SHA256Cng' from assembly 'System.core, Version=4.0.0.0, Culture=neutral, PublicKeyToken =

I have no idea idea what this means; found some references to it online from other users doing unrelated projects but no answer; not sure if this is PS malfunction or a permissions error. Any advice?

Azure Tenants without 0365 Email script failure

🐛 Summary

On domains with Azure tenants, that are not leveraging 0365 for email, there is not a 'null' choice for the 0365 option which causes the PowerShell Exchange module to not find the 0365 tenant and errors with:

New-ExoPSSession : AADSTS500014: The service principal for resource 'https://outlook.office365.com' is disabled. This indicate that a subscription within the tenant has lapsed, or that the administrator for this tenant has disabled the application, preventing tokens from being issued for it.

This occurs prior to being asked for the license type (of which there is no license type)

To reproduce

Steps to reproduce the behavior:

  1. Identify or create an Azure tenant without 0365 email.
  2. Create a user with adequate Azure permissions per instructions. No ability to create this user with required 0365 email permissions OR use an Azure Global Administrator (not recommended).
  3. Run .\sparrow.ps1 as Powershell Admin

Expected behavior

Hoping the expected behavior would allow me to skip the Exchange portion of the analysis but still provide analysis of anything found in Azure tenant.

Any helpful log output or screenshots

Paste the results here:

Add any screenshots of the problem here.
image

Exchange Online Access denied

I met below error while running this script, and would like to know do you have any guidance on how to run this script in steps.

New-ExoPSSession : Connecting to remote server partner.outlook.cn failed with the following error message : Access is
denied. For more information, see the about_Remote_Troubleshooting Help topic.
At C:\Program Files\WindowsPowerShell\Modules\ExchangeOnlineManagement\2.0.3\ExchangeOnlineManagement.psm1:426 char:30

  • ... PSSession = New-ExoPSSession -ExchangeEnvironmentName $ExchangeEnviro ...
  •             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : ResourceUnavailable: (:) [New-ExoPSSession], PSRemotingTransportException
    • FullyQualifiedErrorId : System.Management.Automation.Remoting.PSRemotingDataStructureException,Microsoft.Exchang
      e.Management.ExoPowershellSnapin.NewExoPSSession

Readme Update - Explicit Export Information

💡 Summary

Modify the readme file to include detailed information on where the Export CSV file will be saved to.

Motivation and context

The PS1 file does explain where the export is saved... however, it would be nice to spell out upfront where the export gets saved.
This would save a lot of time.

Implementation notes

Modify this line in the readme
The tool then outputs the data into multiple CSV files in a default directory.

Updated
The tool outputs the data into multiple CSV files that are located in the user's default home directory in a folder called 'ExportDir' (ie: Desktop/ExportDir).

Acceptance criteria

How do we know when this work is done?

  • Criterion

Add docs on how to setup/retreive credentials

💡 Summary

Explain what to do when prompted for InlineCredential.

Motivation and context

When running the script Sparrow.ps1in an online shell, the user gets prompted with the following output:

...
cmdlet New-ExoPSSession at command pipeline position 1
Supply values for the following parameters:
(Type !? for Help.)
InlineCredential:

Why does this work belong in this project? I think that many should check their environment ASAP and might get stuck without that information.

This would be useful because most people are lost in the Azure admin center anyway and maybe have no clue how to continue with a script.

Acceptance criteria

How do we know when this work is done?

  • Docs contain additional steps to set up things in an Azure online shell environment

Automated Execution (Scheduled Task)

Is it possible to execute this script as a scheduled task or is that not recommended? I noticed when running for the first time I am prompted for input so perhaps a scheduled task is not possible. If that is the case, does anyone have any suggestions for automating the execution via an alternate method?

Thanks

Unable to get the SaveAs property of the Workbook class

Unable to get the SaveAs property of the Workbook class
At C:\support\Sparrow.ps1:113 char:5

  • $Workbook.SaveAs((Join-Path $ExportDir 'Summary_Export.xlsx'))
    
  • ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : OperationStopped: (:) [], COMException
    • FullyQualifiedErrorId : System.Runtime.InteropServices.COMException

Add usgov environment to Connect-AzureAD

Just looking quickly on the mobile site and it looks like there isn't an option for setting the 'Connect-AzureAD -AzureEnvironmentName AzureUSGovernment' option. Otherwise looks great and will check it out!

$ExportDir not used for export

Not sure if it's been mentioned yet. I haven't run the script yet as I'm waiting on the permissions to be provisioned, but looking over the script I noticed the $ExportDir variable that can be changed to wherever the user prefers their exports/output to go, but it seems if a user were to want to change this, it wouldn't have any effect:

Line 22:
$ExportDir = "$home\Desktop\ExportDir"
If (!(Test-Path $ExportDir)){
New-Item -Path $ExportDir -ItemType "Directory" -Force
}

Line 358
If ($AppendType -eq "Append"){
$DataArr | Export-csv $home\Desktop\ExportDir$CsvName.csv -NoTypeInformation -Append
} Else {
$DataArr | Export-csv $home\Desktop\ExportDir$CsvName.csv -NoTypeInformation
}

I'd imagine you would want this to be something more like:

    If ($AppendType -eq "Append"){
        $DataArr | Export-csv $ExportDir\$CsvName.csv -NoTypeInformation -Append
    } Else {
        $DataArr | Export-csv $ExportDir\$CsvName.csv -NoTypeInformation
    }

Script fails to fetch audit logs

Audit log search argument startDate should be after 1/4/2020 1:02:22 PM UTC.
    + CategoryInfo          : InvalidArgument: (:) [Search-UnifiedAuditLog], ArgumentException
    + FullyQualifiedErrorId : [Server=BL0PR12MB4674,RequestId=e9594a36-7cf9-4f8f-b740-6db2782716ec,TimeStamp=1/4/2021
   1:02:22 PM] [FailureCategory=Cmdlet-ArgumentException] F788A15A,Microsoft.Exchange.Management.SystemConfigurationT
  asks.SearchUnifiedAuditLog
    + PSComputerName        : outlook.office365.com

It's trying to get more than 1 year of logs, which will never work.

Unable to get the SaveAs property of the Workbook class

🐛 Summary

At \Sparrow.ps1:113 char:5

  • $Workbook.SaveAs((Join-Path $ExportDir 'Summary_Export.xlsx'))
    
  • ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : OperationStopped: (:) [], COMException
    • FullyQualifiedErrorId : System.Runtime.InteropServices.COMException

Comment Doesn't Match Actual search string

🐛 Bug Report

The echo to the user mentions search for three different operations , but the parameter passed to the search function only uses one.

Write-Verbose "Searching for 'Add app role assignment to service principal', 'Add app role assignment grant to user', and 'Add app role assignment to group' in the UAL."

$AppRoleData = Search-UnifiedAuditLog -StartDate $StartDate -EndDate $EndDate -RecordType AzureActiveDirectory -Operations "Add app role assignment" 

What exactly is the expected powershell output?

Do you have an Office 365/Microsoft 365 E5/G5 license? Y/N: Y
Would you like to investigate a certain application? Y/N: N
Skipping AppID investigation

VERBOSE: Searching for 'Set domain authentication' and 'Set federation settings on domain' operations in the UAL.
VERBOSE: Searching for 'Update application' and 'Update application ? Certificates and secrets management' in the UAL.
VERBOSE: Searching for 'Update service principal' and 'Add service principal credentials' in the UAL.
VERBOSE: Searching for 'Add app role assignment to service principal', 'Add app role assignment grant to user', and 'Add app role assignment to group' in the UAL.
VERBOSE: Searching for 'Add OAuth2PermissionGrant' and 'Consent to application' in the UAL.
VERBOSE: Searching for 16457 in UserLoggedIn and UserLoginFailed operations in the UAL.
VERBOSE: Searching for PowerShell logins into mailboxes in the UAL.
VERBOSE: Searching for PowerShell logins using known PS application ids in the UAL.

Once I make the selections above, all I receive are details of my personal account using the headers detailed below:
Account Environment TenantId TenantDomain AccountType


File gives error for line 156

🐛 Bug Report

Line |
156 | … etails px-3 px-md-4 px-lg-5 flex-wrap flex-md-nowrap" role="banner" >
| ~
| Missing file specification after redirection operator.

To Reproduce

Steps to reproduce the behavior:

Run the Sparraw.ps1 file

Expected behavior

File was supposed to run

Any helpful log output

Line |
156 | … etails px-3 px-md-4 px-lg-5 flex-wrap flex-md-nowrap" role="banner" >
| ~
| Missing file specification after redirection operator.

"Choose Your Environment" comes up blank if AzureADPreview is installed instead of AzureAD module

🐛 Summary

What's wrong? The script fails to install AzureAD module naturally since the AzureADPreview contains the same cmdlets, then the script tries to import the AzureAD module and failts and has no logic to import AzureADPreview instead if present.

To reproduce

Steps to reproduce the behavior:

  1. Have AzureADPreview installed instead of AzureAD
  2. Run the tool

Expected behavior

Load the correct AzureADPreview module

Update PowerShell module load

🚀 Feature Proposal

PowerShell modules on Windows 10 do not load properly the first run and PowerShell needs to be run as administrator for those modules to install properly.

Motivation

Cleaner way to get this going and run the script direct from the raw Github content for security.

Example

Install-Module CloudConnect -AllowClobber
Install-Module AzureAD -AllowClobber
Install-Module MSOnline -AllowClobber

Import-Module CloudConnect -Global
Import-Module AzureAD -Global
Import-Module MSOnline -Global

Set-Location $env:USERPROFILE\desktop
$SparrowGitHub = Invoke-WebRequest "https://raw.githubusercontent.com/cisagov/Sparrow/develop/Sparrow.ps1"
Invoke-Expression $($SparrowGitHub.Content)

Pitch

Why does this feature belong in this project?

Not loading required modules

🐛 Bug Report

Running the script out of C:\Scripts on a Windows 2012 R2 server. We have a local AD domain sync'd with AzureAD, and I'm logged into local server with a Domain Admin account. Get the error below for the CloudConnect, AzureAD, and MSOnline modules...

Import-Module : The specified module 'CloudConnect' was not loaded because no valid module file was found in any
module directory.
At C:\scripts\Sparrow.ps1:13 char:13

  •         Import-Module -Name $ReqModule
    
  •         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : ResourceUnavailable: (CloudConnect:String) [Import-Module], FileNotFoundException
    • FullyQualifiedErrorId : Modules_ModuleNotFound,Microsoft.PowerShell.Commands.ImportModuleCommand

Output of $env:PSModulePath is shown below...
PS C:\scripts> $env:PSModulePath
C:\Users\netfrank.KISH\Documents\WindowsPowerShell\Modules;C:\Program Files\WindowsPowerShell\Modules;C:\Windows\system32\WindowsPowerShell\v1.0\Modules;C:\Program Files\Microsoft Azure AD Sync\Bin;C:\Program Files\Microsoft Azure AD Connect Health Sync Agent\PowerShell\

WorkloadType "AAD" does not include ApplicationId property

WorkloadType "AAD" does not include the ApplicationId property, though the value is returned within ($_.AuditData | ConvertFrom-Json).ApplicationId

I noticed that the value was missing from the SAMLToken_Operations_Export.csv export.

Issues found after executing script

At C:\Admin\Crowdstrike Tool\Get-CRTReport.ps1:168 char:21

  •             Sign up
    
  •                 ~
    

The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it as part of a string.
At C:\Admin\Crowdstrike Tool\Get-CRTReport.ps1:195 char:190

  • ... ata-ga-click="(Logged out) Header, go to Features">Features <span cla ...
  •                                                             ~
    

The '<' operator is reserved for future use.
At C:\Admin\Crowdstrike Tool\Get-CRTReport.ps1:195 char:261

  • ... ="Bump-link-symbol float-right text-normal text-gray-light">→</s ...
  •                                                             ~
    

The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it as part of a string.
At C:\Admin\Crowdstrike Tool\Get-CRTReport.ps1:209 char:255

  • ... ogged out) Header, go to Customer stories">Customer stories <span cla ...
  •                                                             ~
    

The '<' operator is reserved for future use.
At C:\Admin\Crowdstrike Tool\Get-CRTReport.ps1:209 char:326

  • ... ="Bump-link-symbol float-right text-normal text-gray-light">→</s ...
  •                                                             ~
    

The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it as part of a string.
At C:\Admin\Crowdstrike Tool\Get-CRTReport.ps1:210 char:231

  • ... ata-ga-click="(Logged out) Header, go to Security">Security <span cla ...
  •                                                             ~
    

The '<' operator is reserved for future use.
At C:\Admin\Crowdstrike Tool\Get-CRTReport.ps1:210 char:302

  • ... ="Bump-link-symbol float-right text-normal text-gray-light">→</s ...
  •                                                             ~
    

The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it as part of a string.
At C:\Admin\Crowdstrike Tool\Get-CRTReport.ps1:233 char:222

  • ... a-click="(Logged out) Header, go to Explore">Explore GitHub <span cla ...
  •                                                             ~
    

The '<' operator is reserved for future use.
At C:\Admin\Crowdstrike Tool\Get-CRTReport.ps1:233 char:293

  • ... ="Bump-link-symbol float-right text-normal text-gray-light">→</s ...
  •                                                             ~
    

The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it as part of a string.
At C:\Admin\Crowdstrike Tool\Get-CRTReport.ps1:236 char:107

  • ... text-normal text-mono f5 mb-2 border-lg-top pt-lg-3">Learn & con ...
  •                                                             ~
    

The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it as part of a string.
Not all parse errors were reported. Correct the reported errors and try again.
+ CategoryInfo : ParserError: (:) [], ParseException
+ FullyQualifiedErrorId : AmpersandNotAllowed

Not Getting output to CSV when Sparrow is run

🐛 Summary

I have loaded all the modules and then I run Sparrow as specified, the script runs and asks for credentials multiple times, after that the script continues and ultimately finishes but I don't get any CVS files, it almost seems that nothing happened. Note: on the licensing question I select No to E5 and No to the app question.

To reproduce

Steps to reproduce the behavior:

  1. Just simply run the script

Expected behavior

Did not get a CSV output

Any helpful log output or screenshots

Paste the results here:

Add any screenshots of the problem here.

Choose your Azure Environment prompt does not proceed

🐛 Bug Report

The script prompts to choose my Azure environment, but does not proceed when entering "AzureCloud", tenant ID or any other options.

To Reproduce

Steps to reproduce the behavior:

The script ran until loading "Microsoft.Online.Identity.Federation.PowerShell.dll" which failed. I have Security Admin role in AzureAD.

Expected behavior

Readme did not list acceptable input.

Any helpful log output

Paste the results here:

Change parameter value

🚀 Feature Proposal

In Export-UALData function, change "Sharepoint" value of -WorkloadType parameter to "SharePoint"

  • function call/parameter passing on line 143
  • use of parameter value on line 324

Motivation

"Sharepoint" is an incorrect capitalization of the service. A PR was raised to fix this in the comments (#13), but the repo owners should make sure any parameter value changes have no additional effect.

Example

Minor typographical change.

Pitch

This is only to correctly identify the queried service in Microsoft M365.

cmdlet error

🐛 Summary

I am getting the following erros

Do you have an Office 365/Microsoft 365 E5/G5 license? Y/N: N
Would you like to investigate a certain application? Y/N: N
Skipping AppID investigation

VERBOSE: Searching for 'Set domain authentication' and 'Set federation settings on domain' operations in the
UAL.
Search-UnifiedAuditLog : The term 'Search-UnifiedAuditLog' is not recognized as the name of a cmdlet,
function, script file, or operable program. Check the spelling of the name, or if a path was included,
verify that the path is correct and try again.
At C:\Windows\system32\Sparrow.ps1:154 char:19

  • $DomainData = Search-UnifiedAuditLog -StartDate $StartDate -EndDa ...
    
  •               ~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : ObjectNotFound: (Search-UnifiedAuditLog:String) [], CommandNotFoundException
    • FullyQualifiedErrorId : CommandNotFoundException

VERBOSE: Searching for 'Update application' and 'Update application ? Certificates and secrets management'
in the UAL.
Search-UnifiedAuditLog : The term 'Search-UnifiedAuditLog' is not recognized as the name of a cmdlet,
function, script file, or operable program. Check the spelling of the name, or if a path was included,
verify that the path is correct and try again.
At C:\Windows\system32\Sparrow.ps1:161 char:16

  • $AppData = Search-UnifiedAuditLog -StartDate $StartDate -EndDate  ...
    
  •            ~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : ObjectNotFound: (Search-UnifiedAuditLog:String) [], CommandNotFoundException
    • FullyQualifiedErrorId : CommandNotFoundException

Feature: DeviceCode Login Flag

💡 Summary

Ability to use -DeviceCode for logins

What is the work, as a high-level summary?
Add flag to use -DeviceCode for logins

Motivation

Why does this work belong in this project?
Some orgs use SSO but have vaulted account this can cause login to automatically use SSO login even if another account is entered in the login Window.

This would be useful because...

Implementation notes

Please provide details for implementation, such as:

  • an example for how this would be used
    Sparrow.ps1 -DeviceCode

Acceptance criteria

How do we know when this work is done?

  • Criterion

Invalid characters

🐛 Summary

I am getting invalid character errors when trying to run the script. Windows Powershell gives me more errors than PowerShell 7 does, but both fail to successfully run the script.

To reproduce

Steps to reproduce the behavior:

  1. Open Windows PowerShell or PowerShell 7
  2. Execute Sparrow.ps1

Expected behavior

I would expect the script to execute as described.

Any helpful log output or screenshots

Windows PowerShell

PS C:\Users\redacted\Downloads> .\Sparrow.ps1
At C:\Users\redacted\Downloads\Sparrow.ps1:167 char:21
+                 Sign&nbsp;up
+                     ~
The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it as part of a string.
At C:\Users\redacted\Downloads\Sparrow.ps1:194 char:190
+ ... ata-ga-click="(Logged out) Header, go to Features">Features <span cla ...
+                                                                 ~
The '<' operator is reserved for future use.
At C:\Users\redacted\Downloads\Sparrow.ps1:194 char:261
+ ... ="Bump-link-symbol float-right text-normal text-gray-light">&rarr;</s ...
+                                                                 ~
The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it as part of a string.
At C:\Users\redacted\Downloads\Sparrow.ps1:208 char:255
+ ... ogged out) Header, go to Customer stories">Customer stories <span cla ...
+                                                                 ~
The '<' operator is reserved for future use.
At C:\Users\redacted\Downloads\Sparrow.ps1:208 char:326
+ ... ="Bump-link-symbol float-right text-normal text-gray-light">&rarr;</s ...
+                                                                 ~
The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it as part of a string.
At C:\Users\redacted\Downloads\Sparrow.ps1:209 char:231
+ ... ata-ga-click="(Logged out) Header, go to Security">Security <span cla ...
+                                                                 ~
The '<' operator is reserved for future use.
At C:\Users\redacted\Downloads\Sparrow.ps1:209 char:302
+ ... ="Bump-link-symbol float-right text-normal text-gray-light">&rarr;</s ...
+                                                                 ~
The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it as part of a string.
At C:\Users\redacted\Downloads\Sparrow.ps1:232 char:222
+ ... a-click="(Logged out) Header, go to Explore">Explore GitHub <span cla ...
+                                                                 ~
The '<' operator is reserved for future use.
At C:\Users\redacted\Downloads\Sparrow.ps1:232 char:293
+ ... ="Bump-link-symbol float-right text-normal text-gray-light">&rarr;</s ...
+                                                                 ~
The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it as part of a string.
At C:\Users\redacted\Downloads\Sparrow.ps1:235 char:107
+ ...  text-normal text-mono f5 mb-2 border-lg-top pt-lg-3">Learn &amp; con ...
+                                                                 ~
The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it as part of a string.
Not all parse errors were reported.  Correct the reported errors and try again.
    + CategoryInfo          : ParserError: (:) [], ParseException
    + FullyQualifiedErrorId : AmpersandNotAllowed

PowerShell 7

ParserError: C:\Users\redacted\Downloads\Sparrow.ps1:194
Line |
 194 |  … ata-ga-click="(Logged out) Header, go to Features">Features <span cla …
     |                                                                ~
     | The '<' operator is reserved for future use.

Windows PowerShell
image

PowerShell 7
image

Getting an I/O operation error

🐛 Summary

I am getting an I/O operation error on running the script. This is after we input our choice of Azure and O365 environments. Prior to getting this error, I had just fixed an issue with TLS 1.2 communications.

Starting a command on the remote server failed with the following error message : The I/O operation has been aborted because of
either a thread exit or an application request. For more information, see the about_Remote_Troubleshooting Help topic.
+ CategoryInfo : OperationStopped: (outlook.office365.com:String) [], PSRemotingTransportException
+ FullyQualifiedErrorId : JobFailure
+ PSComputerName : outlook.office365.com

cisasparrowerr

Unable to find type error

🐛 Summary

when running sparrow.ps1 after issuing connect-exchangeonline, we receive the below error:

Unable to find type [Microsoft.Open.Azure.AD.CommonLibrary.AzureEnvironment].
At C:\WINDOWS\system32\Sparrow.ps1:54 char:26

  • ... ironments = [Microsoft.Open.Azure.AD.CommonLibrary.AzureEnvironment]: ...
  •             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : InvalidOperation: (Microsoft.Open....zureEnvironment:TypeName) [], RuntimeException
    • FullyQualifiedErrorId : TypeNotFound

Azure Environments

Choose your Azure Environment [AzureCloud]:

To reproduce

in powershell execute:
Connect-ExchangeOnline
login in with tenant admin that has required permissions
run: Invoke-WebRequest 'https://github.com/cisagov/Sparrow/raw/develop/Sparrow.ps1' -OutFile 'Sparrow.ps1' -UseBasicParsing; .\Sparrow.ps1

error message as above

Expected behavior

What did you expect to happen that didn't?

Any helpful log output or screenshots

Paste the results here:

Add any screenshots of the problem here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.