microsoft/peek

Azure Usage/Billing Analytics Solution for EAs, CSPs and Direct Subscriptions.

Customers are continuing to expand their footprint in Azure and there is a significant need to understand resource usage, optimize it as far as possible and plan for future growth. The Azure Usage/Billing Analytics is a one stop PaaS based solution which allows us to abstract the intricacies of collecting the usage data for different use cases (Enterprise Agreement, CSP and Direct Azure Subscriptions) and provides detailed visuals for near real time data analytics. The Power BI based report samples can be used to create multiple dashboards focused on EA subscribers, CSPs as well as CSP's customers to allow them to view their current usage, billing history and much more. This solution is intended as a Proof-of-Concept sample implementation. [01/18]

Sample Screenshot For more sample reports refer to sample output screenshots

Getting Started

Welcome! These instructions will help you to understand the solution and set up the project components for cloud deployment and data visualization.

Prerequisites

The set of softwares and subscriptions for usage and deployment of the project are mentioned in the requirements document.

Architecture

Refer to the solution architecture guide.

Deployment

Refer to the deployment guide.

Built With

• [.NET] (https://www.microsoft.com/net)
• [Microsoft Transact-SQL] (https://www.microsoft.com/en-in/sql-server/sql-server-2016)
• Microsoft Azure
• Power BI

Contributions

Please refer to CONTRIBUTING.md

License

This project has adopted the [Microsoft Open Source Code of Conduct] (https://opensource.microsoft.com/codeofconduct/). For more information, see the [Code of Conduct FAQ] (https://opensource.microsoft.com/codeofconduct/faq/) or contact [[email protected]] (mailto:[email protected]) with any additional questions or comments.

The MIT License (MIT)

Copyright (c) 2016 Microsoft

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

peek's Issues

Azure peek dashboard - new customers not appearing

I have notist one issue at the Azure peek dashboard
New customers (less than a month) with Azure subscriptions are not appearing in the Dashboard. Nor in the azure sql database as well
New subscriptions for old customers are appearing at the dashboard

For these new customers (less than a month), I checked and see money consumption in the CSP Portal
May be the Azure CSP reflect these new customers for the Peek only after a true billing accrue? At the beginning of the month?

10x

Asking for Azure subscription

Hi ,

I ' ve created the Azurebilling Analytics project in visual studio -2015 -enterprise in my laptop , build was successful , was trying to publish the project through webAPI and just after that it is looking for credentials and i 've put the active azure subscription , but in next drop down it is asking for Azure subscription and nothing coming in the drop down. Any insight please .

please let me know, if you are looking for any more information.

Looking forward for the update.

Thanks,

Saurabh

How to get the daily consumption

Hi ,

Can you please help us to know , how we can get the daily consumption which will be of current day or few days back , any process elaboration will be helpful ?

Thanks ,

Saurabh

Timeout for CspUtilization

Hello, I'm currently using the implimentUtilization branch for my deployment and I am unable to retrieve the CspUtilization data, critically the resource tags which are my entire requirement for setting up Peek!

Whilst checking my API manually on the /swagger page I can retrieve results on each GET, with the exception of the CSPUtilization GET which eventually reports back:

<title>500 - The request timed out.</title>

500 - The request timed out.

The web server failed to respond within the specified time.

Is this a known issue, or something limited to my deployment/implementation?

I'd appreciate any advice or remedy.

Regards,

Webjob returning exception

Hi,

This is linked to issue 33 , where we are in CSP partner is Cloud re seller.

so in swagger we are getting the Response

Response code 200

/api/cspbilling : - Returning back response 200 , but nothing in response body
/api/cspbilling/currentmonth : - Returning back response 200 , but nothing in response body
/api/cspusage :- Response 500.
/api/userbilling :- Response 500.

Error_details.docx

https://online.swagger.io/validator/debug?url=https://deployment-demo-app-service.azurewebsites.net:443/swagger/docs/v1

{}

is showing valid URL

Further moving on Section 4 (Web jobs)

While running the Webjobs , it is giving me back

[02/21/2017 20:59:18 > 622409: SYS INFO] Status changed to Starting
[02/21/2017 20:59:18 > 622409: SYS WARN] 'Always On' doesn't appear to be enabled for this Web App. To ensure your continuous job doesn't stop running when the SCM host is idle for too long, consider enabling 'Always On' in the configuration settings for your Web App. Note: 'Always On' is available only in Basic, Standard and Premium modes.

[02/21/2017 22:03:58 > 622409: INFO]
[02/21/2017 22:03:58 > 622409: INFO] Error encountered:
[02/21/2017 22:03:58 > 622409: INFO] One or more errors occurred.
[02/21/2017 22:03:58 > 622409: INFO] InnerException :
[02/21/2017 22:03:58 > 622409: INFO] Exception of type 'Microsoft.Rest.HttpOperationException1[System.Object]' was thrown. [02/21/2017 22:03:58 > 622409: INFO] StackTrace : [02/21/2017 22:03:58 > 622409: INFO] at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) [02/21/2017 22:03:58 > 622409: INFO] at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification)
[02/21/2017 22:03:58 > 622409: INFO] at System.Threading.Tasks.Task`1.get_Result()
[02/21/2017 22:03:58 > 622409: INFO] at BillingWebJob.Helpers.CspDataHelper.StartCspRoutine(Int32& status, Int32& totalRecordsCount, String& blobStorageUri)
[02/21/2017 22:03:58 > 622409: INFO] at BillingWebJob.Program.CronJob(TimerInfo timer)
[02/21/2017 22:03:58 > 622409: INFO]
[02/21/2017 22:03:58 > 622409: INFO] Adding audit information in AuditData table..
[02/21/2017 22:03:58 > 622409: INFO]
[02/21/2017 22:04:05 > 622409: INFO]
[02/21/2017 22:04:05 > 622409: INFO] Operation complete. Exiting this run.
[02/21/2017 22:04:05 > 622409: INFO]
[02/21/2017 22:04:05 > 622409: INFO] Executed: 'Program.CronJob' (Succeeded)
[02/21/2017 22:04:05 > 622409: INFO] Job host started

CSP Usage webjob throwing exception Microsoft.Rest.HttpOperationException

Hello Everyone,
I am trying to deploy Peek in our environment and use it to abstract the usage data for different use cases like Enterprise Agreement, CSP. EA is working fine for us, however, we are facing issue while collecting the data for CSP. I have tried to increase the timeout but no luck. We are using the latest merge.

Suggestions to troubleshoot the problem are highly appreciated.

The log output is as follows:
[04/30/2017 09:01:00 > a84fe7: INFO] Starting job processing for Customer Type csp
[04/30/2017 09:01:00 > a84fe7: INFO] Starting CSP Routine. Current Usage, Historic Usage and Historic Billing data will be updated..
[04/30/2017 09:01:00 > a84fe7: INFO]
[04/30/2017 09:01:00 > a84fe7: INFO] Fetching records for Current Month's Usage from the API. This may take a while. If this operation is timimg out, consider increasing the TimeOut limit in Configuration file.
[04/30/2017 09:01:00 > a84fe7: INFO]
[04/30/2017 09:01:00 > a84fe7: INFO] Error encountered:
[04/30/2017 09:01:00 > a84fe7: INFO] One or more errors occurred.
[04/30/2017 09:01:00 > a84fe7: INFO] InnerException :
[04/30/2017 09:01:00 > a84fe7: INFO] **Exception of type 'Microsoft.Rest.HttpOperationException1[System.Object]' was thrown.** [04/30/2017 09:01:00 > a84fe7: INFO] StackTrace : [04/30/2017 09:01:00 > a84fe7: INFO] at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) [04/30/2017 09:01:00 > a84fe7: INFO] at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification)
[04/30/2017 09:01:00 > a84fe7: INFO] at System.Threading.Tasks.Task`1.get_Result()
[04/30/2017 09:01:00 > a84fe7: INFO] at BillingWebJob.Helpers.CspDataHelper.StartCspRoutine(Int32& status, Int32& totalRecordsCount, String& blobStorageUri)
[04/30/2017 09:01:00 > a84fe7: INFO] at BillingWebJob.Program.CronJob(TimerInfo timer)_

Thanks,
Sheel

EA usage error: The wait operation timed out

Hi, i'm experiencing an error while fetching the EA logs. CSP logs are working fine. The swagger API is also working for both EA/CSP. I already deployed the ASYNC version to check if this solves anything at all, but unfortunately it does not.

[05/09/2017 14:56:51 > 94928d: INFO] Error encountered:
[05/09/2017 14:56:51 > 94928d: INFO] An error occurred while executing the command definition. See the inner exception for details.
[05/09/2017 14:56:51 > 94928d: INFO] InnerException :
[05/09/2017 14:56:51 > 94928d: INFO] Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
[05/09/2017 14:56:51 > 94928d: INFO] InnerException :
[05/09/2017 14:56:51 > 94928d: INFO] The wait operation timed out
[05/09/2017 14:56:51 > 94928d: INFO] StackTrace :
[05/09/2017 14:56:51 > 94928d: INFO] at System.Data.Entity.Core.EntityClient.Internal.EntityCommandDefinition.ExecuteStoreCommands(EntityCommand entityCommand, CommandBehavior behavior)
[05/09/2017 14:56:51 > 94928d: INFO] at System.Data.Entity.Core.Objects.Internal.ObjectQueryExecutionPlan.Execute[TResultType](ObjectContext context, ObjectParameterCollection parameterValues)
[05/09/2017 14:56:51 > 94928d: INFO] at System.Data.Entity.Core.Objects.ObjectQuery1.<>c__DisplayClass7.<GetResults>b__6() [05/09/2017 14:56:51 > 94928d: INFO] at System.Data.Entity.Core.Objects.ObjectContext.ExecuteInTransaction[T](Func1 func, IDbExecutionStrategy executionStrategy, Boolean startLocalTransaction, Boolean releaseConnectionOnSuccess)
[05/09/2017 14:56:51 > 94928d: INFO] at System.Data.Entity.Core.Objects.ObjectQuery1.<>c__DisplayClass7.<GetResults>b__5() [05/09/2017 14:56:51 > 94928d: INFO] at System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.Execute[TResult](Func1 operation)
[05/09/2017 14:56:51 > 94928d: INFO] at System.Data.Entity.Core.Objects.ObjectQuery1.GetResults(Nullable1 forMergeOption)
[05/09/2017 14:56:51 > 94928d: INFO] at System.Data.Entity.Core.Objects.ObjectQuery1.<System.Collections.Generic.IEnumerable<T>.GetEnumerator>b__0() [05/09/2017 14:56:51 > 94928d: INFO] at System.Data.Entity.Internal.LazyEnumerator1.MoveNext()
[05/09/2017 14:56:51 > 94928d: INFO] at System.Collections.Generic.List1..ctor(IEnumerable1 collection)
[05/09/2017 14:56:51 > 94928d: INFO] at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
[05/09/2017 14:56:51 > 94928d: INFO] at BillingWebJob.Helpers.EaDataHelper.GetFilteredEaBillingDataFromApi(String date) in C:\Users\mhkbc67\stack\Work\Realdolmen\Internal projects\Peek\peek-master\AzureBillingAnalytics\BillingWebJob\Helpers\EaDataHelper.cs:line 216
[05/09/2017 14:56:51 > 94928d: INFO] at BillingWebJob.Helpers.EaDataHelper.StartEaRoutine(Int32& status, Int32& totalRecordsCount, String& blobStorageUri) in C:\Users\mhkbc67\stack\Work\Realdolmen\Internal projects\Peek\peek-master\AzureBillingAnalytics\BillingWebJob\Helpers\EaDataHelper.cs:line 79
[05/09/2017 14:56:51 > 94928d: INFO] at BillingWebJob.Program.CronJob(TimerInfo timer) in C:\Users\mhkbc67\stack\Work\Realdolmen\Internal projects\Peek\peek-master\AzureBillingAnalytics\BillingWebJob\Program.cs:line 140

"Message": "An error has occurred." for CSP Usage api

both for my two CSP portals

Peek vs. AUBI

Greetings,

As a CSP, the company I work for is excited by the possibilities of Peek. We have also been introduced recently to a similar Microsoft project on GitHub, Azure Usage and Billing Insights (AUBI) Portal - https://github.com/Microsoft/AzureUsageAndBillingPortal.

I am interested in knowing if the Peek programmers are familiar with AUBI and how they view it relative to Peek.

Thanks - Mike

Unable to load AzureBillingAnalytics in Visual Studio Professional

Hey there,

I'm trying to deploy Peek in our environment since the source code for AUBI is so broken that I can't get it to work. That being said, when I try to load the project into Visual Studio I get the following error:
"We were unable to automatically populate your Visual Studio Team Services accounts.
The following error was encountered: TF400813: Resource not available for anonymous access. Client authentication required."

Alternatively, when I moved the Peek-Master folder to a different location on my PC, I received this error when attempting to open it:
"The mappings for the solution could not be found.
The active solution has been temporarily disconnected from source control because the server is unavailable. To attempt to reconnect to source control, close and then re-open the solution when the server is available. If you want to connect this solution to another server, use the Change Source Control dialog."

I don't work in Visual Studio typically so if these are simple issues I apologies for not knowing how to resolve this, but if you could help me I would really like to deploy this in my environment.

Error running webjob for csp data. database not being populated

This is the output of the BillingWebJob.exe
It is not populating the database with any information
Please help

[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO] Adding audit information in AuditData table..
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO] Job started for fetching billing data at 05/23/2019 09:00:02
[05/23/2019 09:00:02 > fca8fa: INFO] Starting job processing for Customer Type csp
[05/23/2019 09:00:02 > fca8fa: INFO] Starting CSP Routine. Current Usage, Historic Usage and Historic Billing data will be updated..
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO] Fetching records for Current Month's Usage from the API. This may take a while. If this operation is timimg out, consider increasing the TimeOut limit in Configuration file.
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO] Error encountered:
[05/23/2019 09:00:02 > fca8fa: INFO] One or more errors occurred.
[05/23/2019 09:00:02 > fca8fa: INFO] InnerException :
[05/23/2019 09:00:02 > fca8fa: INFO] Operation returned an invalid status code 'Unauthorized'
[05/23/2019 09:00:02 > fca8fa: INFO] StackTrace :
[05/23/2019 09:00:02 > fca8fa: INFO] at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
[05/23/2019 09:00:02 > fca8fa: INFO] at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification) [05/23/2019 09:00:02 > fca8fa: INFO] at System.Threading.Tasks.Task1.get_Result()
[05/23/2019 09:00:02 > fca8fa: INFO] at BillingWebJob.Helpers.CspDataHelper.StartCspRoutine(Int32& status, Int32& totalRecordsCount, String& blobStorageUri) in C:\Users\USERNAME\Desktop\peek-masterog\peek-master\AzureBillingAnalytics\BillingWebJob\Helpers\CspDataHelper.cs:line 534
[05/23/2019 09:00:02 > fca8fa: INFO] at BillingWebJob.Program.CronJob(TimerInfo timer) in C:\Users\USERNAME\Desktop\peek-masterog\peek-master\AzureBillingAnalytics\BillingWebJob\Program.cs:line 122
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO] Adding audit information in AuditData table..
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO] Operation complete. Exiting this run.
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO] Executed 'Program.CronJob' (Succeeded, Id=ad8b7ec0-a4be-4ddd-a652-0370be6acba9)

doesn't work with vs 2019

is there any updated version of this?

Filter data on basis of customer

Hi ,

Here we want to fetch the data for one client/customer , i think we have 2 option , please correct if i'm wrong

a. Pull all the data for all customer from CSP site and put the data in SQL Azure database
as we have 3 tables ( CspBillingData, CspSummaryData and CspUsageData ) for csp and we want to pull the data for 1 customer about the daily billing ,usage and consumption, how we can achieve , can you guide please ?

b. If possible, can we filter down the data from web.config (BillingdataAPI) while passing down the information to SQLAzure database based on 1 customer/client.

Can you provide some light on above please.

Thanks,

Saurabh

How we can fetch data SQL Azure database based on client/customer or any prior filter possible in the web.config of BillingdataAPI so the data stored in SQL AZURE database based on one client only

EA - Error when getting data

Hi,
I have successfully deployed the application and it start collecting data, however, when it is looping through the monthly report for the EA, it gets to 2015-11 and throws the following error:

One or more errors occurred.
InnerException :
Exception of type 'Microsoft.Rest.HttpOperationException1[System.Object]' was thrown. StackTrace : at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification)
at System.Threading.Tasks.Task`1.get_Result()
at BillingWebJob.Helpers.EaDataHelper.GetFilteredEaBillingDataFromApi(String date)
at BillingWebJob.Helpers.EaDataHelper.StartEaRoutine(Int32& status, Int32& totalRecordsCount, String& blobStorageUri)
at BillingWebJob.Program.CronJob(TimerInfo timer)

Any idea how to get around `this?```

Display Tags of ressources

Dear All

Is it planned to get a overview which ressources have which tags assigned?

Thanks
P

get a timeout for (api) myapp/api/cspusage

support for MFA requirement

as it been updated by Microsoft in last month
we will be required to use MFA for any app running over partner center
will this app will be supported also ?

No Issue

For you information your app registration must has:
• allowPublicClient: true
• oauth2AllowImplicitFlow: true
Also, the code is old with old libraries check newer sample:
https://github.com/Microsoft/Partner-Center-DotNet-Samples

ResourceUtilizationList.json is missing

While loading the utilization records from the Partner Center SDK, the following error was displayed in our logs:

[10/19/2017 13:31:41 > 22e0db: INFO] Now fetching utilization records from the Partner Center SDK.
[10/19/2017 13:31:41 > 22e0db: INFO]
[10/19/2017 13:31:41 > 22e0db: INFO] Error encountered:
[10/19/2017 13:31:41 > 22e0db: INFO] Could not find file 'D:\local\Temp\jobs\continuous\BillingWebJob\fkw4mq3h.5ph\ResourceUtilizationList.json'.
[10/19/2017 13:31:42 > 22e0db: INFO] StackTrace :
[10/19/2017 13:31:42 > 22e0db: INFO] at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
[10/19/2017 13:31:42 > 22e0db: INFO] at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
[10/19/2017 13:31:42 > 22e0db: INFO] at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
[10/19/2017 13:31:42 > 22e0db: INFO] at System.IO.StreamReader..ctor(String path, Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize, Boolean checkHost)
[10/19/2017 13:31:42 > 22e0db: INFO] at System.IO.File.InternalReadAllText(String path, Encoding encoding, Boolean checkHost)
[10/19/2017 13:31:42 > 22e0db: INFO] at System.IO.File.ReadAllText(String path)

The CspUtilizationHelper tries to load this file, but it's not provided in this project. Can you guys please correct this?

Webjob returning exception.Swagger/cspusage is working but cspbilling error

Webjob returning exception.Swagger/cspusage is working but cspbilling error

Response Body
{
"Message": "An error has occurred."
}
Response Code
500

Release date of the promised new version?

Microsoft promotes this tool as the official cost tracking solution for CSP customers and unfortunately many have notices major issues during the actual implementation. Instead of banging their head to the wall, many have decided to implement their internal solutions based on the offical API, because the support for this solution has been quite clearly nonexistent. Many users have had issues with the webjobs and other errors. All errors were promised to be fixed during May with the next version of the solution. We still haven't seen the actual implementation and are unsure if any issues will be fixed. API is constantly changing, so the real question is that, should we even bother to test this or create our our solution?

Could it be possible to show the actual roadmap for this solution to the public and reassure that this solution is still worked on and has enough resourcess for future development . Currently this still looks like a POC implementation that might or might not work without any guarantee of future development.

Where are the offline deployment binaries for FTP deployment option

Deployment guide has the line below in the Data Extraction Service API section, but I'm unsure where the source code link is, and I'm unable to find the "OfflineDeploymentBinaries" folder anywhere in the github code repository.

"For deployment via FileZilla FTP Client: From the source code link, copy the “BillingDataApi” folder from “OfflineDeploymentBinaries” folder to local computer."

Not able to see the Azure Tag information

Hi,

Peek is a nice tool and give us a good feeling on the Azure spending's for our customers.
But is it possible to figger out what the cost is for a specific resource, for example we have a couple of virtual machine and we need to now what the spending's are of this virtual machines. I thought we can use the Tags on a Virtual Machine but i'm not able to see tag information in peek and can't find the tag information in the database or in the webinterface XXXX.azurewebsites.net/swagger.

Can you tell me if this is possible?

Regards,
Berry

This repo is missing a LICENSE file

This repository is currently missing a LICENSE file.

A license helps users understand how to use your project in a compliant manner. You can find the standard MIT license Microsoft uses at: https://github.com/microsoft/repo-templates/blob/main/shared/LICENSE.

If you would like to learn more about open source licenses, please visit the document at https://aka.ms/license (Microsoft-internal guidance).

Expose LicenseType Flag (to monitor AHUB usage)

Hi Guys,

Can you consider exposing this flag [LicenseType] to the available consumption Data:
https://docs.microsoft.com/en-us/powershell/resourcemanager/azurerm.compute/v1.3.4/new-azurermvm

This is a VM property but directly related with "consumption" and Azure Cost as this is Something you need to activate to get Azure Hybrid Usage Benefit. This is an extra information very very usefull to optimize Customer consumption

Thanks
-Benjamin

Anyone still using this code ?

Hi
Is this code still working ?
I tried to build/use it, but get this error:
[03/05/2019 14:52:33 > 206484: INFO] Starting CSP Routine. Current Usage, Historic Usage and Historic Billing data will be updated..
[03/05/2019 14:52:33 > 206484: INFO]
[03/05/2019 14:52:33 > 206484: INFO] Fetching records for Current Month's Usage from the API. This may take a while. If this operation is timimg out, consider increasing the TimeOut limit in Configuration file.
[03/05/2019 15:08:13 > 206484: INFO]
[03/05/2019 15:08:13 > 206484: INFO] Error encountered:
[03/05/2019 15:08:13 > 206484: INFO] One or more errors occurred.
[03/05/2019 15:08:14 > 206484: INFO] InnerException :
[03/05/2019 15:08:14 > 206484: INFO] Operation returned an invalid status code 'InternalServerError'
[03/05/2019 15:08:14 > 206484: INFO] StackTrace :
[03/05/2019 15:08:14 > 206484: INFO] at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
[03/05/2019 15:08:14 > 206484: INFO] at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification) [03/05/2019 15:08:14 > 206484: INFO] at System.Threading.Tasks.Task1.get_Result()
[03/05/2019 15:08:14 > 206484: INFO] at BillingWebJob.Helpers.CspDataHelper.StartCspRoutine(Int32& status, Int32& totalRecordsCount, String& blobStorageUri) in C:\Users\mlh\Desktop\Azure PEEK\peek-master\AzureBillingAnalytics\BillingWebJob\Helpers\CspDataHelper.cs:line 534
[03/05/2019 15:08:14 > 206484: INFO] at BillingWebJob.Program.CronJob(TimerInfo timer) in C:\Users\mlh\Desktop\Azure PEEK\peek-master\AzureBillingAnalytics\BillingWebJob\Program.cs:line 122
[03/05/2019 15:08:14 > 206484: INFO]
[03/05/2019 15:08:14 > 206484: INFO] Adding audit information in AuditData table..
[03/05/2019 15:08:14 > 206484: INFO]

And im not sure, if it is because the solution is unsupported ?

/Martin

This repo is missing important files

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

Webjob thrown exception for csp. Unable to populate database

This is the output of the BillingWebJob.exe
It is not populating the database with any information
Please help
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO] Adding audit information in AuditData table..
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO] Job started for fetching billing data at 05/23/2019 09:00:02
[05/23/2019 09:00:02 > fca8fa: INFO] Starting job processing for Customer Type csp
[05/23/2019 09:00:02 > fca8fa: INFO] Starting CSP Routine. Current Usage, Historic Usage and Historic Billing data will be updated..
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO] Fetching records for Current Month's Usage from the API. This may take a while. If this operation is timimg out, consider increasing the TimeOut limit in Configuration file.
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO] Error encountered:
[05/23/2019 09:00:02 > fca8fa: INFO] One or more errors occurred.
[05/23/2019 09:00:02 > fca8fa: INFO] InnerException :
[05/23/2019 09:00:02 > fca8fa: INFO] Operation returned an invalid status code 'Unauthorized'
[05/23/2019 09:00:02 > fca8fa: INFO] StackTrace :
[05/23/2019 09:00:02 > fca8fa: INFO] at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
[05/23/2019 09:00:02 > fca8fa: INFO] at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification) [05/23/2019 09:00:02 > fca8fa: INFO] at System.Threading.Tasks.Task1.get_Result()
[05/23/2019 09:00:02 > fca8fa: INFO] at BillingWebJob.Helpers.CspDataHelper.StartCspRoutine(Int32& status, Int32& totalRecordsCount, String& blobStorageUri) in C:\Users\USERNAME\Desktop\peek-masterog\peek-master\AzureBillingAnalytics\BillingWebJob\Helpers\CspDataHelper.cs:line 534
[05/23/2019 09:00:02 > fca8fa: INFO] at BillingWebJob.Program.CronJob(TimerInfo timer) in C:\Users\USERNAME\Desktop\peek-masterog\peek-master\AzureBillingAnalytics\BillingWebJob\Program.cs:line 122
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO] Adding audit information in AuditData table..
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO] Operation complete. Exiting this run.
[05/23/2019 09:00:02 > fca8fa: INFO]
[05/23/2019 09:00:02 > fca8fa: INFO] Executed 'Program.CronJob' (Succeeded, Id=ad8b7ec0-a4be-4ddd-a652-0370be6acba9)

BillingWebJob error

Hello,

I experience a problem on running BillingWebJob.exe for the first time.

In https://[MY APP NAME].scm.azurewebsites.net/azurejobs/#/jobs/continuous/BillingWebJob there is the following message:

The configuration is not properly set for the Microsoft Azure WebJobs Dashboard.
In your Microsoft Azure Website configuration you must set a connection string named AzureWebJobsDashboard by using the following format DefaultEndpointsProtocol=https;AccountName=NAME;AccountKey=KEY pointing to the Microsoft Azure Storage account where the Microsoft Azure WebJobs Runtime logs are stored.

Screenshot in Figure 1 below

The log output is as follows:

[02/07/2017 19:47:41 > 40f151: SYS INFO] Detected WebJob file/s were updated, refreshing WebJob
[02/07/2017 19:47:41 > 40f151: SYS INFO] Status changed to Stopping
[02/07/2017 19:47:41 > 40f151: INFO] Job host stopped
[02/07/2017 19:47:41 > 40f151: SYS INFO] Status changed to Success
[02/07/2017 19:47:41 > 40f151: SYS INFO] Status changed to Stopped
[02/07/2017 19:47:41 > 40f151: SYS INFO] Status changed to Starting
[02/07/2017 19:47:41 > 40f151: SYS INFO] Job directory change detected: Job file 'BillingWebJob.exe.config' timestamp differs between source and working directories.
[02/07/2017 19:47:47 > 40f151: SYS INFO] Run script 'BillingWebJob.exe' with script host - 'WindowsScriptHost'
[02/07/2017 19:47:47 > 40f151: SYS INFO] Status changed to Running
[02/07/2017 19:47:49 > 40f151: INFO] Found the following functions:
[02/07/2017 19:47:49 > 40f151: INFO] BillingWebJob.Program.CronJob
[02/07/2017 19:47:49 > 40f151: INFO] Executing: 'Program.CronJob' - Reason: 'Timer fired at 2017-02-07T19:47:49.4738668Z'
[02/07/2017 19:47:49 > 40f151: INFO] Job started for fetching billing data at 02/07/2017 19:47:49
[02/07/2017 19:47:49 > 40f151: INFO] Starting job processing for Customer Type csp
[02/07/2017 19:47:49 > 40f151: INFO] Starting CSP Routine. Current Usage, Historic Usage and Historic Billing data will be updated..
[02/07/2017 19:47:50 > 40f151: INFO]
[02/07/2017 19:47:50 > 40f151: INFO] Fetching records for Current Month's Usage from the API. This may take a while. If this operation is timimg out, consider increasing the TimeOut limit in Configuration file.
[02/07/2017 19:57:50 > 40f151: INFO]
[02/07/2017 19:57:50 > 40f151: INFO] Error encountered:
[02/07/2017 19:57:50 > 40f151: INFO] One or more errors occurred.
[02/07/2017 19:57:50 > 40f151: INFO] InnerException :
[02/07/2017 19:57:50 > 40f151: INFO] Exception of type 'Microsoft.Rest.HttpOperationException1[System.Object]' was thrown. [02/07/2017 19:57:50 > 40f151: INFO] StackTrace : [02/07/2017 19:57:50 > 40f151: INFO] at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) [02/07/2017 19:57:50 > 40f151: INFO] at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification)
[02/07/2017 19:57:50 > 40f151: INFO] at System.Threading.Tasks.Task`1.get_Result()
[02/07/2017 19:57:50 > 40f151: INFO] at BillingWebJob.Helpers.CspDataHelper.StartCspRoutine(Int32& status, Int32& totalRecordsCount, String& blobStorageUri)
[02/07/2017 19:57:50 > 40f151: INFO] at BillingWebJob.Program.CronJob(TimerInfo timer)
[02/07/2017 19:57:50 > 40f151: INFO]
[02/07/2017 19:57:50 > 40f151: INFO] Adding audit information in AuditData table..
[02/07/2017 19:57:50 > 40f151: INFO]
[02/07/2017 19:57:56 > 40f151: INFO]
[02/07/2017 19:57:56 > 40f151: INFO] Operation complete. Exiting this run.
[02/07/2017 19:57:56 > 40f151: INFO]
[02/07/2017 19:57:56 > 40f151: INFO] Executed: 'Program.CronJob' (Succeeded)
[02/07/2017 19:57:56 > 40f151: INFO] Job host started

The exception is written to the database table AuditData, but no data is written to any other table. I see no problem with the Storage connection strings. I started with a single Storage account, but have also tried with unique accounts for AzureWebJobsDashboard and AzureAnalyticsDbModel.

Suggestions to help troubleshoot the problem are appreciated.

Figure 1