Error while running the report

[trackit]

Error while running the report

[sathyajackson]

any update would be of great help

[thibautcornolti]

Hello,

Can I have more logs and the exact command you typed?

Thanks.

[sathyajackson]

here you go.

Command:

./run.py --billing default billing-aws-cost-report billing/XXX.csv --ec2 default --xlsx-name billing.

Here is the complete log.

 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  582M  100  582M    0     0  41.3M      0  0:00:14  0:00:14 --:--:-- 43.2M
Download billings for default...
Fetching ec2 data for all accounts in ap-south-1...
Fetching ec2 data for all accounts in eu-west-3...
Fetching ec2 data for all accounts in eu-north-1...
Fetching ec2 data for all accounts in eu-west-2...
Fetching ec2 data for all accounts in eu-west-1...
Fetching ec2 data for all accounts in ap-northeast-2...
Fetching ec2 data for all accounts in ap-northeast-1...
Fetching ec2 data for all accounts in sa-east-1...
Fetching ec2 data for all accounts in ca-central-1...
Fetching ec2 data for all accounts in ap-southeast-1...
Fetching ec2 data for all accounts in ap-southeast-2...
Fetching ec2 data for all accounts in eu-central-1...
Fetching ec2 data for all accounts in us-east-1...
Fetching ec2 data for all accounts in us-east-2...
Fetching ec2 data for all accounts in us-west-1...
Fetching ec2 data for all accounts in us-west-2...
[default - ap-northeast-1] Getting reserved instances...
[default - ap-northeast-2] Getting reserved instances...
[default - ap-south-1] Getting reserved instances...
[default - eu-north-1] Getting reserved instances...
[default - eu-west-1] Getting reserved instances...
[default - ap-southeast-1] Getting reserved instances...
[default - eu-west-2] Getting reserved instances...
[default - eu-west-3] Getting reserved instances...
[default - ca-central-1] Getting reserved instances...
[default - us-east-1] Getting reserved instances...
[default - sa-east-1] Getting reserved instances...
[default - us-west-1] Getting reserved instances...
[default - us-east-2] Getting reserved instances...
[default - us-west-2] Getting reserved instances...
[default - eu-central-1] Getting reserved instances...
[default - ap-southeast-2] Getting reserved instances...
[default - us-east-1] Getting on-demand instances...
[default - ca-central-1] Getting on-demand instances...
[default - us-east-2] Getting on-demand instances...
[global - ca-central-1] Getting offerings for all instances...
[global - us-east-2] Getting offerings for all instances...
[default - eu-west-1] Getting on-demand instances...
[global - ca-central-1] Matching on-demand instances with reserved instances...
[global - ca-central-1] Done!
[default - eu-west-2] Getting on-demand instances...
[global - us-east-2] Matching on-demand instances with reserved instances...
[global - us-east-2] Done!
[default - eu-west-3] Getting on-demand instances...
[global - eu-west-1] Getting offerings for all instances...
[default - us-west-1] Getting on-demand instances...
[default - eu-north-1] Getting on-demand instances...
[global - eu-west-2] Getting offerings for all instances...
[default - us-west-2] Getting on-demand instances...
[default - eu-central-1] Getting on-demand instances...
[global - eu-west-1] Matching on-demand instances with reserved instances...
[global - eu-west-1] Done!
[global - eu-west-2] Matching on-demand instances with reserved instances...
[global - eu-west-2] Done!
[default - ap-northeast-1] Getting on-demand instances...
[global - eu-west-3] Getting offerings for all instances...
[global - us-west-1] Getting offerings for all instances...
[global - us-west-2] Getting offerings for all instances...
[global - eu-north-1] Getting offerings for all instances...
[global - eu-west-3] Matching on-demand instances with reserved instances...
[global - eu-west-3] Done!
[global - eu-central-1] Getting offerings for all instances...
[global - us-east-1] Getting offerings for all instances...
[default - us-east-1] Getting offerings for instance 1/8...
[global - us-west-1] Matching on-demand instances with reserved instances...
[global - us-west-1] Done!
[default - sa-east-1] Getting on-demand instances...
[default - ap-south-1] Getting on-demand instances...
[global - us-west-2] Matching on-demand instances with reserved instances...
[global - us-west-2] Done!
[global - eu-north-1] Matching on-demand instances with reserved instances...
[global - eu-north-1] Done!
[global - eu-central-1] Matching on-demand instances with reserved instances...
[global - eu-central-1] Done!
[default - ap-northeast-2] Getting on-demand instances...
[global - ap-northeast-1] Getting offerings for all instances...
[global - ap-northeast-1] Matching on-demand instances with reserved instances...
[global - ap-northeast-1] Done!
[default - ap-southeast-1] Getting on-demand instances...
[default - ap-southeast-2] Getting on-demand instances...
[global - sa-east-1] Getting offerings for all instances...
[global - ap-south-1] Getting offerings for all instances...
[global - sa-east-1] Matching on-demand instances with reserved instances...
[global - sa-east-1] Done!
[global - ap-northeast-2] Getting offerings for all instances...
[global - ap-south-1] Matching on-demand instances with reserved instances...
[global - ap-south-1] Done!
[global - ap-northeast-2] Matching on-demand instances with reserved instances...
[global - ap-northeast-2] Done!
[global - ap-southeast-2] Getting offerings for all instances...
[global - ap-southeast-1] Getting offerings for all instances...
[global - ap-southeast-2] Matching on-demand instances with reserved instances...
[global - ap-southeast-2] Done!
[global - ap-southeast-1] Matching on-demand instances with reserved instances...
[global - ap-southeast-1] Done!
Fetched ec2 data for all accounts in ap-south-1
Fetched ec2 data for all accounts in eu-west-3
Fetched ec2 data for all accounts in eu-north-1
Fetched ec2 data for all accounts in eu-west-2
Fetched ec2 data for all accounts in eu-west-1
Fetched ec2 data for all accounts in ap-northeast-2
Fetched ec2 data for all accounts in ap-northeast-1
Fetched ec2 data for all accounts in sa-east-1
Fetched ec2 data for all accounts in ca-central-1
Fetched ec2 data for all accounts in ap-southeast-1
Fetched ec2 data for all accounts in ap-southeast-2
Fetched ec2 data for all accounts in eu-central-1
[default - us-east-1] Getting offerings for instance 2/8...
[default - us-east-1] Getting offerings for instance 3/8...
[default - us-east-1] Getting offerings for instance 4/8...
[default - us-east-1] Getting offerings for instance 5/8...
[default - us-east-1] Getting offerings for instance 6/8...
[default - us-east-1] Getting offerings for instance 7/8...
[default - us-east-1] Getting offerings for instance 8/8...
[global - us-east-1] Matching on-demand instances with reserved instances...
[global - us-east-1] Done!
Fetched ec2 data for all accounts in us-east-1
Fetched ec2 data for all accounts in us-east-2
Fetched ec2 data for all accounts in us-west-1
Fetched ec2 data for all accounts in us-west-2
Fetching ec2 metadata for default in ap-south-1...
Fetching ec2 metadata for default in eu-west-3...
Fetching ec2 metadata for default in eu-north-1...
Fetching ec2 metadata for default in eu-west-2...
Fetching ec2 metadata for default in eu-west-1...
Fetching ec2 metadata for default in ap-northeast-2...
Fetching ec2 metadata for default in ap-northeast-1...
Fetching ec2 metadata for default in sa-east-1...
Fetching ec2 metadata for default in ca-central-1...
Fetching ec2 metadata for default in ap-southeast-1...
Fetching ec2 metadata for default in ap-southeast-2...
Fetching ec2 metadata for default in eu-central-1...
Fetching ec2 metadata for default in us-east-1...
Fetching ec2 metadata for default in us-east-2...
Fetching ec2 metadata for default in us-west-1...
Fetching ec2 metadata for default in us-west-2...
[574925386644 - ap-southeast-1] Getting instances metadata...
[574925386644 - sa-east-1] Getting instances metadata...
[574925386644 - eu-west-2] Getting instances metadata...
[574925386644 - us-east-1] Getting instances metadata...
[574925386644 - eu-west-1] Getting instances metadata...
[574925386644 - ap-southeast-2] Getting instances metadata...
[574925386644 - us-east-2] Getting instances metadata...
[574925386644 - ap-south-1] Getting instances metadata...
[574925386644 - eu-central-1] Getting instances metadata...
[574925386644 - eu-west-3] Getting instances metadata...
[574925386644 - ap-northeast-1] Getting instances metadata...
[574925386644 - us-west-1] Getting instances metadata...
[574925386644 - us-east-2] Done!
[574925386644 - eu-north-1] Getting instances metadata...
[574925386644 - ca-central-1] Getting instances metadata...
[574925386644 - us-west-2] Getting instances metadata...
[574925386644 - us-east-1] Done!
[574925386644 - ca-central-1] Done!
[574925386644 - eu-west-1] Done!
[574925386644 - eu-west-2] Done!
[574925386644 - ap-northeast-2] Getting instances metadata...
[574925386644 - us-west-1] Done!
[574925386644 - eu-central-1] Done!
[574925386644 - eu-west-3] Done!
[574925386644 - us-west-2] Done!
[574925386644 - sa-east-1] Done!
[574925386644 - eu-north-1] Done!
[574925386644 - ap-northeast-1] Done!
[574925386644 - ap-southeast-2] Done!
[574925386644 - ap-southeast-1] Done!
[574925386644 - ap-northeast-2] Done!
[574925386644 - ap-south-1] Done!
Fetched ec2 metadata for default in ap-south-1
Fetched ec2 metadata for default in eu-west-3
Fetched ec2 metadata for default in eu-north-1
Fetched ec2 metadata for default in eu-west-2
Fetched ec2 metadata for default in eu-west-1
Fetched ec2 metadata for default in ap-northeast-2
Fetched ec2 metadata for default in ap-northeast-1
Fetched ec2 metadata for default in sa-east-1
Fetched ec2 metadata for default in ca-central-1
Fetched ec2 metadata for default in ap-southeast-1
Fetched ec2 metadata for default in ap-southeast-2
Fetched ec2 metadata for default in eu-central-1
Fetched ec2 metadata for default in us-east-1
Fetched ec2 metadata for default in us-east-2
Fetched ec2 metadata for default in us-west-1
Fetched ec2 metadata for default in us-west-2
Processing billing data (1/5 - billing diff)...
Traceback (most recent call last):
  File "src/get_bill_diff.py", line 65, in <module>
    first_month = preserved_months[0]
IndexError: list index out of range
Processing billing data (2/5 - instance history)...
Processing billing data (3/5 - ec2 last month)...
Processing billing data (4/5 - ebs last month)...
Processing billing data (5/5 - s3 cost)...
Generating xlsx file...
Traceback (most recent call last):
  File "src/make_xlsx.py", line 630, in <module>
    main(name)
  File "src/make_xlsx.py", line 612, in main
    gen_weekly_variations(workbook, header_format, val_format)
  File "src/make_xlsx.py", line 201, in gen_weekly_variations
    with open(IN_ABSOLUTE_COST_PER_MONTH) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'out/absolute.csv'

[sathyajackson]

The bill file I am using is more than 30 G and more than 3 million records.

[thibautcornolti]

I guess your bucket is billing-aws-cost-report, is it a bucket with a report enabled?
You can check that here https://console.aws.amazon.com/billing/home?#/reports

You should see something as below:

[sathyajackson]

Well no, what i did is i created a bucket in one of testing account and uploaded the bill file in to the bucket. I am doing a POC to showcase on what type of reports we can get using this tool. All our accounts are merged with parent account for billing and i don't have access to that acciunt, i can down load the bill data and can upload the same to another s3 bucket for testing.

…

On Tue, 8 Jan 2019, 10:24 pm Thibaut Cornolti ***@***.*** wrote: I guess your bucket is billing-aws-cost-report, is it a bucket with a report enabled? You can check that here https://console.aws.amazon.com/billing/home?#/reports [image: image] <https://user-images.githubusercontent.com/4880378/50846000-71516200-136e-11e9-8a23-8a07580394ca.png> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#37 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Afb7tCWL0EHGBlF3N37jZaW_t-pgSVTYks5vBM1NgaJpZM4ZzyBD> .

[thibautcornolti]

Got it. So you should have a report enabled on your master account. A report is more than a single .csv, you have to download the entire bucket (sync it with the cli for example).
Are you allowed to do that?

To clarify everything, you can get two types of report with AWS :

• Old report (https://console.aws.amazon.com/billing/home?#/preferences)
  

• New report (https://console.aws.amazon.com/billing/home?#/reports)
  

Old report will be removed by AWS.
Our tool only works with the new type of report. You have to be sure you're using it.

[sathyajackson]

I understand about the old report and new report on AWS. Just to clarify, can't i just run this script against one whole months bill and get the required informations? Let's say i have a full details billing available for December in the zip formate (Just one file)can't i use that as a source file on any of my s3 bucket and run the script against it to get all my finds for the month of decemeber? ( including the sizing recommendation )

…

On Tue, 8 Jan 2019, 11:02 pm Thibaut Cornolti ***@***.*** wrote: Get it. So you should have a report enabled on your master account. A report is more than a single .csv, you have to download the entire bucket (sync it with the cli for example). Are you allowed to do that? To clarify everything, you can get two types of report with AWS : - Old report (https://console.aws.amazon.com/billing/home?#/preferences) [image: image] <https://user-images.githubusercontent.com/4880378/50848072-8250a200-1373-11e9-85bb-eb9cbaca4c9c.png> - New report (https://console.aws.amazon.com/billing/home?#/reports) [image: image] <https://user-images.githubusercontent.com/4880378/50848090-8f6d9100-1373-11e9-8d4d-a48273b3bc04.png> Old report will be removed by AWS. Our tool only works with the new type of report. You have to be sure you're using it. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#37 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Afb7tAeYsMgHTnAJ1hU_siJyu2U_Qlqxks5vBNYTgaJpZM4ZzyBD> .

[thibautcornolti]

Sorry you can't, your file is probably in the old format (typically one .csv.zip generated per month), we do not support that report.
However, you can use our tool without billing, just remove the --billing argument. You just won't see any cost.

[sathyajackson]

Ok will try and let you know...

…

On Wed, 9 Jan 2019, 12:18 am Thibaut Cornolti ***@***.*** wrote: Sorry you can't, your file is probably in the old format (typically one .csv.zip generated per month), we do not support that report. However, you can use our tool without billing, just remove the --billing argument. You'll just see no cost. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#37 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Afb7tDKfj6SU_wLnimgUv-vjqdl1Mp2Eks5vBOfogaJpZM4ZzyBD> .

[thibautcornolti]

Ok! I look forward to your answer.

[sathyajackson]

I used the following command to execute without --billing and end up with the same error message.

 ./run.py   --ec2 default --xlsx-name bill

Logs

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  585M  100  585M    0     0  39.9M      0  0:00:14  0:00:14 --:--:-- 41.3M
Fetching ec2 data for all accounts in ap-south-1...
Fetching ec2 data for all accounts in eu-west-3...
Fetching ec2 data for all accounts in eu-north-1...
Fetching ec2 data for all accounts in eu-west-2...
Fetching ec2 data for all accounts in eu-west-1...
Fetching ec2 data for all accounts in ap-northeast-2...
Fetching ec2 data for all accounts in ap-northeast-1...
Fetching ec2 data for all accounts in sa-east-1...
Fetching ec2 data for all accounts in ca-central-1...
Fetching ec2 data for all accounts in ap-southeast-1...
Fetching ec2 data for all accounts in ap-southeast-2...
Fetching ec2 data for all accounts in eu-central-1...
Fetching ec2 data for all accounts in us-east-1...
Fetching ec2 data for all accounts in us-east-2...
Fetching ec2 data for all accounts in us-west-1...
Fetching ec2 data for all accounts in us-west-2...
[default - eu-north-1] Getting reserved instances...
[default - eu-west-3] Getting reserved instances...
[default - eu-west-1] Getting reserved instances...
[default - ap-south-1] Getting reserved instances...
[default - eu-west-2] Getting reserved instances...
[default - ap-northeast-1] Getting reserved instances...
[default - ap-northeast-2] Getting reserved instances...
[default - eu-central-1] Getting reserved instances...
[default - ap-southeast-1] Getting reserved instances...
[default - ap-southeast-2] Getting reserved instances...
[default - us-west-2] Getting reserved instances...
[default - us-east-2] Getting reserved instances...
[default - sa-east-1] Getting reserved instances...
[default - ca-central-1] Getting reserved instances...
[default - us-east-1] Getting reserved instances...
[default - us-west-1] Getting reserved instances...
[default - us-east-2] Getting on-demand instances...
[default - ca-central-1] Getting on-demand instances...
[default - us-east-1] Getting on-demand instances...
[global - us-east-2] Getting offerings for all instances...
[default - eu-west-3] Getting on-demand instances...
[global - us-east-2] Matching on-demand instances with reserved instances...
[global - us-east-2] Done!
[default - eu-west-2] Getting on-demand instances...
[global - ca-central-1] Getting offerings for all instances...
[global - ca-central-1] Matching on-demand instances with reserved instances...
[global - ca-central-1] Done!
[default - eu-west-1] Getting on-demand instances...
[default - us-west-2] Getting on-demand instances...
[default - eu-central-1] Getting on-demand instances...
[default - eu-north-1] Getting on-demand instances...
[default - us-west-1] Getting on-demand instances...
[global - eu-west-3] Getting offerings for all instances...
[global - eu-west-1] Getting offerings for all instances...
[default - ap-northeast-1] Getting on-demand instances...
[global - eu-west-3] Matching on-demand instances with reserved instances...
[global - eu-west-3] Done!
[global - eu-west-2] Getting offerings for all instances...
[global - eu-west-1] Matching on-demand instances with reserved instances...
[global - eu-west-1] Done!
[global - us-west-2] Getting offerings for all instances...
[default - ap-south-1] Getting on-demand instances...
[global - eu-north-1] Getting offerings for all instances...
[default - sa-east-1] Getting on-demand instances...
[global - eu-central-1] Getting offerings for all instances...
[global - eu-west-2] Matching on-demand instances with reserved instances...
[global - eu-west-2] Done!
[global - us-west-2] Matching on-demand instances with reserved instances...
[global - us-west-2] Done!
[default - ap-northeast-2] Getting on-demand instances...
[default - ap-southeast-2] Getting on-demand instances...
[global - eu-central-1] Matching on-demand instances with reserved instances...
[global - eu-central-1] Done!
[global - us-east-1] Getting offerings for all instances...
[default - us-east-1] Getting offerings for instance 1/8...
[global - us-west-1] Getting offerings for all instances...
[global - eu-north-1] Matching on-demand instances with reserved instances...
[global - eu-north-1] Done!
[global - sa-east-1] Getting offerings for all instances...
[global - us-west-1] Matching on-demand instances with reserved instances...
[global - us-west-1] Done!
[default - ap-southeast-1] Getting on-demand instances...
[global - ap-northeast-1] Getting offerings for all instances...
[global - ap-northeast-1] Matching on-demand instances with reserved instances...
[global - ap-northeast-1] Done!
[global - sa-east-1] Matching on-demand instances with reserved instances...
[global - sa-east-1] Done!
[global - ap-south-1] Getting offerings for all instances...
[global - ap-south-1] Matching on-demand instances with reserved instances...
[global - ap-south-1] Done!
[global - ap-northeast-2] Getting offerings for all instances...
[global - ap-southeast-2] Getting offerings for all instances...
[global - ap-northeast-2] Matching on-demand instances with reserved instances...
[global - ap-northeast-2] Done!
[global - ap-southeast-2] Matching on-demand instances with reserved instances...
[global - ap-southeast-2] Done!
[global - ap-southeast-1] Getting offerings for all instances...
[global - ap-southeast-1] Matching on-demand instances with reserved instances...
[global - ap-southeast-1] Done!
Fetched ec2 data for all accounts in ap-south-1
Fetched ec2 data for all accounts in eu-west-3
Fetched ec2 data for all accounts in eu-north-1
Fetched ec2 data for all accounts in eu-west-2
Fetched ec2 data for all accounts in eu-west-1
Fetched ec2 data for all accounts in ap-northeast-2
Fetched ec2 data for all accounts in ap-northeast-1
Fetched ec2 data for all accounts in sa-east-1
Fetched ec2 data for all accounts in ca-central-1
Fetched ec2 data for all accounts in ap-southeast-1
Fetched ec2 data for all accounts in ap-southeast-2
Fetched ec2 data for all accounts in eu-central-1
[default - us-east-1] Getting offerings for instance 2/8...
[default - us-east-1] Getting offerings for instance 3/8...
[default - us-east-1] Getting offerings for instance 4/8...
[default - us-east-1] Getting offerings for instance 5/8...
[default - us-east-1] Getting offerings for instance 6/8...
[default - us-east-1] Getting offerings for instance 7/8...
[default - us-east-1] Getting offerings for instance 8/8...
[global - us-east-1] Matching on-demand instances with reserved instances...
[global - us-east-1] Done!
Fetched ec2 data for all accounts in us-east-1
Fetched ec2 data for all accounts in us-east-2
Fetched ec2 data for all accounts in us-west-1
Fetched ec2 data for all accounts in us-west-2
Fetching ec2 metadata for default in ap-south-1...
Fetching ec2 metadata for default in eu-west-3...
Fetching ec2 metadata for default in eu-north-1...
Fetching ec2 metadata for default in eu-west-2...
Fetching ec2 metadata for default in eu-west-1...
Fetching ec2 metadata for default in ap-northeast-2...
Fetching ec2 metadata for default in ap-northeast-1...
Fetching ec2 metadata for default in sa-east-1...
Fetching ec2 metadata for default in ca-central-1...
Fetching ec2 metadata for default in ap-southeast-1...
Fetching ec2 metadata for default in ap-southeast-2...
Fetching ec2 metadata for default in eu-central-1...
Fetching ec2 metadata for default in us-east-1...
Fetching ec2 metadata for default in us-east-2...
Fetching ec2 metadata for default in us-west-1...
Fetching ec2 metadata for default in us-west-2...
[574925386644 - ap-south-1] Getting instances metadata...
[574925386644 - eu-west-1] Getting instances metadata...
[574925386644 - eu-north-1] Getting instances metadata...
[574925386644 - eu-west-3] Getting instances metadata...
[574925386644 - eu-west-2] Getting instances metadata...
[574925386644 - ap-northeast-1] Getting instances metadata...
[574925386644 - ca-central-1] Getting instances metadata...
[574925386644 - sa-east-1] Getting instances metadata...
[574925386644 - ap-southeast-1] Getting instances metadata...
[574925386644 - eu-central-1] Getting instances metadata...
[574925386644 - ap-southeast-2] Getting instances metadata...
[574925386644 - us-west-1] Getting instances metadata...
[574925386644 - us-east-1] Getting instances metadata...
[574925386644 - eu-west-1] Done!
[574925386644 - us-east-2] Getting instances metadata...
[574925386644 - us-west-2] Getting instances metadata...
[574925386644 - ca-central-1] Done!
[574925386644 - us-east-2] Done!
[574925386644 - eu-west-3] Done!
[574925386644 - eu-west-2] Done!
[574925386644 - eu-north-1] Done!
[574925386644 - us-west-1] Done!
[574925386644 - us-east-1] Done!
[574925386644 - eu-central-1] Done!
[574925386644 - ap-south-1] Done!
[574925386644 - us-west-2] Done!
[574925386644 - ap-northeast-2] Getting instances metadata...
Fetched ec2 metadata for default in ap-south-1
Fetched ec2 metadata for default in eu-west-3
Fetched ec2 metadata for default in eu-north-1
Fetched ec2 metadata for default in eu-west-2
Fetched ec2 metadata for default in eu-west-1
[574925386644 - ap-northeast-1] Done!
[574925386644 - sa-east-1] Done!
[574925386644 - ap-southeast-2] Done!
[574925386644 - ap-southeast-1] Done!
[574925386644 - ap-northeast-2] Done!
Fetched ec2 metadata for default in ap-northeast-2
Fetched ec2 metadata for default in ap-northeast-1
Fetched ec2 metadata for default in sa-east-1
Fetched ec2 metadata for default in ca-central-1
Fetched ec2 metadata for default in ap-southeast-1
Fetched ec2 metadata for default in ap-southeast-2
Fetched ec2 metadata for default in eu-central-1
Fetched ec2 metadata for default in us-east-1
Fetched ec2 metadata for default in us-east-2
Fetched ec2 metadata for default in us-west-1
Fetched ec2 metadata for default in us-west-2
Processing billing data (1/5 - billing diff)...
Traceback (most recent call last):
  File "src/get_bill_diff.py", line 65, in <module>
    first_month = preserved_months[0]
IndexError: list index out of range
Processing billing data (2/5 - instance history)...
Processing billing data (3/5 - ec2 last month)...
Processing billing data (4/5 - ebs last month)...
Processing billing data (5/5 - s3 cost)...
Generating xlsx file...
Traceback (most recent call last):
  File "src/make_xlsx.py", line 630, in <module>
    main(name)
  File "src/make_xlsx.py", line 612, in main
    gen_weekly_variations(workbook, header_format, val_format)
  File "src/make_xlsx.py", line 201, in gen_weekly_variations
    with open(IN_ABSOLUTE_COST_PER_MONTH) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'out/absolute.csv'

Correct me if i am worng in using the .run.py command.

[thibautcornolti]

I just talked with the team:
Finally, we can't run the tool without billing, really sorry for that.

I can help you to setup a report bucket though.

[sathyajackson]

Hi Thanks for your response, one more quick question, the tool actually use the CPU as the base point for the recommendation, I would like to know if we can have both memory and CPU?

I know that cloud watch will target only of the CPU and not memory ( other than writing custom metrics), but yeh, just thought of check if we can have this included on the script to somehow get the memory information of the machine and based on that providing the recommendation.

[thibautcornolti]

No you can't use memory to influence recommendations, as you said, it's not supported by CloudWatch and you would need to install a custom software on every single instance.

[sathyajackson]

alright thanks 👍

[thibautcornolti]

Thanks to you! 👍

You can close this issue as soon as you have no more question.

[vmpowercli]

I get similar error when running the script. Can you please tell me what to enter "--billing profile_name". When I ran the script with --billing default i get the same error message.

Generating xlsx file...
Traceback (most recent call last):
File "src/make_xlsx.py", line 630, in
main(name)
File "src/make_xlsx.py", line 612, in main
gen_weekly_variations(workbook, header_format, val_format)
File "src/make_xlsx.py", line 201, in gen_weekly_variations
with open(IN_ABSOLUTE_COST_PER_MONTH) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'out/absolute.csv'

[thibautcornolti]

Hello,

$> ./run.py --billing profile_name billing-bucket-name prefix

• profile_name is default or another profile available in ~/.aws/credentials.
• billing_bucket_name is you billing bucket. It has to be configured here as following:
  
• prefix is prefix put in report config, you can get it by clicking on a report here:
  
  Prefix here would be bills, billing being the report name.

Thanks.

[AashishChughHome]

I can see the same error while running the reports, can you please suggest where it's going wrong.
Command I am running-

./run.py --billing EBS esg-ebs-costing ebs-nv --ec2 EBS --xlsx-name EBS

MyError

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "src/get_ec2_data.py", line 73, in
compute_instance_costs = json.load(f)
File "/usr/lib64/python3.6/json/init.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "src/get_ec2_data.py", line 73, in
compute_instance_costs = json.load(f)
File "/usr/lib64/python3.6/json/init.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "src/get_ec2_data.py", line 73, in
compute_instance_costs = json.load(f)
File "/usr/lib64/python3.6/json/init.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "src/get_ec2_data.py", line 73, in
compute_instance_costs = json.load(f)
File "/usr/lib64/python3.6/json/init.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "src/get_ec2_data.py", line 73, in
compute_instance_costs = json.load(f)
File "/usr/lib64/python3.6/json/init.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "src/get_ec2_data.py", line 73, in
compute_instance_costs = json.load(f)
File "/usr/lib64/python3.6/json/init.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "src/get_ec2_data.py", line 73, in
compute_instance_costs = json.load(f)
File "/usr/lib64/python3.6/json/init.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "src/get_ec2_data.py", line 73, in
compute_instance_costs = json.load(f)
File "/usr/lib64/python3.6/json/init.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "src/get_ec2_data.py", line 73, in
compute_instance_costs = json.load(f)
File "/usr/lib64/python3.6/json/init.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Fetched ec2 data for all accounts in eu-north-1
Fetched ec2 data for all accounts in ap-south-1
Fetched ec2 data for all accounts in eu-west-3
Fetched ec2 data for all accounts in eu-west-2
Fetched ec2 data for all accounts in eu-west-1
Fetched ec2 data for all accounts in ap-northeast-2
Fetched ec2 data for all accounts in ap-northeast-1
Fetched ec2 data for all accounts in sa-east-1
Fetched ec2 data for all accounts in ca-central-1
Fetched ec2 data for all accounts in ap-southeast-1
Fetched ec2 data for all accounts in ap-southeast-2
Fetched ec2 data for all accounts in eu-central-1
Fetched ec2 data for all accounts in us-east-1
Fetched ec2 data for all accounts in us-east-2
Fetched ec2 data for all accounts in us-west-1
Fetched ec2 data for all accounts in us-west-2
Fetching ec2 metadata for EBS in eu-north-1...
Fetching ec2 metadata for EBS in ap-south-1...
Fetching ec2 metadata for EBS in eu-west-3...
Fetching ec2 metadata for EBS in eu-west-2...
Fetching ec2 metadata for EBS in eu-west-1...
Fetching ec2 metadata for EBS in ap-northeast-2...
Fetching ec2 metadata for EBS in ap-northeast-1...
Fetching ec2 metadata for EBS in sa-east-1...
Fetching ec2 metadata for EBS in ca-central-1...
Fetching ec2 metadata for EBS in ap-southeast-1...
Fetching ec2 metadata for EBS in ap-southeast-2...
Fetching ec2 metadata for EBS in eu-central-1...
Fetching ec2 metadata for EBS in us-east-1...
Fetching ec2 metadata for EBS in us-east-2...
Fetching ec2 metadata for EBS in us-west-1...
Fetching ec2 metadata for EBS in us-west-2...
[954326812311 - eu-north-1] Getting instances metadata...
[954326812311 - ap-south-1] Getting instances metadata...
[954326812311 - eu-west-3] Getting instances metadata...
[954326812311 - eu-north-1] Done!
[954326812311 - eu-west-2] Getting instances metadata...
[954326812311 - eu-west-1] Getting instances metadata...
[954326812311 - eu-west-3] Done!
[954326812311 - ap-south-1] Done!
[954326812311 - ap-northeast-1] Getting instances metadata...
[954326812311 - eu-west-2] Done!
[954326812311 - eu-west-1] Done!
Fetched ec2 metadata for EBS in eu-north-1
[954326812311 - ap-northeast-2] Getting instances metadata...
[954326812311 - sa-east-1] Getting instances metadata...
[954326812311 - ca-central-1] Getting instances metadata...
Fetched ec2 metadata for EBS in ap-south-1
Fetched ec2 metadata for EBS in eu-west-3
[954326812311 - ap-southeast-1] Getting instances metadata...
Fetched ec2 metadata for EBS in eu-west-2
Fetched ec2 metadata for EBS in eu-west-1
[954326812311 - ap-southeast-2] Getting instances metadata...
[954326812311 - ca-central-1] Done!
[954326812311 - ap-northeast-1] Done!
[954326812311 - eu-central-1] Getting instances metadata...
[954326812311 - sa-east-1] Done!
[954326812311 - us-east-1] Getting instances metadata...
[954326812311 - ap-northeast-2] Done!
[954326812311 - us-east-2] Getting instances metadata...
[954326812311 - us-west-1] Getting instances metadata...
[954326812311 - us-west-2] Getting instances metadata...
Fetched ec2 metadata for EBS in ap-northeast-2
Fetched ec2 metadata for EBS in ap-northeast-1
Fetched ec2 metadata for EBS in sa-east-1
Fetched ec2 metadata for EBS in ca-central-1
[954326812311 - us-east-2] Done!
[954326812311 - eu-central-1] Done!
[954326812311 - us-east-1] Done!
[954326812311 - ap-southeast-2] Done!
[954326812311 - us-west-1] Done!
[954326812311 - ap-southeast-1] Done!
[954326812311 - us-west-2] Done!
Fetched ec2 metadata for EBS in ap-southeast-1
Fetched ec2 metadata for EBS in ap-southeast-2
Fetched ec2 metadata for EBS in eu-central-1
Fetched ec2 metadata for EBS in us-east-1
Fetched ec2 metadata for EBS in us-east-2
Fetched ec2 metadata for EBS in us-west-1
Fetched ec2 metadata for EBS in us-west-2
Processing billing data (1/5 - billing diff)...
Traceback (most recent call last):
File "src/get_bill_diff.py", line 65, in
first_month = preserved_months[0]
IndexError: list index out of range
Processing billing data (2/5 - instance history)...
Processing billing data (3/5 - ec2 last month)...
Processing billing data (4/5 - ebs last month)...
Processing billing data (5/5 - s3 cost)...
Generating xlsx file...
Traceback (most recent call last):
File "src/make_xlsx.py", line 630, in
main(name)
File "src/make_xlsx.py", line 612, in main
gen_weekly_variations(workbook, header_format, val_format)
File "src/make_xlsx.py", line 201, in gen_weekly_variations
with open(IN_ABSOLUTE_COST_PER_MONTH) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'out/absolute.csv'