Ufdjrw

School performs periodic password audits. Is my password compromised?

Can the Witch Sight warlock invocation see through the Mirror Image spell?

What will happen if my luggage gets delayed?

Converting from "matrix" data into "coordinate" data

Either of .... (Plural/Singular)

Smooth vector fields on a surface modulo diffeomorphisms

How can I portion out frozen cookie dough?

Professor forcing me to attend a conference, I can't afford even with 50% funding

How do I increase the number of TTY consoles?

Should we avoid writing fiction about historical events without extensive research?

Can one live in the U.S. and not use a credit card?

Are E natural minor and B harmonic minor related?

Did Amazon pay $0 in taxes last year?

What would be the most expensive material to an intergalactic society?

Will expression retain the same definition if particle is changed?

Can I negotiate a patent idea for a raise, under French law?

What is this tube in a jet engine's air intake?

Graphic representation of a triangle using ArrayPlot

Too soon for a plot twist?

Rationale to prefer local variables over instance variables?

Why aren't there more Gauls like Obelix?

If sound is a longitudinal wave, why can we hear it if our ears aren't aligned with the propagation direction?

Giving a career talk in my old university, how prominently should I tell students my salary?

Why does this boat have a landing pad? (SpaceX's GO Searcher) Any plans for propulsive capsule landings?

Cloudwatch generates a lot of costs due to large number of PutMetricData requests

0

cost-wise, the majority of our invoice comes from PutMetricData. I can't really see which piece sends that huge amount.
According to https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/logging_cw_api_calls.html, cloud trail does not trace it.
I am looking for some sort of a sum-up to find out where all that requests come from.

Any help will be appreciated,
thanks in advance

EDIT
attaching API cost spreadout

amazon-web-services amazon-cloudwatch

share|improve this question

edited 2 days ago

Łukasz

asked Mar 6 at 13:27

ŁukaszŁukasz

39011024

add a comment | 

0

cost-wise, the majority of our invoice comes from PutMetricData. I can't really see which piece sends that huge amount.
According to https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/logging_cw_api_calls.html, cloud trail does not trace it.
I am looking for some sort of a sum-up to find out where all that requests come from.

Any help will be appreciated,
thanks in advance

EDIT
attaching API cost spreadout

amazon-web-services amazon-cloudwatch

share|improve this question

edited 2 days ago

Łukasz

asked Mar 6 at 13:27

ŁukaszŁukasz

39011024

add a comment | 

cost-wise, the majority of our invoice comes from PutMetricData. I can't really see which piece sends that huge amount.
According to https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/logging_cw_api_calls.html, cloud trail does not trace it.
I am looking for some sort of a sum-up to find out where all that requests come from.

Any help will be appreciated,
thanks in advance

EDIT
attaching API cost spreadout

amazon-web-services amazon-cloudwatch

share|improve this question

edited 2 days ago

Łukasz

asked Mar 6 at 13:27

ŁukaszŁukasz

39011024

share|improve this question

edited 2 days ago

Łukasz

asked Mar 6 at 13:27

ŁukaszŁukasz

39011024

share|improve this question

edited 2 days ago

Łukasz

asked Mar 6 at 13:27

ŁukaszŁukasz

39011024

asked Mar 6 at 13:27

ŁukaszŁukasz

39011024

asked Mar 6 at 13:27

ŁukaszŁukasz

39011024

1 Answer
1

active

oldest

votes

1

When you say "the majority of our invoice," do you mean that CloudWatch metrics cost more than or a significant percentage of all of the other AWS services that you use?

This is extremely surprising: you can call PutMetricData 100,000 times for $1, so your usage numbers must be astronomical (calling once per second is only 86,400 per day). So surprising that I have to ask are you sure that it's the API calls that are running up the cost, and not the number of metrics? (and are you aware that each combination of dimensions represents a distinct metric?)

If it really is the number of PutMetricData calls, then the only way that you could accumulate those numbers is to be making a call from within a loop -- one that executes many times per second. So I would start by using your IDE (or grep) to find all references to the SDK function, and determining which of those is called in a loop.

There are very few reasons to call PutMetricData from within a loop, unless you expect the body of the loop to take a significant amount of time (seconds to minutes) and are using metrics to track the amount of time this takes.

share|improve this answer

answered Mar 6 at 15:14

guestguest

111

New contributor

guest is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Attached reports. PutMetricData is responsible for around 30% of the costs. I breezed thru our repositories. All manual API requests are scheduled via cron with the highest possible granularity of 1 minute. I preliminary estimated our load and it should be roughly 70 times smaller.
  
  – Łukasz
  2 days ago
  
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55024250%2fcloudwatch-generates-a-lot-of-costs-due-to-large-number-of-putmetricdata-request%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

1

When you say "the majority of our invoice," do you mean that CloudWatch metrics cost more than or a significant percentage of all of the other AWS services that you use?

This is extremely surprising: you can call PutMetricData 100,000 times for $1, so your usage numbers must be astronomical (calling once per second is only 86,400 per day). So surprising that I have to ask are you sure that it's the API calls that are running up the cost, and not the number of metrics? (and are you aware that each combination of dimensions represents a distinct metric?)

If it really is the number of PutMetricData calls, then the only way that you could accumulate those numbers is to be making a call from within a loop -- one that executes many times per second. So I would start by using your IDE (or grep) to find all references to the SDK function, and determining which of those is called in a loop.

There are very few reasons to call PutMetricData from within a loop, unless you expect the body of the loop to take a significant amount of time (seconds to minutes) and are using metrics to track the amount of time this takes.

share|improve this answer

answered Mar 6 at 15:14

guestguest

111

New contributor

guest is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Attached reports. PutMetricData is responsible for around 30% of the costs. I breezed thru our repositories. All manual API requests are scheduled via cron with the highest possible granularity of 1 minute. I preliminary estimated our load and it should be roughly 70 times smaller.
  
  – Łukasz
  2 days ago
  
  

  
  
  
  

add a comment | 

1

When you say "the majority of our invoice," do you mean that CloudWatch metrics cost more than or a significant percentage of all of the other AWS services that you use?

This is extremely surprising: you can call PutMetricData 100,000 times for $1, so your usage numbers must be astronomical (calling once per second is only 86,400 per day). So surprising that I have to ask are you sure that it's the API calls that are running up the cost, and not the number of metrics? (and are you aware that each combination of dimensions represents a distinct metric?)

If it really is the number of PutMetricData calls, then the only way that you could accumulate those numbers is to be making a call from within a loop -- one that executes many times per second. So I would start by using your IDE (or grep) to find all references to the SDK function, and determining which of those is called in a loop.

There are very few reasons to call PutMetricData from within a loop, unless you expect the body of the loop to take a significant amount of time (seconds to minutes) and are using metrics to track the amount of time this takes.

share|improve this answer

answered Mar 6 at 15:14

guestguest

111

New contributor

guest is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Attached reports. PutMetricData is responsible for around 30% of the costs. I breezed thru our repositories. All manual API requests are scheduled via cron with the highest possible granularity of 1 minute. I preliminary estimated our load and it should be roughly 70 times smaller.
  
  – Łukasz
  2 days ago
  
  

  
  
  
  

add a comment | 

When you say "the majority of our invoice," do you mean that CloudWatch metrics cost more than or a significant percentage of all of the other AWS services that you use?

This is extremely surprising: you can call PutMetricData 100,000 times for $1, so your usage numbers must be astronomical (calling once per second is only 86,400 per day). So surprising that I have to ask are you sure that it's the API calls that are running up the cost, and not the number of metrics? (and are you aware that each combination of dimensions represents a distinct metric?)

If it really is the number of PutMetricData calls, then the only way that you could accumulate those numbers is to be making a call from within a loop -- one that executes many times per second. So I would start by using your IDE (or grep) to find all references to the SDK function, and determining which of those is called in a loop.

There are very few reasons to call PutMetricData from within a loop, unless you expect the body of the loop to take a significant amount of time (seconds to minutes) and are using metrics to track the amount of time this takes.

share|improve this answer

answered Mar 6 at 15:14

guestguest

111

New contributor

guest is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share|improve this answer

answered Mar 6 at 15:14

guestguest

111

New contributor

guest is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

answered Mar 6 at 15:14

guestguest

111

New contributor

guest is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

answered Mar 6 at 15:14

guestguest

111