Cloud Cost Optimization, FinOps, and Cloud Cost Management are new terms that many tech companies have learned the hard way. 

Not knowing them cost a lot. 

According to Flexera’s State of the Cloud Report, cost optimization is a top priority for most companies. Around 61% want to see the cloud costs optimized and under control, while 45% want to see better financial reporting on cloud costs. 

Flexera’s State of the Cloud Report: Top Cloud Initiatives 2021

AWS is by far the leader among the cloud services providers, bringing in a third [33%] of overall cloud infrastructure revenue in Q2 2021, followed by Azure at 20% and GCP at 10%. 

Q2 2021 revenues for PaaS and IaaS and hosted private cloud services totaled $42 billion, while 12-month revenues [from Q3 20 to Q2 21] amounted to $152 billion. The industry has not yet reached its peak, with growth continuing and no plateau in sight. 

Cloud Infrastructure Services Market
Source: Cloud Infrastructure Services Market

This is why Dev.Pro is sharing some insights on how to reduce AWS costs in a sustainable and controlled manner across all teams, services, tools, and regions.

How To Reduce AWS Costs: Use AWS Tools to Monitor and Control Expenditures

Use Cost Explorer to analyze, monitor, and forecast your cloud spending and usage

The Cost Explorer tool is located in the Billing and Cost Management Console of your AWS account.

It helps users review, compare, monitor, and forecast usage and bills for cloud services. Forecasting can be performed up to three months ahead. 

You can review your cloud costs by service, usage type, and even by cost allocation tag [which we highly recommend creating]. 

You will need to use a daily view to analyze suspicious spikes in consumption and a monthly view to see patterns and high-level trends.

AWS Cost Explorer monthly costs by service report
Source: AWS Cost Explorer monthly costs by service report

You can also get recommendations on the utilization of Reserved Instances with the help of Cost Explorer.

Savings Plan, another cost optimization tool, is a useful feature for projecting these three cases over a period of 1 or 3 years:

  • Compute Savings Plans [Amazon EC2, AWS Lambda, and AWS Fargate]
  • EC2 Instance Savings Plans [EC2 Instances]
  • Amazon SageMaker Savings Plans [Amazon SageMaker]

It can be paid for fully or partially upfront and is applied automatically as a discount calculated in $ per hour. 

AWS Cost Explorer Saving Plans
Source: AWS Cost Explorer Saving Plans

Drill Down into Finer Details With a CUR File [AWS Cost and Usage Report]

The AWS CUR report provides the most granular insight into your cloud costs and usage. 

Sometimes you need to investigate a spike in cloud costs and want to review hourly usage. Or perhaps you want  to attribute the Reserved Instance discount to the right department or office inside your organization. Setting up your CUR files can be useful for reference, compliance, audit, and nano-level control.

You can create your CUR file in the Billing and Cost Management console. There are two ways you can store and view them later on:

  • As a raw CSV file from the S3 bucket 
  • As a report in Amazon Athena, AWS QuickSight, AWS RedShift.
Amazon QuickSight visualization of the CUR data

Use AWS Budgets to Optimize AWS Costs

The AWS Budgets tool can be found in AWS Billing Console and helps create a system for monitoring and alerting for two cases:

  1. When your actual or forecasted cost and usage goes above your budget
  2. When your utilization of RI and Savings plan is below your budget or forecast

Users can program AWS Budgets to send them email or SMS messages if a specific service or workload goes above the budgeted estimate for a period as short as 1 day [week, or month].

Additionally, users can automate counter actions based on how a cost or utilization has been exceeded or underused.

AWS Budgets Monitor your cost usage RI and savings plan costs and utilization
AWS Budgets: Monitor your cost, usage, RI and savings plan costs and utilization

Make AWS Trusted Advisor Your Trusted Advisor

AWS Trusted Advisor offers an overview of four major categories: performance, security, fault tolerance, and cost optimization.

The dashboard shows issues that require investigation [orange triangle] or action [red circle].

The list of all 20 cost management checks includes items like Amazon Comprehend underutilized endpoints, idle load balancers, EC2 instances optimization and lease expiration, as well as RedShift Reserved Node optimization.

AWS Trusted Advisor conducts 20 cost optimization checks
AWS Trusted Advisor conducts 20 cost optimization checks

How To Setup AWS to Reduce Costs: Right-Sizing Techniques

Right-sizing is one of the key approaches for AWS cloud cost optimization. It involves tweaking your system settings so that your actual consumed capacity is as close as possible to the available capacity. It means choosing the cheapest instances without jeopardizing your technical performance parameters.

According to Statista, around 30% of cloud spend is wasted. Imagine paying $1 million only to consume $700K worth of a service. With 30% being an average, it’s safe to assume that some people have it far worse. 

Statista Percentage of cloud computing spend wasted in 2020
Statista: Percentage of cloud computing spend wasted in 2020

The biggest problem with right-sizing is that many companies that migrate from onprem to cloud or from one cloud to another shift and lift their applications in an “as is manner,” scheduling the right-sizing exercise for a later date.

Here are some tips for ways to reduce AWS costs by right sizing [AKA re-sizing or downsizing].

The Right EC2 Instance Types: Choose, Review, or Delete

Pay attention to your EC2 instance types. 

Starting a Project or Migrating:

Starting a project small and scaling is an easily arrangeable alternative with AWS. If you are not sure which EC2 instance type to choose from, consult a company that offers AWS cost optimization services. There are several major types, including general purpose, compute optimized, memory optimized, accelerated computing, and storage optimized. 

Ongoing Project: 

You can resize or delete an idle instance. Use your monitoring, reporting, and alerting tools to review existing instances and resize accordingly. 

Downsize. Underutilized instances can be shifted to a more suitable type. This is a continuous process.

Delete idle instances. 

Example of the General Purpose EC2 instance type with M4 parameter
Source: Example of the General Purpose EC2 instance type with M4 parameters.

Choose the Right Storage Class

How frequently do you access your stored data? If you need to access it, how urgent is that need?

There are six Amazon S3 Storage Classes and if latency and access frequency are not an issue [as in data for archiving, for example], the process can result in significant savings. 

To give you a sense of the potential savings, S3 standard class can cost as much as $0.023 per GB for the first 50 TB/ month, while Glacier Deep Archive is only $0.00099 per GB for the entire month.

Amazon S3 storage classes
Amazon S3 storage classes

Right-Size RDS Instances

Just like the EC2 instances, relational database instances also have different types. These are subdivided into 7 general purpose and 9 memory optimized types. Continuous monitoring and a robust alerting system help keep them as close to actual consumption levels as possible in a rapidly changing business environment.

AWS: RDS General Purpose Instance Types: M4
AWS: RDS General Purpose Instance Types: M4

How to Save Money on AWS EC2 

Use the AWS Instance Scheduler to Reduce the Costs of EC2 and Amazon RDS Instances

AWS Instance Scheduler is a prime AWS cost optimization best practice. If your production team only works 8 hours a day, 5 days a week, it makes sense to stop instances running outside of those hours, which can save up to 70%. The same principle is applicable to Amazon Relational DataBase instances.

Remove or Right Size Underutilized or Idle Amazon EC2 Instances 

An idle instance with CPU utilization of 1% or less in two consecutive weeks can be removed to gain savings.

An underutilized instance will have both CPU and memory utilization of 1-60% for the last week or two. It can be right sized to the more appropriate instance family or type.

There are a few ways to detect idle and underutilized instances:

Other AWS cost management tools can help automating these processes: 

  • AWS Instance Scheduler can be programmed to delete all idle instances
  • AWS Operations Conductor can automate the rightsizing process
How to reduce AWS cost: AWS Ops Conductor helps automate resize instances
Source: AWS Ops Conductor helps automate resize instances

Use Amazon EC2 Spot Instances to Save up to 90% on EC2

Using Amazon EC2 Spot Instances can save up to 90% of your cloud cost for specific workloads. These Spot Instances can be reclaimed by EC2 with just 2 minute’s notice, so they are only good for fault tolerant or time-flexible applications, like: 

  • Web servers
  • High-Performance Computing
  • Big data
  • Image and media rendering
  • Containerized workload 
  • CI/CD processes

For further flexibility, users can combine Spot Instances with On-Demand, Saving Plan and RIs. To automate the management process, Spot Instances can be hibernated, terminated, or stopped when AWS Spot Instances reclaims its capacity.

Drill Down in EC2 AutoScaling Groups Configuration

EC2 Autoscaling Groups is a sophisticated feature that needs timely configuration revisions when your business requirements, subscription plans, or consumption patterns change.

This cost optimization technique in AWS allows users to match the fluctuating capacity demand with the available instances closely. It’s achieved by a number of tweaks set in the config module of the tool. They include, among others:

  • The possibility to combine different instance types in one group: RI, On Demand, Savings Plan, and Spot Instances can be part of one group to ensure savings with guaranteed minimal capacity.
  • The ability to prioritize Savings Plan and Reserved Instances, which can optimize expenditure if certain conditions are met.
  • Usage of the Capacity Rebalancing feature. This finds and replaces a Spot Instance at the brink of interruption with a new one, which has not yet received a rebalance recommendation. [remember: Spot Instances are up to 90% discount].

How to Reduce AWS S3 Costs [Amazon Simple Storage Service]

Move Your Infrequently Accessed Storage Into Lower Cost Storage Tiers 

If you rarely access your storage for production purposes, but keep it for backup or compliance reasons, there are a few ways you can optimize the cloud usage associated with it:

  • Check out AWS S3 Analytics to spot infrequently accessed storages that are still being stored in Standard storage class, not Standard-IA [infrequently accessed].
  • Use S3 Intelligent-Tiering to shift storage to a tier according to frequency of its access automatically.

Use Archive and Deep-Archive Access Tiers for Asynchronous Access With Low Access Frequency Stats

To transition infrequently accessed storage to even more cost-effective tiers like Archive or Deep Archive Tiers, users need to create a bucket, prefix, or object tag config in an S3 console, AWS CLI, or PUT API.

AWS Cost Optimization for Amazon DynamoDB 

Optimize Amazon DynamoDB Costs With Autoscaling

Autoscaling for AmazonDB lets users with predictable patterns of workload fluctuations save versus the statistically provisioned capacity. In a statically provisioned scenario, you would set a buffer on top of your expected peak capacity, ensuring uninterrupted performance.

Use CloudWatch to set alarms for maximum and minimum Read and Write capacity levels. You should also set your expected utilization thresholds here, which will trigger the Autoscaling module to follow the consumed capacity as it goes up or down.

Statistically provisioned capacity white space below the red line is unused capacity
Statistically provisioned capacity: white space below the red line is unused capacity
Autoscaling provisioned capacity follows consumed capacity closely with minimal unused resources
Autoscaling: provisioned capacity follows consumed capacity closely with minimal unused resources

Use On-Demand DynamoDB Pricing for Fault-Intolerant Workloads That Are Unpredictable, New, or Serverless

If your application cannot afford any downtime but has unknown or highly fluctuating bursts in utilization, you’ll have to stick with On-Demand pricing. Unlike autoscaling, it instantly matches your demand with capacity. Serverless environments are another good fit for this DynamoDB pricing strategy.

Combine Reserved Capacity With AutoScaling for Workloads With Predictable Base Consumption

Reserved Capacity can be purchased at 46% of the standard rate, meaning that users can save by introducing this approach at times in the day when activity is over 46% of the standard. This cost optimization technique in AWS will work for applications with predictable scaling patterns. In the example below, noon is the peak consumption.

Combination of Reserved Capacity and Auto Scaling for AmazonDB cost optimization
Combination of Reserved Capacity and Auto Scaling for AmazonDB cost optimization

How To Reduce Your AWS Costs: Bonus Tips

There are hundreds of small tweaks you can introduce to harness your cloud spend, which can lead to significant savings.

Right-costing is a twin brother of right-sizing. It deals with the negotiation part of the process. Consider these right costing tricks:

It’s not that hard — just ask. There are way too many terms and conditions to discuss here, but knowing about credits is a good start to asking for them, which is a good start to getting them. They are applied to your monthly bill according to your contract’s conditions.

  • Say yes to testing opportunities with AWS. Then ask for credits.

If AWS offers you an opportunity to take part in testing of a new feature, say yes [when it comes to non-critical applications, of course]. AWS may offer credits for your participation.  If they don’t you’ll still have access to new AWS features before anybody else.

  • Go for another geographical zone

Study your options when it comes to geographical zones. You may find significant savings by switching your workload to a different time zone. AWS has 26 regions with 84 availability zones [AZ] to choose from.

With so many variables in this meticulous process and amazing savings to be gained, many enterprises opt to outsource AWS cost optimization to experienced cloud architects.

Hire a Cloud Cost Optimization Services Provider

Gartner provides a multi-step algorithm for cost management that we decided to share with readers who feel confident they have the right team to perform this complex process.

Gartner Framework of Cloud Cost Optimization
Source: Gartner Framework of Cloud Cost Optimization

Setting up a system of cloud cost monitoring and control is a sophisticated procedure that requires years of experience in FinOps and AWS-specific cloud cost management.

If your team thinks your company lacks the right experts on your team, investing in AWS-certified engineer consulting is guaranteed to result in a high ROI.

DevPro has over 30 certified engineers who help US clients design, develop, and deploy in cloud-native, multi-cloud, and hybrid environments.

Schedule a 15 minute call to discuss your AWS bill savings.

FinTech White Paper