Spinning up cloud services is almost as easy as grabbing food to go at your favorite burger joint. We’ve made fast food so convenient, affordable, and packed with calories, it’s become a national problem, because our consumption doesn’t match our nutritional needs or activity levels. Now imagine how much worse it would be if we had no ingredient lists, no easy way to count calories, and all of our weight gain for the month showed up in a single day.
Enterprises’ love affair with cloud technology and services, and the consequent struggle to keep cloud spending in check, is a lot like monitoring our daily eating habits.
Avoiding Cloud Fat
How is cloud consumption like eating a cheeseburger? We love the convenience and ability to spin up resources at the drop of a credit card, and the way we can easily scale resources up and down at will to match demand. On the other hand, it’s easy to have too much of a good thing. That’s how you end up with ‘Cloud Fat,’ which occurs when you consume resources you don’t really need. But instead of expanding your waist line, you’re expanding your “waste line.” In fact, research shows that up to 30 percent of cloud spend can be wasted.
Why do we have all this waste? Well, cloud makes it easy to spend without governance and control. Per unit costs seem so low and priced by the hour, how could they possibly add-up to tens of thousands of dollars each month?
How do you prevent Cloud Waste?
Most businesses rely on monthly bills from their cloud providers, which can pose problems. Cloud bills are in technical terms, rather than terms that make sense to your business. They are also after-the-fact, long past the point at which you can react to a problem in a timely fashion, and lack the historical context required to pinpoint problems or plan for future spend. While most cloud providers offer some sort of tagging capability that enables customers to categorize resources, compliance and consistency is difficult to enforce, making it hard to understand who used which resources and at what cost.
There are many different sources of waste, but some of the most common include purchasing oversized virtual machines (VMs), leaving VMs turned on when they are no longer needed, using expensive storage for infrequently accessed data, or just spinning up unauthorized resources (aka, Shadow IT). In order to manage these resources, you first have to measure them – you need a “smart meter” for your cloud that will allow you to match usage with users and put costs against it, so you can see who is using how much of what, and what it is costing you. The next step is to use this information to ensure that you are using your cloud resources efficiently and effectively.
As you adopt more cloud services, and as spend increases, managing cloud costs becomes a much larger and more critical task. You need to consider how the data is to be used, how often, and by whom, in addition to the factors below:
- Which cloud platform(s) are used or planned?
- Public cloud, Private cloud, Multi- or hybrid cloud?
- What are our reporting requirements?
- Showback, chargeback, billing?
- Do we have customizable reports, dashboards for business users?
- How much data do we have?
- How accurate does the data need to be?
- Do we need “real-time” results?
- What is our budget for a cloud analytics app and how does it compare to the labor costs (and opportunity costs) of tracking spend and usage manually?
There are four steps required to transform your data into meaningful intelligence, otherwise known as Cloud Analytics: normalization, enrichment, relevancy and visualization.
Normalizing data into common buckets and units of measure across your different cloud platforms provides more holistic and useful analytics that ultimately drive better decisions.
One of the key factors in data transformation is the ability to map your cloud data to organizational and financial information, to provide the business context that matters to you and that you need to drive effective decision making.
In order to trim the cloud fat, your cloud consumers need fast access to timely, relevant information, so they can see where they’re spending and trim where necessary. Role-based permissions allow you to apply filters to cut down on extraneous information “noise” -- and also protects company data by restricting access to confidential information.
Raw data or data in spreadsheets lacks the context and the visual ‘ah-hah!’ you need to optimize your cloud spend and make better business decisions. Different roles require different levels of information, or different kinds of information – the finance folks typically are concerned with costs, while DevOps may be more focused on units – VMs, network throughput, storage volumes, and the like.
As they say, a picture is worth a thousand words – in this case, hundreds of thousands of rows of data. The amount of data is growing exponentially, with more than 2.5 quintillion bytes of data being produced every day. It’s no wonder the BI and analytics market is predicted to reach $16.9B this year. Our poor, limited brains do much better when looking at pretty pictures than trying to find meaning in column after column of numbers.
Formulating a Plan of Attack
Every organization is different, so it’s best to start out small and build from there. Start with the highest cost resources or the resources that have the greatest impact on your business. If CPU performance is mission-critical for your customer-facing apps, then target VM utilization first. If keeping costs under budget for a particular project is your main objective, then target costs by project.
To help your organization develop a better plan of attack, we have developed 12 tips to help manage cloud spend:
- If you have more than $20,000 of cloud spend per month, eliminate the spreadsheet and automate.
- Collect cloud usage and spend on a daily basis.
- Normalize data across cloud platforms, service bundles, and units of measure to see everything in a single, holistic manner.
- Think of the end results you want first, and work backwards to identify users, visualizations and data requirements.
- Add both organizational and financial context to your cloud data to give business meaning.
- Eliminate any IT bottleneck – give end users self-service reporting access. Save time and money.
- Develop use cases to ensure your tagging strategy considers both financial and operational goals.
- People and processes are not enough to enforce consistent tagging – technology must be used to add, correct, or transform tags.
- Data filtering should be enforced to ensure both relevancy and confidentiality.
- Use the right visualization chart for the story you’re trying to tell with your data.
- Implement multiple levels of control to ensure usage and spending stays within pre-set thresholds.
- Focus first on the areas of greatest spend and/or the areas of greatest impact to the business.
Count Your Cloud Calories
Don’t let your consumption get out of hand. Make it easy to count those “cloud calories” – empower your users with easy to read, easy to access data on cloud usage. Don’t make IT a roadblock – or even a speed bump – or users will resort to shadow IT (just like that secret stash of candy bars in your drawer). With relevant data presented visually and in context, users will do a better job of matching consumption to business activity. Encourage them to weigh in daily in order to spot spikes in usage or spending before they balloon into bloat.
We love the cloud. It’s dynamic, agile, scales and drives business value. However, the ease of access to resources can lead to waste, inefficiency, and out-of-control spending. Fortunately, with the right mix of people, process, and technology, you can defeat Cloud Bloat. A trim “waste line” and a healthy, sustainable approach to growth and technology use will keep your organization agile and flexible.
About the Author
Andrew Atkinson is Senior Director of Product Marketing at Cloud Cruiser where he has responsibility for working with engineering, product management and sales teams to bring products successfully to market. Andrew has proven ability to achieve and act upon a deep understanding of customers, markets and competitors. Previously at E2open, Andrew has more than two decades of startup, international and IPO experience with various high technology companies. Andrew started his career in Boston, MA after he completed his bachelor’s degree at Harvard University and then went on to complete his Masters in Applied Economics and Finance at the University of California, Santa Cruz.
Edited by Alicia Young