Why data empowerment is key to effective cloud operations

An effective cloud strategy must include more than just moving everything to the cloud. For optimum business outcomes, organisations need a robust data strategy from migration to maintenance

Most organisations are on a journey to update their IT infrastructure securely and sustainably. Digital transformation is an almost universal business strategy and, at its core, it is about modernisation. An urgency to transform technology usually means something is not as it should be within the organisation, whether that’s feeling the rise of competitive pressures, the burden of legacy systems holding the company back or a need to be more efficient and productive.

There are many possible catalysts for change but, no matter what, a business need will be in the driver’s seat. Cloud technology is one such pillar of digital transformation where organisations have increasingly realised the potential value and ramped up investment. Even those reluctant to embrace cloud technology will have likely been compelled to change tack to navigate the disruption of the pandemic. And in 2022, at least 74% of organisations have a hybrid cloud strategy.

But a plan that stops at cloud migration, or doesn’t take into account exactly how an organisation will need to use its cloud infrastructure (or might need to in the future) is destined for problems. A cloud strategy is a continual journey, not a destination, and doesn’t end once an organisation has transferred workloads to a new system. A strategy should be mapped out from careful planning and preparation to execution and operation.

Often this is less an oversight and more a lack of resource and ineffective budget allocation. One of the key issues organisations come up against on their cloud journey is having the right skills and expertise. Michael O’Donnell, Ph.D, senior analyst at Quest, explains “[many organisations] can’t get the skills, people and resources to deliver the value promised by cloud. For example, a big challenge right now is that companies may have a desire to shift to a cloud-native microservices architecture. However, the skills shortage means organisations are still virtualising workloads and moving them to cloud services.” With this ‘lift and shift’ approach, organisations are missing a real opportunity for change. 

“By extension, the skills shortage is in itself driving the need for automation, and again the need for cybersecurity professionals as more and more devices come onto the edge, e.g. connected cars and the rise in IoT devices,” says O’Donnell. “That’s replicated across the board, in every role that a company might need for re-architecting or their re-platforming/cloud migration programme.”

To tackle this issue, organisations need to work with a clear framework and roadmap, prioritising what they can do today with the resources they have and what they need to work towards. “When you look at a data mesh, or data fabric or data control plane, that’s a roadmap they can work towards,” adds O’Donnell. For example, organisations can work out what tasks to automate, and prioritise ensuring that data is interoperable so other applications can use it. If you have a roadmap, you can break the programme into manageable chunks and build it up. To quote the famous proverb: “how do you eat an elephant? One bite at a time.” 

Batch adoption has the advantage of breaking down what can be an overwhelming project, both financially and in terms of skill resourcing, into more manageable pieces over time. “It’s an effective way to build on a company’s tacit knowledge of cloud services,” says O’Donnell.

However, sometimes the costs for commercial-off-the-shelf (COTS) on-prem software may actually end up smaller than the stack up of microservices. It would be impossible to know this unless organisations have done some upfront enterprise architecture analysis. The business outcomes need to be in balance with the desire of IT to push technology boundaries and operational efficiency gains.

A key challenge is cloud operational costs. The cloud market is growing and spend is increasing, but many organisations are struggling to manage their cloud budgets. A recent survey from Cast AI found that organisations are overspending threefold on cloud costs. The main reason for this is overspending on expensive resources, with little to no return. “It’s really easy to over-allocate IT services within the cloud, with scope creep and all these ‘dark cloud’ services that get spun up like mushrooms,” says O’Donnell. Organisations themselves estimate that they waste 30% of their cloud spend. Given that cloud budgets are on the rise, organisations must ensure they are getting a return on this increased investment and allocating that spend wisely to enable proper management of their growing cloud ecosystem. 

O’Donnell adds, “the costs of data movement between cloud service providers and on-premises can be a little bit underestimated as well. The big cloud vendors are happy to let you put in as much data as possible and won’t charge you for it. But when you want to take it back out, they hit you with even bigger charges.” 

It’s therefore really important for organisations to be monitoring their cloud usage, cloud services and cloud workloads. “Because now we’re looking at hybrid cloud, as opposed to one single cloud operator, companies certainly want to look at a single window where they can get all those services and workloads together in a single place; independently from the cloud service provider’s own monitoring services,” says O’Donnell.

A lot of cloud pitfalls also lie in an organisation focusing a short-term effort on cloud migration, and not putting enough thought into operations. O’Donnell notes that this is a bit of a chicken and egg situation. “When you’re a chicken, you’re thinking about going to the cloud, and then the egg is ‘oh, I’m in the cloud, what next?’ Well, you’re gonna need to get more systems and services up there.” 

Most organisations get the initial setup sorted and then realise they don’t have much of a strategy for what to do once they get there and the continual work that needs to be done to ensure success. And when there’s no clear strategy, it’s very difficult to understand what is working and where there is a genuine ROI vs overspending and cost inefficiencies. “This is where an investment in an overall enterprise architecture will pay for itself; discovering and identifying services, prioritising based on business need and planning for the future,” says O’Donnell.

This lack of visibility can also cause concerns over data security and proper use. This is troubling for organisations, who are increasingly moving large swathes of critical data into cloud services. According to Flexera, only 16% of organisations now keep all of their consumer data on-prem, and only 23% of organisations do the same with their financial data. Yet the same report found organisations cited security as a top cloud challenge (81% of respondents), with governance and compliance also scoring high in the concerns. 

Effective cloud operations must involve an ongoing, robust data strategy that includes data operations, data protection and data governance. Quest calls this strategy ‘data empowerment’, a combination of technology and processes to ensure data in the cloud is migrated and managed properly, retains its quality and is available to everyone and every application that needs it within the organisation. 

It’s vital that organisations have this strategy in place from the initial cloud migration to ongoing maintenance and expansion. Key considerations include: ensuring there is a plan for data recovery during migration and once in the cloud, identifying and protecting personal data prior to migration and ensuring data transformation takes into account data type mappings and other platform differences to maintain data quality. Once organisations are certain of the quality and safety of data, it’s paramount that they have a plan for its effective use. 

O’Donnell notes that everything starts with data, but data in and of itself doesn’t enable an organisation to succeed. “Data is at the core of everything, but its empowerment comes from the information and the ability for consumers to connect with that information. And a lot of the work that we do around governance is the core of data enablement, empowerment and enterprise intelligence. And at the core of data governance is your metadata at the core of that is your data itself.”

So, if data is at the centre of everything, how can businesses get more out of it? Organisations must have a rigorous framework in place to understand their data processes – what data they are collecting, where it’s going, how it’s cleaned, how it’s stored and processed, and who has access to it. 

Organisations must simultaneously consider the costs of all of this if they are to see a return on their investment. They will need processes and technology in place to test database workloads and assess and compare performance and cost in preparation for migration, and then monitor infrastructure performance, resource consumption and cost once in the cloud.

A cloud strategy can’t be created and implemented in isolation. It impacts the entire organisation and so requires input from every stakeholder group. Infrastructure may be the obvious focal point, but there is also a golden opportunity for data engineers to futureproof an organisation’s data architecture. 

O’Donnell notes the challenge here, given the trend of disaggregation in tech teams. While IT teams will continue to stay separate, he explains that siloed working must be addressed. “We’re probably going to still keep disaggregated functions, but working very closely together, moving forward. And that’s what’s key to companies. They really need to find a rhythm of promoting collaboration and striking that efficiency piece. That’s the core of DevOps, site reliability engineering, BizDevOps – the need to break down silos for efficiency, productivity, quality and stability.”

With all of this in place, cloud has the potential to bring organisations firmly into the 21st century. They can fully modernise their operations, enable the democratisation of data and ensure better business outcomes to stay one step ahead of the curve in a competitive market. To get the most from investment in the cloud, organisations may want to enlist the help of a trusted provider. Quest provides solutions for today’s pressing data and information challenges, including demands for data democratisation, data privacy and cybersecurity. Quest can help organisations build a sustainable foundation that enables agility, adaptability and resilience, and maximises the business impact of their data.

Courtesy: https://www.raconteur.net/