How are big data, cloud computing, advanced analytics and process automation impacting the energy industry? How can these technologies help us respond to the increasing speed and complexity of our world, which still has to manage physical as well as electronic trading? I recently had a great discussion with a panel of senior executives in the energy industry about the promises and challenges of digital transformation.

Big data: drinking from the fire hose

Today’s data scientists are creators who build models, algorithms and workflows. They deliver timely insights to add value and drive social change such as decarbonisation. But to do this, they need flexible and scalable tools that can cope with an unprecedented rise in data quantity and variety.

Adam Atkinson-Lewis from Shell describes one example of this increase in data complexity from the world of power generation. Instead of a controllable portfolio of legacy power plants, it’s now necessary to manage a much less predictable mix of energy sources like wind and solar. So we need sophisticated forecasting models for both supply and demand that take multiple factors into account, with vast amounts of data coming in from millions of sensors and devices.

Adam says “we did a quick calculation for one device in a 5 minute power market and found that it can yield 28 million data points in one day”. This includes pulling data from the device, moving it into a forecast model and merging it with other data sources.

Embracing the cloud

Data isn’t the only aspect of our digital world which is changing fast. The evolution in attitudes to cloud computing and storage has been rapid too. During our web discussion Glen Mackey tells a vivid anecdote. Less than 10 years ago he explored his employer’s willingness to host some of their data in the cloud. Back then, he says, “It was a hard, hard no!”

Since that time the market has moved to regard many aspects of cloud compute to be as secure as on-premise, if not more so. Public cloud providers offer multi-layered security systems, enhanced monitoring and controls plus better incident detection and response than most organizations can afford to build themselves. The dream of secure pay-as-you-go compute without capital costs is now reality.

Joined-up analytics

So how is digital change impacting models and analytics? It’s clear that two kinds of “live linkage” are becoming important.

First, data is dynamic so when input data is modified, these changes must feed though seamlessly to end-users. Cetin Karakus of BP describes this concept of a “graph” of connected information flows during our discussion.

And second, the connection between models themselves and their users needs to be more dynamic. Energy analytics must be nimble as new use cases or new structural features of the data emerge. Data scientists and quants need to get updated models to traders in short order, so they work alongside software and business experts to develop, evaluate, refine, and deploy these tools. This has increased the demand for rapid and collaborative development environments that work effectively from prototype to production, with appropriate controls and governance.

A robot for every person

Automation brings all these changes to life. And it’s about to get personal. “The idea is to put a robot into the hands of every user in an organisation much like you would a spreadsheet” says Doug Wendler of Machina Automation. This is the promise of a new concept known as Citizen Development where employees get the chance to automate their own regular tasks, such as gathering and analysing data from a set of websites. “Over time people will learn to use those robots in a way that make them 10-30% more efficient,” says Doug.

This is what’s on the horizon right now, but much more established is Robotic Process Automation. This adds value and reduces operational risks by taking well established processes, particularly accounting and regulatory ones, and re-engineering them with solid and efficient automation. “The amount of time, money, productivity and accuracy that can be gained by using RPA is really phenomenal,” adds Doug.

Quantifying the Social Side

The engines of these digital innovations aren’t only profitability and efficiency but also the investor-driven trio of Environmental, Social and Governance (ESG) factors. “For energy firms this represents a unique opportunity,” says Matthew Sekol of Microsoft. “It’s not only the chance to be recognised for internal improvements such as safety, sustainability and transparency – but also the opportuity to give their own customers an understanding of how they’re impacting the environment through the energy they’re consuming.”

It’s a chance to turn that data fire hose into a revenue stream. Matthew says utilities are in a special position now. “For energy firms it represents the chance to say ‘I understand your energy because I’m your supplier – and here’s what that means for you.’ ”

I’d like to say a huge thank you to all my panelists who made this such a fun and lively call. Indeed it all got a bit heated at the end … if you’d like to see why, check out the full panel discussion on Beacon’s YouTube channel.

About the panelists

Adam Atkinson-Lewis, Director of Commercialization, Energy Platforms, Shell New Energies

Clean-tech and renewable energy enthusiast working towards a zero-carbon distributed energy future

Adam is on the Energy Platform team at Shell New Energies, working to improve Shell’s utilization of its diverse and growing energy asset base.

Cetin Karakus, Global Head of Quantitative and Analytical Solutions, BP

Expert technologist with distinctive and rare ability to combine organizational and technical authority

Cetin is leading the design and implementation of sophisticated software systems and solutions across quantitative and data analytical domains.

Glen Mackey, Chief Risk Officer, NRG Energy

Previously Head of Global Risk Management at Nexen Inc. and CRO at Reliant Energy

Glen is helping the power market to become smarter, more sustainable and more reliable for businesses and residents across the country by using diverse generation sources and leaning on new digital technologies and management tools.

Matthew Sekol, Industry Executive, Capital Markets, Microsoft

Guides firms through the digital transformation journey by delivering thought leadership, cultural change, and innovative ideas

Matt is helping global financial services firms understand complex digital concepts and upcoming market changes with an eye on their customers.

Doug Wendler, Founder & CEO, Machina Automation

Seasoned software and services executive with 30+ years of experience creating, selling and deploying energy and financial services technology solutions

Doug founded Machina Automation in 2017 to build and share his team’s deep domain expertise with robotic process automation across the energy value chain.

Energy and Commodity Position Management