In the oil and gas industry, data quality is everything. An empty spreadsheet cell or a mistake while inputting figures there can have huge ramifications for assessing a project’s viability, drilling the right number of wells in the right places, or understanding what resources to commit to a field.
When it comes to managing data quality, everybody holds two roles, often simultaneously. Those are 1) the data creator, those that create data e.g. a new well pick or a new well. And 2) the data customer – the person who receives data from its creator. This second role is often overlooked but in the quest for clean, reliable data is vitally important.
The conversation around data quality and data roles isn’t new. You may have already heard these kinds of questions being asked by people like Dr. Thomas Redman, or “the Data Doc”. In fact, the issue of data quality is one that’s been around in the industry since the 1960s.
But what is new is the business case around investing time and resources in inputting data correctly at source in the right format, every time. This is where the role of the data customer is important, to provide the feedback when there are data errors.
As data points are created instantaneously across departments and fed into AI systems that can’t tell the difference between good and bad figures, firms face an ever-increasing risk if they can’t nail down exactly where a data set has come from and who is accountable for its quality.
Why don’t we return the item to sender?
One of the key problems with the data customer role is that we don’t observe it as a role at all. For all the importance we place on data quality, we’ve also become too tolerant of bad data and its causes. There’s a culture in our industry that sees data heroes – those tireless problem-solvers who spend their evenings and weekends correcting bad data – as a normal and required part of the solution rather than the symptom of the problem.
Imagine your data was in fact a brand new bookcase from Ikea. If you opened the box and found a broken shelf inside, you wouldn’t try to repair it yourself – you’d take the bookcase back to the store. That store would assess why the shelf was broken and apply improvements to make sure the next one is defect-free. So why don’t we do the same when we’re dealing with poor data, such as drilling coordinates or seismic data?
Because we have a culture that tolerates bad data, we create hidden data factories or technical debt. This is the time taken for people to try to find and fix bad data when it arrives. What is missing is the operational discipline of the data customer to be intolerant of bad data and provide feedback to the drilling team to correct at source.
At Rockflow we have seen clients spending as much as 50% of their time in hidden data factories, without once solving the root cause issue.
We need to format data like it was made to order
Bad data is almost never created on purpose. Most of the time, the reason bad data gets passed along the line is because we don’t always think of people as data customers.
When you’re creating any kind of data, you have to be doing so with the person who’s receiving it in mind. And so if you have a problem with bad data, or with data heroes swooping in to correct things after the fact, the first place to start is by asking your data customers what they need and how they want to receive it.
Start by asking what format they want to see data in, or if they have to do any manipulation on their end to put your data into their system. Find out if they need to see every metric and measurement you’ve collected in your spreadsheet, or if different teams only need to see a select slice of data to do their roles.
Sometimes when we talk about “bad data”, we’re not talking about a data creator making mistakes with their measurements. Sometimes it’s as simple as these moments of wasted time, when someone in a downstream team has to convert the data they’ve received from one format to another, or pick through a vast database for the few points that matter most to them.
The story of a geology and drilling team disconnect
However, often the fix is done in the systems that the data customer is most familiar with and not the source of truth that the data creator is using, resulting in a company with different figures in different places.
For example the drilling team inadvertently creates data from their drilling software with an error in the well trajectory. This is picked up by the geologists planning the next well – however, to account for this the geology team corrects the error in their own software.
The same drilling data error is picked up by reservoir engineers who correct the error in Excel and the finance team who submit the final AFE update it in their accounting software. Instead of updating at a single source of truth, there are now multiple versions, less trust in data, and because no one has given feedback to the data creators the likelihood that data errors will continue to be generated is high.
You don’t need the latest bleeding edge technical software, sometimes all you need is a data steward
When it comes to creating better data for data customers, people often assume the answer is by necessity technical and therefore the accountability of the IT or data department. They think delivering higher quality data from the source will mean calling on the data scientists and delving into APIs, Machine Learning or building complex data warehouses.
But in reality, the solution starts with understanding that data is an asset and should be treated as such, with accountability for the data quality with the data creators. It’s a question of culture more than technology, and one that needs the right structures, systems and feedback loops in place to make sure that data customers can get their issues addressed at the source. By creating a network of data stewards who are empowered to make changes to source systems, you create a fast feedback network that connects data customers with data creators, and allows for rapid responses.
The story of a geology and drilling team – part two
Take the example of the drilling and geology teams we mentioned above. That situation didn’t just need a technical fix for the drilling team’s coordinate software.
The two functions were also missing a crucial feedback loop between the geologists and the drillers.
When the geologists received missing or incorrect data from the drilling team, it was too much hassle for them to correct the problem at its source – either because that meant updating a drilling system they weren’t trained to use, or because individual members of the geology team would have to reach out to individual drillers with update requests that could take weeks to go through.
When Rockflow worked with these teams to improve their data quality, one measure we put in place was to have a data steward in each team. These stewards were responsible for the quality of data their team created, and also for flagging problems with data they received with its original creators. Given the assigned data stewards were already involved in day to day data work, the change was deemed as being non-invasive with no additional resourcing.
That helped in two ways. One was through weekly meetings with all the data stewards to review data quality together. The other was to make sure that each team had one designated person to go to with data questions, speeding up the process of updating incorrect data at the source.
When you’re bombarded with software solutions to data problems, it’s easy to forget that you still need a good blueprint in place for what you’re doing as a company.
The Rockflow team can help you drill down into the specifics of data management and forecast production, as well as create a broader roadmap for how data quality fits into your firm’s commercial and production strategy.
But more than that, we’ll also meet teams where they’re at. We know that oil and gas companies can’t put their operations on hold for six months while they implement new data management structures and strategies. Where Rockflow can help is with dovetailing new data practices with ongoing operations, so that realigning your approach to data management doesn’t have to mean disrupting production.
For more on how to create value from your data processes, see our podcast on digital innovations in oil and gas with Lewis Gillhespy and Geoffrey Cann, available on Spotify or Apple Podcasts.
