Why the highest-quality data produces the highest-quality results | Peak Indicators

Actively addressing data quality should be a primary part of any council’s data and analytics strategy.

Data is increasingly being used to drive better decision-making, and to help councils deliver improvements to its services. Often this feeds into analytics and data science activities, such as producing reports and dashboards, or driving predictive analytics.

However, this means the data you use needs to be trustworthy and reliable. In other words you need good quality data. As the old saying goes ‘Rubbish In, Rubbish Out’.

But in practice what does this mean? The notion of data quality covers many themes and has been widely studied and written about in theory and practice. At the heart of it, data quality revolves around two key ideas: is the data you have fit for purpose and how well does it match the real-world.

Download The Full Public Sector Playbook

Why Focus On Data Quality?

For some, data quality may be viewed as an add-on or nice-to-have attribute, but that’s incorrect. Data quality is central to good data management and governance.

Poor quality data could show itself as missing or incorrect values, inconsistent data formats, duplicated data across sources, violation of business rules, inaccessible and insecure data, and factually incorrect or biased data. Common causes of poor quality data include incorrect data entry, system errors and poor data management.

And the impact of poor quality data? Well it can lead to poor productivity, unreliable decision-making and missed opportunities. Data quality can also impact ongoing operations and future strategy. Even worse, failing to manage data quality could lead to non-compliance, being fined and reputational damage.

But reaching ‘good quality’ is not always easy. You need to understand how the data is failing to meet your needs and why.

What Makes Good Quality Data?

Data quality can be captured according to various characteristics or dimensions that help to capture whether data is fit for purpose and matches the real-world. Commonly used dimensions include:

• Completeness — Are all the required values recorded in full?
• Validity — Does data conform to the syntax of its definition (is it formatted correctly)?
• Uniqueness — Is anything recorded more than once?
• Timeliness — Is the data recent and accessible fast enough to be useful?
• Accuracy — Does the data correctly describe a real-world object or event?
• Consistency — Is the data consistent across systems?

A data quality metric is a formula that produces a numerical value reflecting quality for each dimension (e.g. the number of missing values in a table might be a measure of accuracy and completeness). By using metrics you can report the quality of individual attributes in data sets across the business.

This also means you can define ‘acceptable’ levels of quality. Remember data quality is often considered within the context of its intended use — so being fit for purpose doesn’t always mean being perfect in all the attributes all of the time. For example, in some cases having missing values may be acceptable if these are not important for a specific use case.

Download The Full Public Sector Playbook

‘I know what I want, how do I get there?’

A typical data quality process will start by understanding what data an organisation holds and what purposes it’s being used for. This can often be challenging, but auditing and

cataloguing datasets can help.

Identifying attributes of data quality through profiling and mapping out data journeys for specific uses will help you draw up some ideas around what good data quality might mean, both generally and for specific uses, where the issues might be and what steps will need to be taken.

This is often considered at every stage of the journey within a typical data lifecycle — when creating, processing, analysing, preserving, accessing, and re-using or destroying the data.

This starts with how data is gathered. The biggest cause of poor quality data during collection are errors in manual data entry. This could be by customers or employees.

One way to minimise these is through validation mechanisms, such as not letting users proceed until all required fields have been filled in a proper format.

This brings up a key problem with addressing data quality — it’s a people problem not just a technological one. We can use tools to fix technical problems, but we also need to change the culture and business processes.

By establishing a strong culture of data governance (and by extension data quality), people will spot data issues more easily, understand the impact of problems on the business fully, and work to address them.

As well as the culture, any organisation must establish rules (based on the attributes they want their data to have) to ensure it keeps its quality through the data lifecycle. There are lots of open-source tools and data analytics companies that can help with this.

The final step is monitoring and reporting. Ensuring data quality is an ongoing process. Not only are you always collecting new data, but the quality of data you do have could change as you use it for different purposes. Data quality dashboards for specific jobs, functions and departments, could be produced to measure quality and ensure its fitness for purpose.

Data Quality In Local Government

Good quality data is imperative in local government. The Audit Commission describes it as an “essential ingredient in accountability, user choice, reliable performance and financial information”. For councils, poor data quality can cause mistakes and delays in service provision, unnecessary costs at operational and strategic levels, and ill-informed policy decisions.

Central government is promoting the importance of data quality and providing guidance around its management. For example, the UK Government’s National Data Strategy discusses data quality and technical barriers to use and re-use across government. The new Government Data Quality Hub and Data Quality Framework are two example initiatives to support government and the wider public sector’s use of data.

Yet despite these initiatives, it can still be very challenging for local authorities to develop the right frameworks and processes to properly govern data quality. Fortunately, there is a growing supply of literature and examples written by different authorities that can help (like this policy document from West Suffolk Council).

Part of the journey may be assessing your data and its handling using data maturity frameworks. For example, the Local Government Association has developed an online data maturity self-assessment tool that allows you to respond to a number of questions to determine your organisation’s maturity in its collection, management and use of data.

Data quality is hard to deal with. It escapes clear definition. Achieving it requires investing in technology and people. And it is an ongoing process. However, the benefits are clear — if you want to become a data-driven council then the data you stand on must be rock solid and fit-for-purpose. There’s no other way.

If you would like to find out more we’d be happy to discuss this topic with you, simply give us a call or drop us an email to set it up!

Download The Full Public Sector Playbook

Originally published at https://www.peakindicators.com on December 23, 2022.