Springboard your AI journey through our Data Product approach

The challenge

• Users complaining that they don’t have access to data they need
• Widely useful data locked away in silo/legacy systems
• Multiple teams using unauthorised data and building overlapping pipelines

Unser Ansatz

• Identified 100+ candidate data products from across the organisation and prioritised by most requested
• Refined data products with business stakeholders to understand format, access and data privacy
• Setup a cross functional agile delivery team that included metadata governance analysts, data quality stewards and data engineers
• Delivered 30+ Data Products - Captured metadata, created recuring data quality rules, engineered pipelines, setup user access request processes, enabled reuse by making data products discoverable in the data marketplace

Key results

• 100+ daily data marketplace users
• 200+ and counting, user connections into data products
• Reduced widespread duplication of effort by establishing single authoritative data sources
• Organisation wide data marketplace to helps to reduce future effort duplication

Our client, in a regulated industry with strict metadata obligations to meet, had bought a new metadata tool and didn’t have knowledge of the new suite of capabilities. They needed to demonstrate tangible progress to the regulator by the next reporting period.

Projective Group was initially brought in with a mandate to: deliver metadata training in the new tool, perform a current state assessment, put together and implement a best practice metadata management approach using a combination of existing and new tooling. Projective Group brought in experts in the required fields as and when necessary and embedded metadata best practices in every layer of the data lifecycle.

A Data Product approach

Off the back of successful metadata approach setup and implementation across the organisation, Projective Group was given the opportunity to extend the approach to include data engineering with the aim of embedding governance in the pipeline layer. Projective Group led the creation of a dedicated agile cross functional delivery team which included newly trained metadata governance analysts and data quality stewards working alongside existing data engineers.

The newly created Data Product team identified data requirements that would generate the most value from a Data Product approach. The team targeted data that was most widely used across the organisation and prioritized data product creation that would reduce the most amount of unauthorised copies and/or pipeline duplication. To create and setup data products leveraging existing IT infrastructure alongside the metadata management tool. Culminating with publishing data products to the organisation wide data marketplace, achieving the additional benefit of reducing the likelihood of future pipeline duplication and erroneous copies of data.

This engagement marked more that just regulatory metadata compliance. It demonstrated wide business value from embracing regulatory requirements and technological efficiency from separate storage and compute allowing a single copy of a data product to be accessed and processed many times.

Conclusion

When organisations face regulatory pressure, with our approach, can be turned from burden to value realisation. Now with the burgeoning AI landscape, it’s increasingly clear that for organisations to make the best use of AI they need well governed data to feed AI making data products the perfect launchpad for AI initiatives.

Über Projective Group

Established in 2006, Projective Group has deep historic expertise in metadata, regulatory compliance and technical tools.