Data Contracts: Developing Production-Grade Pipelines at Scale

Chad Sanderson, Mark Freeman

Poor data quality can cause major problems for data teams, from breaking revenue-generating data pipelines to losing the trust of data consumers. Despite the importance of data quality, many data teams still struggle to avoid these issues–especially when their data is sourced from upstream workflows outside of their control. The solution: data contracts. Data contracts enable high-quality, well-governed data assets by documenting expectations of the data, establishing ownership of data assets, and then automatically enforcing these constraints within the CI/CD workflow. This practical book introduces data contract architecture with a clear definition of data contracts, explains why the data industry needs them, and shares real-world use cases of data contracts in production. In addition, you’ll learn how to implement components of the data contract architecture and understand how they’re used in the data lifecycle. Finally, you’ll build a case for implementing data contracts in your organization. Authors Chad Sanderson and Mark Freeman will help you: Explore real-world applications of data contracts within the industry Understand how to apply each component of this architecture, such as CI/CD, monitoring, version control data, and more Learn how to implement data contracts using open source tools Examine ways to resolve data quality issues using data contract architecture Measure the impact of implementing a data contract in your organization Develop a strategy to determine how data contracts will be used in your organization