In today’s fast-paced landscape, where agile development and cloud-native platforms dominate, data modelling might seem like a relic of the past. But for anyone building business-critical applications or modern analytics platforms, it remains a cornerstone of success.
Modelling for OLTP: The Backbone of Business Applications
Online Transaction Processing (OLTP) systems power the day-to-day operations of businesses, from CRM platforms to e-commerce engines. These systems demand high performance, reliability, and scalability. A well-structured data model ensures:
- Efficient queries and indexing
- Reduced redundancy
- Lower cloud costs
Without proper modelling, databases can become inefficient, leading to performance bottlenecks and increased operational expenses. The physical model must reflect business logic, even if that logic is implemented in code. Otherwise, the database becomes a black box hard to maintain and expensive to scale.
Agile Development: Who Owns the Model Now?
Traditionally, data modelling was the domain of architects and DBAs. They crafted conceptual, logical, and physical models in a waterfall sequence. But agile methodologies have, in my view, shifted the responsibility for the physical model to the developer/engineer.
Today, developers often modify schemas directly, bypassing formal modelling processes. While conceptual and logical models may still be created, they’re rarely updated as development progresses. This leads to “model drift” a disconnect between design intent and implementation. The more I think about this the more I see the conceptual and logical model being owned by the architect or DBA and acting as part of the specification or user story. The developer building the application will reference as they iterate over the build out of the application and the physical data store will morph with this.
Some teams argue modelling slows innovation. Others, including myself advocate for a hybrid approach, involving data professionals earlier in the cycle to prevent costly mistakes. The key is collaboration: modelling should be a shared responsibility across roles, it is part and parcel of the DevOps collaborative culture.
Modelling for Analytics: Start Before You Build
When looking at the analytics side of data, whether you're building a Lakehouse, data lake, or data warehouse, in my view, modelling must come first. Understanding the questions we need to answer and where that data comes from, gap analysis and where the data comes from is key to a successful build out. Without modelling we are more likely to end up with our data swamp.
Trying to run analytics directly on transactional databases leads to performance issues and resource contention. Modern platforms like Databricks, Amazon Redshift, Google BigQuery, Snowflake, and Microsoft Fabric offer powerful tools, but they don’t eliminate the need for modelling. They demand it.
Dimensional modelling fact and dimension tables, slowly changing dimensions, surrogate keys are the foundation of scalable, reliable analytics. Without it, data pipelines become brittle and reporting becomes inconsistent.
The Renaissance of Data Modelling
Far from fading, data modelling is experiencing a revival. With tools like Redgate Data Modeler making modelling accessible to analysts and engineers not just DBAs.
Modelling is no longer just about database design. It’s about communication. A good model is a shared language between business stakeholders, developers, and data professionals. It gets everyone on the same page, clarifies requirements, and reduces rework.
This is going to become more important as more businesses look to get value from their data by using it in conjunction with AI. The adage that garbage in means garbage out is very much the case here and anything that we can do to prevent having to clean up data to work with AI means that we can get to value faster.
Final Thought
In a world obsessed with speed, data modelling offers clarity. It’s the blueprint that ensures our digital infrastructure is scalable, secure, and aligned with business goals.
Agile development may have changed who owns aspects of the models, but it hasn’t changed why we need it. And as data continues to grow in volume and value, the organizations that invest in modelling, early and often, will be the ones best positioned to thrive.
I’m really interested to understand what you are thinking on this and what you see as you build and manage database platforms today.