Join our virtual tour to learn how WhereScape...
Data Warehouse Development
By WhereScape
| April 16, 2020
Dr Barry Devlin is among the foremost authorities on business insight and one of the founders of data warehousing, having published the first architectural paper on the topic in 1988. Today he is a leading consultant and speaker on data warehouse development.
Barry has published a number of articles for WhereScape, to follow you will find a synopsis and introduction to some of these. Each has a link to the full blog so you can explore that specific subject in more detail.
Designing a Data Warehouse
Always keep in mind the basic goal of your project: to deliver a cross-functional, long-life foundation for data provision and decision support. Data warehouse development project types vary and will continue to mutate over time with requirements that you cannot predict now, and your data warehouse must continue to provide accurate data throughout this evolution.
This blog explains how to:
- Use templates to save time and money rather than building from scratch
- How to define and refine the logical structure of relational tables
- Choose which approach of data modelling is best for you – 3NF, Star Schema, Data Vault etc.
Building a Data Warehouse
This blog explains how every design is only as good as the reality of its source systems, their missing data and poorly defined data structures. The finished design is always a balance between the vision for the model and the constrains of the sources. The article covers:
- The five steps to follow when building a data warehouse
- How Data Warehouse Automation can help
- How to move from design to build
- Building a Data Vault with WhereScape Data Vault Express
Operating a Data Warehouse
This blog explains how to deliver your data warehouse successfully to the business and run it smoothly on a daily basis. We must avoid the problems of past ad hoc data warehouse development approaches that combined manual and semi-automated methods, and adopt advanced data management and automation practices. Find out how:
- Deployment needs to be treated as a long-term, monogamous relationship
- To address issues such as packaging and installation of the code
- To bundle sets of objects and transport from dev to QA and through to production
- To handle interdependencies between the data warehouse, data marts and data lake
- To automate the historical information that tracks performance over time
Maintaining a Data Warehouse
In some development projects, once a piece of software is up and running it needs only minor bug-fixing, but maintaining a data warehouse needs more attention than that. The nature of creative decision-making support is that users are continuously discovering new business requirements, changing their mind about what data they need and thus demanding new data elements and structures on a weekly or monthly basis. Indeed, in some cases, the demands may arrive daily! Read this blog to find out:
- What a data lake should and shouldn’t be used for
- Why and how a Data Vault gives more agility in the maintenance phase
- The role of metadata in data warehouse maintenance
- How to predict downstream impact of changes from automated documentation
What is Data Fabric? A Smarter Way for Data Management
As of 2023, the global data fabric market was valued at $2.29 billion and is projected to grow to $12.91 billion by 2032, reflecting the critical role and rapid adoption of data fabric solutions in modern data management. The integration of data fabric solutions...
Want Better AI Data Management? Data Automation is the Answer
Understanding the AI Landscape Imagine losing 6% of your annual revenue—simply due to poor data quality. A recent survey found that underperforming AI models, built using low-quality or inaccurate data, cost companies an average of $406 million annually. Artificial...
RED 10: The ‘Git Friendly’ Revolution for CI/CD in Data Warehousing
For years, WhereScape RED has been the engine that powers rapidly built and high performance data warehouses. And while RED 10 has quietly empowered organizations since its launch in 2023, our latest 10.4 release is a game changer. We have dubbed this landmark update...
The Assembly Line for Your Data: How Automation Transforms Data Projects
Imagine an old-fashioned assembly line. Workers pass components down the line, each adding their own piece. It’s repetitive, prone to errors, and can grind to a halt if one person falls behind. Now, picture the modern version—robots assembling products with speed,...
The Role of Clean Data in AI Success: Avoiding “Garbage In, Garbage Out”
Co-authored by infoVia and WhereScape Artificial Intelligence (AI) is transforming industries across the globe, enabling organizations to uncover insights, automate processes, and make smarter decisions. However, one universal truth remains: the effectiveness of any...
What is a Cloud Data Warehouse?
As organizations increasingly turn to data-driven decision-making, the demand for cloud data warehouses continues to rise. The cloud data warehouse market is projected to grow significantly, reaching $10.42 billion by 2026 with a compound annual growth rate (CAGR) of...
Simplify Cloud Migrations: Webinar Highlights from Mike Ferguson
Migrating your data warehouse to the cloud might feel like navigating uncharted territory, but it doesn’t have to be. In a recent webinar that we recently hosted, Mike Ferguson, CEO of Intelligent Business Strategies, shared actionable insights drawn from his 40+...
2025 Data Automation Trends: Shaping the Future of Speed, Scalability, and Strategy
As we step into 2025, data automation isn’t just advancing—it’s upending conventions and resetting standards. Leading companies now treat data as a powerful collaborator, fueling key business decisions and strategic foresight. At WhereScape, we’re tuned into the next...
Building Smarter with a Metadata-Driven Approach
Think of building a data management system as constructing a smart city. In this analogy, the data is like the various buildings, roads, and infrastructure that make up the city. Each structure has a specific purpose and function, just as each data point has a...
Your Guide to Online Analytical Processing (OLAP) for Business Intelligence
Streamline your data analysis process with OLAP for better business intelligence. Explore the advantages of Online Analytical Processing (OLAP) now! Do you find it hard to analyze large amounts of data quickly? Online Analytical Processing (OLAP) is designed to answer...
Related Content
What is Data Fabric? A Smarter Way for Data Management
As of 2023, the global data fabric market was valued at $2.29 billion and is projected to grow to $12.91 billion by 2032, reflecting the critical role and rapid adoption of data fabric solutions in modern data management. The integration of data fabric solutions...
Want Better AI Data Management? Data Automation is the Answer
Understanding the AI Landscape Imagine losing 6% of your annual revenue—simply due to poor data quality. A recent survey found that underperforming AI models, built using low-quality or inaccurate data, cost companies an average of $406 million annually. Artificial...
RED 10: The ‘Git Friendly’ Revolution for CI/CD in Data Warehousing
For years, WhereScape RED has been the engine that powers rapidly built and high performance data warehouses. And while RED 10 has quietly empowered organizations since its launch in 2023, our latest 10.4 release is a game changer. We have dubbed this landmark update...
What is Data Fabric? A Smarter Way for Data Management
As of 2023, the global data fabric market was valued at $2.29 billion and is projected to grow to $12.91 billion by 2032, reflecting the critical role and rapid adoption of data fabric solutions in modern data management. The integration of data fabric solutions...
Want Better AI Data Management? Data Automation is the Answer
Understanding the AI Landscape Imagine losing 6% of your annual revenue—simply due to poor data quality. A recent survey found that underperforming AI models, built using low-quality or inaccurate data, cost companies an average of $406 million annually. Artificial...
RED 10: The ‘Git Friendly’ Revolution for CI/CD in Data Warehousing
For years, WhereScape RED has been the engine that powers rapidly built and high performance data warehouses. And while RED 10 has quietly empowered organizations since its launch in 2023, our latest 10.4 release is a game changer. We have dubbed this landmark update...
The Assembly Line for Your Data: How Automation Transforms Data Projects
Imagine an old-fashioned assembly line. Workers pass components down the line, each adding their own piece. It’s repetitive, prone to errors, and can grind to a halt if one person falls behind. Now, picture the modern version—robots assembling products with speed,...