Putting your baby to bed :-(
July 10, 2008
Well, not a real breathing, baby food eating, diaper filling baby anyway. What I’m talking about it is retiring a system that you were also responsible for creating and implementing.
In the beginning, there was a problem. The problem was the client’s inefficiency in closing the financials and reporting said financials out to the masses. Enter LÛCRUM. LÛCRUM has a long history of solving complex data problems. LÛCRUM delivered an enterprise data warehouse that allowed integration of various financial systems into a single repository with a common vocabulary that facilitated timely financial reporting to the organization.
Fast forward 4+ years. The data warehouse keeps on chugging. It performs monthly financial closing processes admirably. Data is still integrated from numerous sources. People still depend on the “numbers” that come from their financial reporting. Enter “change”. The client has been acquired by another company. The buying company also has a data warehouse and decided to integrate the two together.
Before we can decommission the data warehouse, the financial processes need to be migrated. The buying company handles the development aspects of the migration to the new architecture and BI toolset; however the existing support team is the SME with respect to the existing system. Discussing, detailing, and discussing again the AS-IS requirements with the development staff explaining what has to be re-created, the certain business nuances to be aware of, performance and tuning considerations, and other “why’d you do it that way” discussions.
As with all systems reviewed in retrospect, you’re going to find some things that you’d might change if you could have a do-over, however for the most part, LÛCRUM’s EDW has stood the test of time and required very little enhancements/rework. Here are some observations that I believe resulted in a stable, high performing and well accepted data warehouse:
1. Implemented Ralph Kimball’s methodology.
2. Shielded the data warehouse from source system changes based upon SLA’s between the two parties. When the source systems underwent changes, they were responsible for maintaining a standard data submission format.
3. Shielded the BI tools from the underlying database tables. All interaction was through database views thus allowing the data warehouse to make modifications along the way that minimized downstream impacts.
4. Utilized an enterprise job orchestration tool. This allowed for interaction of various upstream and downstream systems in a packaged application capable of communicating with various system platforms.
5. Due to large data volumes, special attention was made to insure high performing interaction with data.
6. All “heavy” lifting of data was performed at ETL time resulting in simple SELECTs from the BI toolset. This prevented the client from being tied to a BI toolset.
7. An online meta data repository was utilized to allow users to understand what data was in the data warehouse and/or what a particular business term meant.
8. Visual production support processes were implemented that permitted the support team to be proactive in working through issues rather than being reactive to problems.
9. Extensive testing was performed during development and implementation. Unit, String, Regression, Performance, and User acceptance. Almost an obnoxious amount of testing, but well worth it in the end.
10. LÛCRUM built it! (Come on, you saw that coming didn’t you?)
Next steps are the decommissioning project. We are figuring out what data needs to be archived for potential future reference. Where will this subset of legacy data reside? How might it be accessed? What dependencies exist with the server architecture? What parties need to be involved (server operations, UNIX team, storage team, dbas, etc.)? How quickly can it be done?
All good things must come to an end. I am proud to say that I was a part of this initiative at the start as well as at the end. Our solution solved a problem. Now it’s someone/something else’s turn to reach up to the bar that has been set.
Time to start singing a lullaby…
Dave
Comments
2 Responses to “Putting your baby to bed :-(”
Got something to say?
Polls
Loading ...Latest Headlines
Recent Comments
- Christie Bautch on A Secret from an Old Man
- Jodie on A Secret from an Old Man
- Jodie on When You Have Trust
- Guy on A Secret from an Old Man
- Steven McWhorter on A Secret from an Old Man
WOW!
Dave, that was very insightful for a newbie like me. I like your writing style.
I love and hate number 9! Its so vital to the sucess, but this rigor of testing is brutal. It’s way under the water level of the iceberg - we see it pay off in quality of data/information, other than this, its ‘unappreciated’ to folks who want it now!
Number 7 is also key, as this is where understanding is coordinated!
Feel good, Dave - This delivered excellent value for more than 50 months! I am sure the buying company will ’school off of you guys’ and take those hard lessons to heart… HOME RUN!
~Scott