Data Vault: Applying the Story of John Henry
May 9, 2011
John Henry is an American folk hero, notable for having raced against a steam powered hammer and won, only to die in victory with his hammer in his hand. He has been the subject of numerous songs, stories, plays, and novels. He had a 20 pound (9 kg) hammer that he thought was light.
Like other “Big Men” such as Paul Bunyan and Pecos Bill, John Henry also served as a mythical representation of a group within the melting pot of the 19th-century working class. He grows to become the greatest “steel-driver” in the mid-century push to erect the railroads across the mountains to the West. When the owner of the railroad buys a steam-powered hammer to do the work of his mostly black driving crew, to save his job and the jobs of his men, John Henry challenges the owner to a contest: himself alone versus the steam hammer. John Henry beats the machine, but exhausted, collapses and dies.
His story is usually seen as an archetypal illustration of the futility of fighting the technological progress that was evident in the 19th century upset of traditional physical labor roles. Some labor advocates interpret the legend as illustrating that even the most skilled workers of time-honored practices are marginalized when companies are more interested in efficiency and production. Although John Henry proved himself more powerful than the steam-drill, he worked himself to death and was replaced by the machine anyway.
Note: the above excerpt was taken from http://en.wikipedia.org/wiki/John_Henry_(folklore).
So how does this story apply to the Information Technology industry in the 21st century? I find in my travels a similar resistance to efficiency and production, although it would be due to different motives. In this day and age, the most opportune ways to drive down costs over the long term is to automate as much as possible. I still see and talk to technicians that are still trying to build and manage the entire Business Intelligence environment by hand, basically wearing themselves out trying to keep up with maintenance and the rate of change demanded by business requirements.
So what is the root cause of the resistance? I would suggest that there is a comfort level in the technologist in us that entire works hard to reach a level of competence and skills and then, once attaining that level, resists enabling technologies. This is akin to John Henry feeling that he was able or better than a machine in doing a particular job. We, myself included, are comfortable with what we know how to do. Fear holds us back as well, because if we are able to generate the ETL job process, we will be replacing ETL developers with machines. Change requires us to leave our comfort zone and see beyond our own fears, in order to deliver real value to our customers.
In the information technology industry, as it has evolved over time, components move more to a modular or commodity approach as efficiency and refinement processes are applied over time. You see this with software products on the market today that have “wizards” that do routine or repetitive tasks. The more repetitive tasks can be commoditized, the more efficient the design and development process becomes. This saves real time and real dollars on development projects.
So what is one component, among many, to target for this approach? So in the realm of Business Intelligence, there is a pattern based approach to data modeling called the Data Vault ( www.learndatavault.com ). This pattern is manifested in three basic types of structures: hubs, links, and satellites. The nature of these structures allow for a repetitive process to be involved in the loading of the Enterprise Data Warehouse (EDW). For the most part, information is inserted, not updated or deleted. It tracks all changes over time, is integrated, and is auditable back to the source system. From the EDW, the standard data marts can be generated (and in some cases, this can be done with views). Because the pattern is restricted to three basic types, it becomes simpler to generate the loading ETL process using a machine. It is still not a perfect science, and depends on the value of the source model – but the point is the more that can be generated will save time and money over the long term.
So how does the technologist address the “John Henry” in each of us in order to provide the best service to our customers? First, one has to admit that it is good and beneficial to have a machine generate code and processes that were once done by hand. Secondly, one has to be open-minded to realize the benefit. Thirdly, one has to have the courage to take on change, by realizing that inside the change component, there is opportunity for future growth. This is manifested in customers being pleased that high quality projects are produced within a predictable cost and timeframe. By products of this method also reduce support costs over time. It’s a win for the technologist as well as success for the customer.
It’s time to embrace the “machine”, and allow it to do what it does best. The technician needs to move on ahead and add the thought leadership to the projects that will harness the power of smart design with efficient development patterns. The knowledge and knowhow is here today. I know. I have seen it done. I will continue to lay out the design patterns over the next few entries of what it takes to move in this direction and leverage the power of the data vault to bring world class solutions to our clients.
Comments
6 Responses to “Data Vault: Applying the Story of John Henry”
Got something to say?
RT @LUCRUMinc: Data Vault: Applying the Story of John Henry http://bit.ly/mC7iVU
RT @LUCRUMinc: Data Vault: Applying the Story of John Henry http://bit.ly/mC7iVU Check it out: http://thefuturevalueofbusiness.com/
RT @dlinstedt: RT @LUCRUMinc: Data Vault: Applying the Story of John Henry http://bit.ly/mC7iVU Check it out: http://thefuturevalueofbu …
[...] 5. Use a tool or combination of tools to generate ETL code and database structures as well as document. These tools exist on the market today and obviously code generators will consistently outperform even the best coders (see http://thefuturevalueofbusiness.com/data-vault-applying-the-story-of-john-henry.htm). [...]
RT @LUCRUMinc: Data Vault: Applying the Story of John Henry http://t.co/W3z0NkR
[...] 5. Use a tool or combination of tools to generate ETL code and database structures as well as document. These tools exist on the market today and obviously code generators will consistently outperform even the best coders (see http://thefuturevalueofbusiness.com/data-vault-applying-the-story-of-john-henry.htm). [...]