Much of the business world is turning towards Data Driven organizations. By that, they are leveraging their down into assets, to derive insight and produce change. As in more profit, reduce costs and streamlined processes.
With the rise of Big Data, Open Data Sources and Self Service Reporting, a new role has sprung forth to handle the massive amounts of collected data. The Chief Data Officer is one such role. What are some of the issues confronting the Chief Data Officer?
Organizations have always collected data, going back to the punch card days. Since data is now center stage, Data Quality has become very important. The Chief Data Officer is responsible to provide accurate reports based on accurate data. By removing duplicate data, incorrect data and outdated data, the reports can be consumed by staff to run the business. Thus, becoming a data driven organization.
IT has gotten a bad reputation over the years for not meeting the needs of the business. Their reporting efforts were fairly slow, using a first in first out queue system. The reports were not trusted by the business for accuracy. Many times, multiple reports showed different results, leading to rogue developers hired by individual business units. Report maintenance was sometimes an issue, as the report writers hard coded the business logic into the reports, which was undocumented and made changing business rules cumbersome.
Proprietary databases purchased through specific vendors made data integration difficult. Typically, there was no easy way to mash data sets across desperate vendor databases, so data integrated was a big problem. That led to reports on specific segments of the business. For example, Call Center reports would report on calls made to the call center only. They were not tied to the Financial data or the Customer data, making the process of identifying trends difficult. They were not able to see the entire picture as the reports were fragmented by business unit. They may have had some Dashboards at the Executive level, but not for widespread consumption.
Some organizations had report writers who knew an entire segment of the business. Internal staff who knew the business, and could easily write reports with the ability to translate into information. Because the report writer had insights into the data and the business, they sometimes had elevated status, as in the gatekeepers of the information. The business units were at the mercy of the report writer. They may have 50 report requests in the queue, so prioritization was difficult. If the report writer suddenly left the company, that left a gaping hole in the business, because all the business knowledge just walked out the door, stored only in the report writers brain, undocumented.
For a long time, reports were an afterthought. Yes, we just spent a million dollars on the new system, perhaps we should create some reports now. Very common theme for a long time. With the rise of data driven organizations, the need for formal process’ have arisen. Data Dictionaries to document where the data is. Documents describing the business and how data flows from one system to another. Where are the servers located, who maintains them, how often are they patched or recycled? Do we have a formal process to validate the reports prior to release, to ensure accuracy? Can we create a process to report on the data before the month end runs? What system do we use to archive the reports, store off the reports in source code repository, and a very important question, who owns the data?
Sometimes the business wants their reports, and they want them now. In some instances, a user from the business would send the same exact report request to three different people, to see which one would complete first, taking up valuable time and resources. Sometimes the business users would spend more time trying to prove the inaccuracy of the data. Many times, a user makes additional requests to the original, known as scope creep. And more often than not, some users would completely bypass the report queue and go directly to a report writer for a quick report, completely bypassing the queue. Lastly, some users put in dozens of report request, monopolizing the time of the limited IT staff.
Lack of Data Culture
In some organizations, reports are not trusted because the top executives don’t believe in them. They don’t support a data culture. In that, reports are an afterthought. Sometimes the reporting infrastructure doesn’t receive adequate budget. Other times, there’s no collaboration between staff to discuss the reports. Or no mobile strategy for real time reporting. Although Data Warehousing has been around for twenty years, many organizations do not put the effort into a centralized data solution with a single source of the truth. Typically, reporting issues do not appear overnight. They grow slowly over time, snowballing, until they become unmanageable. How does an organization get their house in order?
The Executive leadership team needs to commit to a change in direction on how they handle data. There needs to be a consensus at the top to transform into a “Data Driven” culture. To assist in this endeavor, a new role has emerged to manage the data within an organization, known as the Chief Data Officer or CDO. The CDO typically reports to an Executive and is responsible for all aspects of the data. By that, he or she is tasked with taking a data inventory for every piece of data within the organization's ecosystem. For example, what applications are connected to what database? Who owns each system? Who Maintains the system? Who Archives the system? How often? What data is stored in each repository. What applications pull data from those repositories? What reporting tools are used within the organization? Who are the report writers and data developers? Which departments depend on that data? What are their current reporting needs and future expectations? Where are the systems hosted, On-Premise, the Cloud or a Vendor's site? What documentation already exists?
Last but not least, the Chief Data Officer needs to document the processes for each system.
For example, who does what when? How does the data flow through each of the systems? What are the timing issues and dependencies? Does someone massage the data along the path, data entry? What are the business rules? Are they documented?
Once everything is documented, the analysis can begin. First off, are there any data redundancies. Can we consolidate databases to a single vendor or server to reduce license costs and bigger discounts? Can we migrate some of the data or reporting solutions to the Cloud? Can we reduce the number of Business Intelligence tools to streamline development and costs? Do we have a data quality team in place? Is there a document or WIKI that contains the business logic within the organization? Does it document the flow of data through each of the servers and departments? Should we bring in consultants to get a jump start on a particular domain or technology? Can we leverage internal staff for domain knowledge and expertise through a matrix system by borrowing experts from different departments for specific projects? Do we have a formal process to submit and track tickets for new data requests? Do we have multiple people cross trained in the data and analytics in order to back up the developer while on vacation or if they’re hit by a bus? Are we following “Best Practices” and proper coding standards? Are we promoting the "Data Culture" from the top down, like an octopus having tentacles in every department?
The Chief Data Officer is a fairly new concept. Yet almost every organization has data to be managed, like an asset. By introducing a Chief Data Officer into a centralized department, who's responsible for all the data within the organization, you begin the journey of the data driven culture. Which will accelerate sales, reduce costs and streamline processes. All of which can impact your bottom line.
Once the Chief Data Officer gets the data house in order, the next step will be to leverage your data assets into real insights. Perhaps introduce Big Data to analyze unstructured or semi structured data. Or bring in a team of Data Scientists to do statistical computations, predictive analytics or machine learning. The main goal of the Chief Data Officer is to turn data into an asset by documenting and streamlining processes, managing the data for accuracy, quality and consistency, so decisions made in the organization are based on data.