DataOps-IST: Toolmaking and Delivery

“In God we trust; all others must bring data.”

-W. Edwards Deming

We should all have concerns about the modern business world. We are asking our colleagues to do things they are not capable of doing. We are setting unrealistic expectations for individual performance. As data professionals, we provide the material support, and then neglect to provide the requisite skill and training to take advantage of those resources. We ignore evidence that suggests it is much more difficult to master a new skill than we allow ourselves to believe. We allow managers to repeat buzzwords like ‘analytics’, ‘big data’, and ‘business intelligence’ without providing the infrastructure necessary to handle the weight of those terms.

We can do better. At Pluralsight, we are doing better; DataOps-IST enables this.

DataOps is a domain, by which we can begin to tackle the problems associated with the waves of data and the corresponding undue expectations we thus put upon our users.

There are three primary pillars of DataOps-IST:

In previous posts, we covered the first two pillars. Toolmaking and Delivery is the aspect of DataOps that first provides users with a resource, but along with that, also provides the underpinning of continual monitoring and improvement, which leads to optimal utilization of that resource.

If we are to accomplish the objective of increasing data literacy, analytical skill, and data-supported output throughout the organization, then it is imperative that the Data Team (however it manifests itself in your org) works towards a role that entails less report-making, and more ‘tool’ provisioning. To put it another way, they need to provide the means rather than the ends of the reporting cycle.

And those means include the following: Shared Data Sources that users can access and analyze, Plug-and-Play Report Templates that allow for quick and sensible data visualizations, Data Models, as well as Data Dictionaries that denote agreed upon contexts and definitions of fields, tables, views, etc.

Everything mentioned above, in addition to our previous posts, is only the beginning though. The last component of DataOps is the most important, for it affords us the opportunity of ceaseless evaluation of our previous efforts. It kicks off the invaluable cycle of assessment. How do we determine that the tools and resources we’ve provided our colleagues are of the utmost value and relevance? How do we ensure that our users are leveraging our data to provide the best possible end product?

We study the logs, of course. Logentries provides us the ability to do so. If we are falling short in any particular aspect of the process, our examination of the logs will show us that. Even if it’s to simply identify and remove those legacy reports and leftover dashboards that inevitably take up too much space on your server. The monitoring and reevaluation step of DataOps allows us to get back to the data again, to begin the process anew, and ensure that our colleagues are continually equipped to ‘bring the data’ they need in order to do their work in a meaningful way.




Leave a Reply

Your email address will not be published. Required fields are marked *