DataOps-IST: Toolmaking and Delivery

“In God we trust; all others must bring data.”

-W. Edwards Deming

We should all have concerns about the modern business world. We are asking our colleagues to do things they are not capable of doing. We are setting unrealistic expectations for individual performance. As data professionals, we provide the material support, and then neglect to provide the requisite skill and training to take advantage of those resources. We ignore evidence that suggests it is much more difficult to master a new skill than we allow ourselves to believe. We allow managers to repeat buzzwords like ‘analytics’, ‘big data’, and ‘business intelligence’ without providing the infrastructure necessary to handle the weight of those terms.

We can do better. At Pluralsight, we are doing better; DataOps-IST enables this.

DataOps is a domain, by which we can begin to tackle the problems associated with the waves of data and the corresponding undue expectations we thus put upon our users.

There are three primary pillars of DataOps-IST:

In previous posts, we covered the first two pillars. Toolmaking and Delivery is the aspect of DataOps that first provides users with a resource, but along with that, also provides the underpinning of continual monitoring and improvement, which leads to optimal utilization of that resource.

If we are to accomplish the objective of increasing data literacy, analytical skill, and data-supported output throughout the organization, then it is imperative that the Data Team (however it manifests itself in your org) works towards a role that entails less report-making, and more ‘tool’ provisioning. To put it another way, they need to provide the means rather than the ends of the reporting cycle.

And those means include the following: Shared Data Sources that users can access and analyze, Plug-and-Play Report Templates that allow for quick and sensible data visualizations, Data Models, as well as Data Dictionaries that denote agreed upon contexts and definitions of fields, tables, views, etc.

Everything mentioned above, in addition to our previous posts, is only the beginning though. The last component of DataOps is the most important, for it affords us the opportunity of ceaseless evaluation of our previous efforts. It kicks off the invaluable cycle of assessment. How do we determine that the tools and resources we’ve provided our colleagues are of the utmost value and relevance? How do we ensure that our users are leveraging our data to provide the best possible end product?

We study the logs, of course. Logentries provides us the ability to do so. If we are falling short in any particular aspect of the process, our examination of the logs will show us that. Even if it’s to simply identify and remove those legacy reports and leftover dashboards that inevitably take up too much space on your server. The monitoring and reevaluation step of DataOps allows us to get back to the data again, to begin the process anew, and ensure that our colleagues are continually equipped to ‘bring the data’ they need in order to do their work in a meaningful way.




DataOps-IST: Creating a Culture of Data Analysts

Today’s rapidly evolving workplace provides a context where our decision makers must be able to leverage their data to make optimal judgments. This is not limited to managers and organizational leaders. On the contrary, we are (or soon will be) all data-driven decision makers in our organizations.

The issue at hand is identifying and building an organizational framework that enables people, especially non-technical staff, to use data to become better at their work, to provide more meaningful outputs, and to boost their own sense of personal worth by giving them the ability to measure and analyze their (and their team’s) efforts. The way to accomplish this is through DataOps-IST.

In our previous post, we looked at the importance of understanding your data and how it’s used in your organization. Acquiring that knowledge is a critical first step in developing a data-centered environment. This knowledge underpins the next pillar of DataOps, and that is the social aspect.

Developing the social dimension of an analytics culture is indispensable. Without it, an organization will be left with regrettable and failed technology initiatives, resulting in sunk costs in software, equipment, and other tools. The social dimension is THE forgotten piece of the puzzle. The one that all managers pay lip service to, yet fail to truly develop or understand.

Perhaps the biggest obstacle that stands before an organization that hopes to build a successful analytics and data-oriented culture is the lack of Data Literacy, which begs the question: How do you overcome the lack of data literacy

While there is certainly a multitude of ideas in the tech sphere regarding how we can improve the ability of our people to use data. What’s usually missing from those prescriptions is an understanding of the psychology and social aspect of getting people to engage with their data. Here’s how Pluralsight goes about fostering a sense of data-ownership from everyone:

            Make your data available to everyone in your organization

Allow your people to see the data that affects their work. Allow them to engage with the granularity that comprises the larger processes they participate in. Doing so will make them more informed, more skilled, and more enthusiastic about the ways they can improve the organization.

            Make analytical tools available to everyone

Data doesn’t belong to IT. It doesn’t belong to management. It doesn’t belong to your analysts. It belongs to everyone! Thus, you need to provide tools that promote your data. Those tools need to let your people create their own analyses and visualizations, and then allow them to freely share their data discoveries amongst their colleagues. (For this, Pluralsight uses Tableau and Logentries.)

            Remove the barriers between technical and non-technical staff

It is one thing to figuratively do this, but at Pluralsight, we literally remove physical barriers. This fosters an environment of increased communication and collaboration.

            Whenever possible, remove technical jargon from discussions, meetings, training, and documentation

Non-technical staff are reticent and discouraged from engaging in their data because they, literally, do not know how to talk about it. Question: What is the percentage of people in your organization that know what a data warehouse is? Or an ETL job? Or Big Data? Or SQL? Or Data Science? Or even analytics, itself? On the contrary, what is the percentage of people in your organization who are affected by these ideas and products? The latter number is obviously much higher than the former. These terms are ubiquitous is the tech realm. Yet, they are not nearly so in the rest of the business world. If we truly want our people to take ownership of their data, to analyze it, and to make decisions based on it, then we need to be able to speak differently to each other about it.

            Encourage all staff to engage in opportunities to develop their data skills

You must continually assess and improve the data skills of your employees. Does your staff know the basics in data manipulation, statistics, and data visualization? In order to achieve a viable ROI on analytics tools (such as Tableau), users must be able to successfully leverage the features that a tool provides. For example, we routinely provide training (live and archived video) to all staff. We then provide shared data sources that everyone in the company can access and analyze. Transparency of data goes hand-in-hand with the development of a strong data-centered culture.

            Recognize and reinforce any and all data curiosity

Every time an employee says something like, “I wonder how many times X happened before Y occurred,” we recognize this as an opportunity for data exploration and analysis, as well as an opportunity to encourage subsequent and ongoing research.

            Do not rely on an “Intuitive UI to engage your people

System-based enhancements and optimizations are prerequisites for DataOps and data literacy, not a panacea.

            Leadership buy-in and fluency

From the CEO to our front-line managers, our leadership uses data to support and validate their decision making processes. When there is data to be had, they believe in discovering the truth, not creating it based on their intuition or hunches.

The so-called analytics skills gap is a very real thing. We have endless data, and not nearly enough skilled people to create value from it. Employing DataOps strategies fills this gap by fostering an environment of social engagement with data. The solution to the skills gap problem will not be found in traditional education, standard BI strategies, or from systems themselves. Instead, it will be found when we encourage the data-curious among us to take ownership of their data, improve their own skills, and become analysts themselves.


This post was originally published on in May 2015.


What is DataOps-IST & Why Do You Need It?

In the technology world, things change quickly. It’s common knowledge, so we don’t feel there’s a huge need to convince people of the fact. Often times, there are buzz words which surface and catch on like rampant wildfire: there’s no escaping it. Some of these continue on and become verifiable domains and others wither and are quickly extinguished.

One word (or domain) which has certainly taken root in an abundance of phenomenal technology and devices is Data; big, small or any size between, it’s simply data and we have plenty of it. The theme that follows is our belief that Data needs it’s own domain and cannot exist without a solid foundation in, well, Data.

There are domains which serve to enhance and support the DataOps realm. Some examples: IT, Development and Engineering are a few that, in some form, support a piece of the larger DataOps domain. Companies abound with Analysts and reporting experts but there is no specific focus on the merit and value of Data as a whole. We propose that without a focus on Data, businesses will fail to realize the bigger picture in which each piece of the organization connects to the other.

DataOps, and more specifically, DataOps-IST is simple and involves three foundational pillars:

  • Infrastructure: Data and an appropriate Analytics Infrastructure (Tableau in our case)
  • Social Engineering (and Research)
  • Toolmaking & Delivery

DataOps (1)

Between each pillar, there are the obvious feedback loops wherein each connects to the other and serves to enhance and support the domain. Around the domain exists a logging and monitoring aspect; we want to know how well and to what extent we are serving our customers. For this function, we use Logentries to answer questions such as:

  • Who is using our particular report and what are the load times?
  • Where are there gaps in performance?Is it hardware or report related?
  • Can we potentially remove parts of a report/dashboard because of performance?
  • Can we, in real time, monitor a new report and quickly (also in real time) update parts which are not performing?

Again, the domain monitoring is critical since it enhances how we discuss progress and future enhancements with customers.

In the DataOps domain, there is a vast opportunity, mostly because of the natural growth of other fields, to firmly establish a pattern of analytics and data in any organization (big and small).   We are in a transitional time where the technology has promoted the growth of analytics as a separate, yet, equally foundational part of any company.

Look for further updates on the topic. We’ll start to discuss the specifics of each pillar and why it’s important to leverage them correctly, not to mention the functionality which makes each uniquely DataOps-oriented.



 This post was originally published on in April 2015.