The Great Expectations Of Data

You might be able to tell from the title of this post that I’m still under the influence of Dickens. This post is not about Pip and his development over the years neither is it about the data testing tool that goes by a similar name.

So you are a new data leader in the company and the red carpet has been rolled out to welcome you. Spirits are high on both sides as the company has finally got their ideal candidate who ticked all the boxes in a laundry list of technologies, skills and experiences and on your side you are happy that you don’t need to navigate multiple interview rounds (at least for the next couple years) and it’s a new experience - new team, tools and problems. Exciting!

You spend the next couple months talking with business stakeholders, understanding the business operations and problems in the data ecosystem of the company, establishing partnerships internally and externally and you identify a peculiar opportunity for data - that rare low hanging fruit that brings maximum lift. You set all your ducks in a row (forgive me, this is the last corporate consultant-speak in this post) and you successfully kick off the project. Invigorating!

Amidst all the clinking of glasses during the fanfare, you experience a very familiar feeling when a key business stakeholder mentions that he has shared with the board and executives that this new data project will solve all the problems in his department and transform his whole operations to enable him double his targets - a data great expectation. Everyone around the table raises their glasses to this new silver bullet but your glass suddenly feels heavy as you try to join in the toast. Anxiety!

It’s not unusual for business stakeholders to have great expectations of data projects - a new data warehouse or predictive model or analytics project. In fact, despite the angst that often comes with it, I believe most data leaders will take a work environment that has such expectations over one that has no expectations of data. Often this great expectations always sneak up on you and evoke some degree of nervousness, apprehension and worry. So how do we deal with data great expectations and manage them in a way that will not douse the business stakeholder’s appetite for data and also not create panic for the data team?

Get out of the Ivory Tower

Most often, the data team is seen as a set of smart guys in their ivory tower creating magical portions. This is great and it works in companies whose data teams are just for research purposes only. In other companies where data teams are expected to solve business problems and drive data-driven decision making, staying in the ivory tower often leads to a disconnect - the data team not knowing much about what’s happening in the business and the business stakeholders thinking the magic portions being developed in the tower will make all the problems disappear. A good starting point to get out of the ivory tower is to review the data organisational model as some models make it easier to form impactful collaborations with the business.

Deliver Value in Short Cycles

It’s not uncommon for data projects to take months and even years. For example, most of us might have seen data warehousing projects take 2-3 years before it is queried or a predictive model take 18 months before it is put in production. What I’ve noticed is that the longer you take before delivering a data project the greater the expectation increases and this increase is exponential with time. Also, the requirements might have changed or the business might have evolved over the last 2 years and you deliver something that will not meet current expectations. So think of how to deliver smaller chunks that create bsuiness value, you can iterate over in short cycles (in the order of 2-4 weeks) and get constant feedback. The best feedback is often given after usage. So try to get something valuable into their hands as quickly and frequently as possible and keep on refining.

Broaden your Engagement Strategy

Many data leaders often talk about not being invited to the table. For that, I say, set your own table and invite the business. This can take the form of organising demos, starting your own Slack or Teams channel to create a platform of bi-directional communication, involving stakeholders in sprint plannings and retrospectives and any other thing that will bring the business into your world. This helps bring clarity and transparency that often helps curb the sprout of a great expectation.

Furthermore, I believe most data leaders have frequent one-on-one’s with business stakeholders as part of their engagement strategy. The discussions in these sessions often centre around problems and impact of data deliveries. What is missing from the this conversation most times is what the data team is not working on. Some times, when we talk about the scope of a project, the boundary markers make sense in our heads but another person might place the boundary marker somewhere else. So it’s important to explicitly state out what is not in scope. Going further to write it out goes a long way in achieving this.

It’s not absurd nor ludicrous for the business stakeholders to have great expectations and it shouldn’t be responded to as such. I believe with the right approach to development and stakeholder engagement, data leaders can manage these great expectations of data and reduce the angst that often comes with it.

Thanks for reading.