Working with Data Products
With so much hype going around regarding machine learning and AI, it’s obvious for Managers to think that ML and AI algorithms are going to solve everything. We always think about all the cool stuff ML can do for us but, absorbed in that delusion, we forget to ask ourselves an important question: “So what?”
So what if you can tell me if a movie would be a blockbuster based on its characters, screenplay, story, soundtrack, release date, etc, after its made? The money is already been spent. The information regarding the success of the movie is of zero value now. Just because something is cool doesn’t make it useful and Product managers need to stick to the latter side of things.
As Product owners, this is what we should yearn for. Always ask yourself whether the information from the prediction/classification algorithm can give you some advantage. And as Porter describes it, the advantage has to be directly affecting your P&L. It should either increase your revenue(willingness to pay) or decrease your cost.
Based on the above Let’s try to decipher the role of management folks in the development of “useful” data products:
a) Ask the right question: Let’s say LinkedIn builds an algorithm that can predict which person is going to get a new job within the next month based on his/her profile and usage frequency. Do you think that it is valuable? It is good to know but how exactly can LinkedIn’s PnL benefit from this knowledge.
That’s why asking the right question is important. In this case, the right question could have been: Which profile features are more likely to attract recruiters? LinkedIn can use this information and give it to its premium users and help improve their experience and consequently revenue.
Asking wrong questions can also lead to wastage of time and resources for your engineering team. Your stakeholders can get frustrated, which is a criminal offense for a product manager.
So how do we ask the right question?
Simple, Don’t think about what the algorithm can do and start thinking about what you want. In my experience, for most cases, your analysis would not require an ML intervention. But even if it does, the algorithm libraries are so easy to deploy that thinking about deployment from the start is not worth it.
Think about your problem from a pure business lens. “Automation comes after Solution”
For example when Walmart started using data to optimize product positions in its retail stores. The problem would have been: how do we increase the ticket size of buyers? and not, how do we know what items do people usually buy together?
Although the answer to the latter question can solve the first question, you cannot start from there.
Now let’s say we ask the right questions, can we now just hand it over to our genius engineers? No!
While you should be a part of the entire process of model building, there is one more step where the product manger’s role is important.
b) Choose the right data: No one should know your customers better than you. If that’s not the case, you need to start spending more time on that. But in a good case scenario, you would have a clear idea of your customer base, their aspirations, bias, patters, etc. Hence, you are the best person to pick the right sample set for analysis.
for example, if you know customers are not going to spend time on your website during an NFL game, you can pick a sample space that removes or treats that data effectively to remove this outlier characteristic. It’s really hard for a data scientist to make sense of such behavioural patterns from data and this can lead to unreliable outcomes.
We know that in most cases wrong data will give wrong results which might lead to severe delays in product delivery. Making a Product Manager in charge of defining the sample data set is the best bet companies can make while building data products.
Post defining the data: there are multiple steps like preprocessing, model training, validation, and production. While all these steps require inputs from the side of the product manager, they still can’t beat the amount of intervention required in the two steps mentioned above.
The following blog is a summary of a few key concepts I learned while attending a course on designing data products by Mr. Lutz Finger. He is an accomplished author who currently works as a Group Product Manager at Google.