Setting minimum acceptance level for AI

A question I have been asked often, “Should we aim for 70% or 80% or 90% accuracy with our prediction rule?” Like most questions in life, the answer is “It depends”. The most important question to ask here is “what is the cost of making a wrong prediction?” For example, if you are compiling a list of leads with a high probability of conversion, the answer might depend on what you plan to do with this list. If this list is used by marketing team for mass events, may be a low accuracy prediction rule might suffice. But if you are qualifying leads towards quarter end for last minute “sales chase”, you might want AI that is much more accurate. Another question that might be worth asking is “What am I replacing with this prediction rule?” Let’s assume it is a human making that decision today. Having an acceptance criteria that just matches human accuracy might be good enough as it will at a minimum relieve people from the stress of decision making. With the right mechanisms in place, AI should learn better over time. That should help improve it’s accuracy. And with that expectation in mind, it might be ok to start with lower accuracy estimates. So be pragmatic and reasonable with the expectations set from AI in terms of accuracy (or any other measure of success), and you will be one step closer to achieving success in practical AI. #abhayPracticalAI #artificialintelligence #machinelearning #ai