OK, you’re about to launch your company’s first AI pilot focused on travel and expense management. You’ve carefully vetted the providers, chosen an important, high-potential area of focus, and lined up the business units to test in. What could possibly go wrong?
Short answer: pretty much everything.
Less snarky answer: there are a few key areas that can make or break the success of your pilot. The stakes are high, not only for this project but as an initial test of AI itself. Here are a few critical issues to nail down right from the start.
1. Ensure Data Quality and Relevance
Data is the lifeblood of any AI system. Ensuring that your data is relevant, substantial, cleansed, and normalized is essential for the success of your AI pilot.
Relevance: Make sure the data you use is directly related to the problem you’re trying to solve. For instance, if you’re piloting an AI tool for expense management, use data from past expense reports, including categories, amounts, dates, and notes.
Volume: Ensure you have a large enough dataset to train and test the AI effectively. A small dataset may lead to biased or inaccurate results.
Cleansing and Normalization: Cleanse your data to remove errors, duplicates, and irrelevant information. Normalize the data to ensure consistency, such as using a standard format for dates and currency.
Compliance and Privacy: Verify that your data handling practices comply with relevant regulations, such as GDPR or CCPA, and ensure the privacy of sensitive information.
2. Define Clear, Measurable Goals
People tend to think AI is a cure-all and will solve all related issues at minimal cost. Do the reality check early by limiting scope and expectations more narrowly. Be sure to have a small set of success measures with a baseline measure of current performance to improve upon. These generally fall into three simple categories with one or two metrics per category, max:
Customer Impact: Net Promoter Score or satisfaction, conversion rate/abandoned carts
Operational Impact: Number of touches, average handle time, quality (error rate or rework)
Financial Impact: Cost per transaction, ROI, reduction in debit memos, refunds or penalties
3. Engage Stakeholders Early and Often
AI, whether it is machine learning or generative AI, depends on training the model through continuous iterations. While more and more of this is done programmatically, be sure to get some old-fashioned human intelligence to balance the artificial kind. Talk to your users, tech teams, internal and external business partners with a mix of specific questions and free form feedback. The tools are so flexible and fast, that this input can be acted on very quickly. That means you can track the impact of changes along the way, compressing multiple enhancements into a single pilot. This gives the added advantage of seeing the impact of the feedback in your metrics.
Feedback Loops: Create formal mechanisms for continuous feedback from users. This can include regular check-ins, in-app feedback, and focus groups.
Informal Feedback: 1:1 casual check-ins will provide deeper insights and give a feel for the emotional as well as technical impact.
Working these approaches into your pilot complements all the rigor of piloting: careful definition of the problem to be solved, technical definitions, identifying the pilot, project execution and agile development throughout the test. And they can give your pilot the boost it needs to get the best results and create momentum for the broader rollout.