An AI trial can produce useful results and still stall when the team wants to put it into regular use. At that point it has to be clear who can approve the next step, what data is permitted and who deals with errors. Those questions are easiest to settle while the application is still manageable.
These are the seven decisions I would bring to the table before a trial becomes part of everyday work.
1. What improvement are we looking for?
Choose a specific objective, such as reducing the time spent gathering information for a case file. Establish how the task is handled today and how you will assess the change. Include the effort spent checking and correcting results. Producing a draft faster does not necessarily shorten the whole process.
2. Who is responsible for its use?
Name someone with authority to decide the application’s purpose, permitted use and continuation. Depending on the task, that person may work in IT. Agree who maintains the system and who needs to be involved in risk decisions. Responsibility requires time and authority, as well as a name in a register.
3. What data can be processed, and where?
Work from the organisation’s existing information classification. Assess the data needed for this application, the service processing it and the people who can access it. Cover storage, reuse and deletion. A label such as “public” or “internal” does not, on its own, establish whether a service is appropriate.
4. What do users need to understand?
Provide instructions and practice relevant to the task. Can people recognise a wrong answer? Do they know which data they must not enter and where to ask for help? The European Commission’s AI literacy guidance takes account of the people involved and the context of use. A short introduction may help, but it does not establish that everyone can perform their role responsibly.
5. How will quality be assessed?
Choose checks that reflect the consequences of error. Specify when a person must review a result and whether approval is needed before action. Decide what records are necessary to investigate problems, who may access them and how long they will be retained. Human review is useful only when the reviewer knows what to look for.
6. When do we continue, change course or stop?
Agree on a review date and the criteria for the decision. Consider results, cost, staff workload and adverse effects. Name the person who can decide. Serious problems must be addressed without waiting for the scheduled review.
7. Where do these decisions belong?
Use the existing forum for decisions about the process, its budget and its risks where possible. Involve IT and relevant specialists early enough to assess feasibility and dependencies. Set a review frequency that reflects how the application is used and how often it changes.
Work through the decisions for one application
Take one application through these seven decisions. Keep what is already working. Use the unresolved points as the agenda for the next discussion.
Want to work through the decisions for one of your applications? Contact Orbis Neo.