For people like us operating in a dynamic and competitive landscape, providing exceptional customer service and support is our key differentiator. So, our support team works hard to respond to client questions, troubleshoot issues, and ensure our point-of-sale (POS) software enables their success. So, when we heard about AI conversation bots that could potentially assist our support team, we were eager to test this emerging technology.

After developing our own AI chatbot prototype and trialling it on actual customer queries, we found there are indeed limitations with AI support today.

We feed it thousands of our customers' queries.

Depending on who accessed its answers, we got these disputed scores. I say disputed as some judges were much more negative than others. This is a consensus score, not an individual view.

- Approximately 23% of the chatbot responses were largely to wholly accurate. These tended to be the stuff that our support staff felt were things they knew well and would have little trouble doing without AI.

- 50% were partially correct but lacked vital details. 

- 15% were unclear due to poorly phrased customer questions. This is a common problem that those who ask questions in the real world to get accurate information need to know much. Rubbish in = Rubbish out.

- 10% were utterly wrong.

Our AI chatbot clearly has room for improvement before it goes live.

However, AI can still provide value when implemented correctly by augmenting our support team’s capabilities. 

Why AI struggles with complex support queries

Through our testing process, we gained insight into factors that limit AI chatbots’ capabilities for software support:

1. Insufficient training data

Like humans, AI systems require extensive ‘practice’ through training data to become highly skilled at specific tasks. Our prototype only had access to a thousand pre-recorded customer support calls from which to learn - this is far too little data to fully grasp the complexity and nuances of troubleshooting what is our system probably the most advanced POS software in Australia.

2. Incomplete understanding of context and ambiguity

Humans draw upon years of world knowledge and experience when interpreting language. AI chatbots currently lack this contextual awareness, so they struggle to clarify vague questions and handle ambiguities. Our clients have a range of English knowledge; some know plenty, we have clients with doctorates and some that barely speak English. So 15% of customer queries stumped our chatbot due to unclear wording that we could easily interpret.

3. Inability to grasp nuanced domain knowledge

POS software is feature-rich and built to handle intricate retailer needs. Most AI systems today would need explicit training to comprehend the industry-specific terminology, workflows, exceptions, integrations, and technical considerations involved. Without directly relevant domain training, the chatbot’s knowledge gaps led it to miss critical details in 50% of test cases.

Best practices for implementation

While the technology still has maturing to do, useful AI assistance is within reach by following guiding principles for implementation:

Ensure sufficient high-quality training data

For accurate learning, training datasets must cover each type of support scenario the AI may encounter. Data should be cleanly recorded without errors that could propagate misleading information. The problem here is that our information is real, and in the real world information with errors is common.

Provide ongoing human feedback.

Humans must continually verify and correct the AI’s responses, allowing its knowledge to evolve through feedback over months and years. This human-in-the-loop oversight is critical. This will take a lot of work.

The future role of AI in customer support

Though AI chatbots cannot wholly substitute for human support staff at this stage, we see bright possibilities for AI assistance going forward:

• Handling high volumes of repetitive queries - Freeing support staff to focus on higher-level, more complex problem-solving for customers.

• Providing 24/7 self-service - Enabling customers to access basic troubleshooting after hours when support staff are offline.

• Supporting human agents - Assisting our support staff with knowledge lookup, documentation, and tracing issues.

While AI support today is imperfect, we are excited by its prospects through careful implementation. Over the next several years, we aim to refine our chatbot to be an invaluable teammate for our human support staff - combining the best human and machine capabilities to deliver exceptional customer assistance. With a spirit of experimentation and patience, as this emerging technology matures, we know that AI promises to improve customer service.

Let me know if you are interested in learning more because this is really interesting stuff.

ChatGPT for Businesses 

Most of our clients are very progressive regarding technology and are as excited about ChatGPT as we are.  

What is ChatGPT

Its an AI chatbot. If you have yet tried out ChatGPT, click here. Make yourself an account, and then give it a try. Ask it some questions, ask it to translate some text into your language, give it some history questions, religious or theological questions, cooking questions, etc. You will be amazed at what it can do. 

This is why businesses today are paying attention to ChatGPT. It is an appealing option for businesses of all sizes looking at improving efficiency because it can save time and money. Exactly what it can do is still being determined.  

Fun fact

I asked ChatGPT to describe itself and put its description into a drawing program; this is how ChatGPT sees itself.

Competitors to ChatGPT

There are several rivals for these products. The most well-known is Bard by Google, but many others are into it; Vicuna is a Japanese product. 

Quality 

Quality is the most crucial point. If the product cannot do the job, what difference does anything else make? Here is a very rough test of the quality of several product results.

I wonder if most users will notice that much difference in quality. All have their strengths and weakness. 

You can check out the quality here and make up your mind.

I feel that the free version of ChatGPT is about as good as Vicuna. Overall I prefer Vicuna for business.

Price

The older ChatGPT is free, but the newer product is $20 per month. The others are all free for now. If you want to try the newer ChatGPT before paying for it, try the others first and the older version of ChatGPT so you will know what you need before jumping into it. One needs to learn to walk before one starts to run.

Availability

All except Bard are available now to an Australian. Bard is officially yet to be open to Australians. If you want to run Bard now in Australia, you need a VPN. 

Everyone is now waiting to see the Chinese product, but it has yet to be available in Australia.

Information

This is a big problem in business; Bard is the only one now with up-to-date information, and the rest are about two years old. Yet a businessperson needs current information. A person will come to a shop and generally wants to know about something recent. This puts significant limitations on its business use in Australia.

Outages

Even with paid services, users need help with using the product. It can be very frustrating. Also, many reports say that OpenAI has made ChatGPT stupider to get faster speeds. 

Accuracy

With a Google search, you know where the information comes from; this is not the case with these products often, and YES, the information they supply is often wrong. 

Since these often do not supply references, you must check the answers you get from these products. Be careful.

Political filter

Except for LLaMA, they all have political filters in place. These can drive you crazy as innocent questions can be refused to be answered as such. Heavy users of these products often will debate if Big Brother is watching them. 

Conclusion

Today we are all paying attention to this technology. It is an exciting technology that has the potential to cut costs and increase productivity. I am more than happy to discuss it if anyone wants. 

Executive summary

> These AI can answer a wide range of questions.
> Businesses are looking for them because of their potential to cut costs, save time, and increase efficiency.
> They all have their pluses and minuses. 
> You may not notice a significant difference in quality 
> These products' limited business applications may be due to outdated information.
> These products give incorrect answers, so you must check their accuracy.