Building a company chatbot: where to start

Large language models made it easy to build a chatbot that writes fluent, confident answers. Fluent responses are only one requirement. The hard part is a chatbot that gives correct answers about your company, says so when it does not know, and hands over to a person at the right moment. Most of that is decided before any model is chosen.

Start with one job and one audience

"A chatbot for the company" is too broad to build well. Pick one job first:

• answering customers' questions on the website (products, prices, delivery, opening hours);
• an internal helpdesk for staff (leave rules, expense claims, how to request a laptop);
• support for the sales team (product specifications, which product fits which case).

Then collect the real questions that audience asks today, from emails, support tickets, chat logs and the people who answer them. A few dozen real questions teach you more than any brainstorm: how people phrase things, what they actually need, and which questions come up again and again.

Answer from your own material

A language model knows a lot in general and nothing in particular about your prices, policies or products. Left to itself it will still answer, and when it does not know it may produce something plausible and wrong (we explain why in What is a large language model, and what are its limitations?).

The usual approach is retrieval: when a question arrives, search your own documents, give the model the few passages that are relevant, and instruct it to answer only from them and to say which document it used. This is called RAG, and we compared it with fine-tuning in fine-tuning or RAG?.

It also means the chatbot is only as good as the documents behind it. Outdated price lists, two versions of the same policy, rules that live only in someone's head: all of them turn into wrong answers. Before launch, gather the material, remove what is out of date, and give each document an owner who keeps it current.

Decide what it is allowed to do

Answering questions from documents is one thing. Taking actions (checking an order's status, booking an appointment, opening a ticket, changing an account) is another, and each action needs its own care:

• connect it through a defined interface with only the permissions that action needs;
• ask the user to confirm before anything that changes something;
• log what was done and why.

A sensible first version only answers. Actions can come once the answers have proved reliable.

Know when to hand over

A good chatbot knows its limits. Plan when it should pass the conversation to a person:

• when it cannot find the answer in its material;
• when the user asks for a person;
• for subjects that need judgement or care: complaints, payments, legal or medical questions.

When it hands over, the person should see the conversation so far, so the customer does not have to start again. And say clearly, from the first message, that the visitor is talking to an automated assistant.

Privacy and security

• Know where the data goes. If the model runs at a provider, the questions and the documents you send go there too. Check what is stored, for how long, and whether it is used for training, and keep personal data out where you can.
• Never give the chatbot more access than the user has. If an employee may not see salaries, the chatbot answering that employee must not be able to read them either.
• Expect manipulation. Text in a user's message or in a document can try to override the bot's instructions ("ignore your rules and…"). This is called prompt injection. Limiting what the bot can reach and do is the defence that holds; instructions alone are not.

If your users write in Persian

Test in Persian, with the way people really write it: colloquial spelling, missing half-spaces, Arabic letters from an Arabic keyboard layout, Persian and Western digits mixed, and sometimes Persian typed in Latin letters. The search behind the chatbot has to treat these variants as the same text, or it will miss the right document (we wrote about this in processing Persian text with AI). Also measure cost and speed with Persian text: the same sentence often takes more tokens in Persian than in English.

Test before launch, and keep testing

Build a test set from the real questions you collected: each with what a good answer contains, plus questions the bot should decline or hand over. Run it on every change, of model, instructions or documents, because a change that fixes one answer can break another. After launch, read conversations regularly. The questions it could not answer show which documents are missing, and they are the most useful list you will have.

What to measure

• how many conversations were resolved without a person;
• how often it handed over, and whether those hand-overs were right;
• wrong answers found when reviewing conversations;
• what users say when asked whether the answer helped.

Start small, with one job, a clean set of documents and an honest test set, and widen it once the answers can be trusted. Contact us to discuss the requirements of a chatbot for your organisation.