What to Ask Before You Sign an Automation Contract

Let's start with the obvious. Rebbix builds automation for a living, so an article about how to buy automation isn't necessarily a neutral piece for us. However, we wrote it anyway, because the version of the market where buyers ask better questions is a better market for everyone working in it. 

A badly scoped project is expensive on both sides of the table. A buyer who knows what to ask arrives with a clearer picture of their own processes, which makes the conversation shorter, the estimates more honest, and the eventual disappointment less likely. We'd rather have that conversation than win a deal that falls apart along the way.

Now, how about a quick test? Find the most recent automation proposal sitting in your inbox. Three questions. What does the system do when it meets an input nobody configured it for? Who owns the workflows and the logic once they're built? And what does leaving look like in month nine, if you decide you want out? If the proposal answers all three, you can close this article and get on with your day.

Most don't. That gap is not usually anyone's fault, and it isn't a technical problem either, which is why it tends to survive the technical review. None of the three questions above requires you to understand how a system in question works. All three determine whether the contract you sign turns into something your business uses or something your finance team keeps paying for.

What follows is a set of questions worth asking before you sign. Some are about the technology. Several are about the contract. The first few are about your own business rather than the vendor's product, and those are the ones people often skip.

Three things to work out before you take the first call

The instinct is to start with vendors, because vendors are where the answers appear to live. In practice, the first three questions have nothing to do with them, and all three can be answered internally in about a week.

Which process, exactly

Most evaluations begin with a category. Somebody decides the company should automate customer service, or use AI in operations, or do something about returns. That sounds like a brief, and it isn't. A category can't be scoped, so the vendor might just scope the thing they already sell and call it a fit.

Arrive instead with one named process, a rough monthly volume, and a description of what currently happens when it goes wrong. The last part matters most. Write or map the process as it actually runs, including the spreadsheet somebody maintains on the side, the message that gets forwarded to a specific person because only they know how to handle it, and the step everyone skips when the queue gets long. Then compare that to how the process is documented. The distance between the two versions is usually where the project succeeds or fails, and it's information no vendor can gather for you in a discovery call.

That exercise tends to surface a second problem, which is the one that follows.

What state your data is in

Automation inherits whatever data quality already exists and then acts on it at speed. A person looking at a malformed supplier record pauses. A system does not, unless somebody built it to.

This is one of the most common structural blockers in the market right now. McKinsey research published in April 2026 found that eight in ten companies cite data limitations as a roadblock to scaling agentic AI. The same research puts the scale of the problem in context: nearly two-thirds of enterprises have experimented with agents, and fewer than one in ten have scaled them to deliver tangible value. 

The specifics look mundane, which is part of why they get skipped. Product content that differs across three supplier feeds, so the same item carries three descriptions and two prices. Order records and shipping records that disagree about what was returned and when. Customer identities split across a support tool, a billing system, and a marketing platform with no shared key between them. None of this is a vendor problem, and none of it gets fixed inside the contract you're about to sign. Finding out now costs a week, whereas finding out in month five costs the project.

This doesn't require a data audit, only an afternoon from someone who works with the system daily, listing the places where the record is wrong often enough that people have built habits around it.

What number you'd accept as proof

The last question is the shortest and the one most likely to change the outcome of the evaluation. Write down, before any vendor proposes anything, the measure that would tell you this worked. Hours returned to a team, resolution time on a specific queue, error rate on a specific step, cost per transaction at current volume, etc. 

Two useful things come out of this. You get a way to compare proposals that otherwise describe themselves in incomparable language. And occasionally you discover that nobody in the business can say what good would look like, which is worth knowing. It means the process hasn't been examined closely enough to be automated yet, and the honest next step is a look at your own operations rather than a vendor shortlist.

With those three answered, the vendor conversations become a great deal shorter, which brings us to what you're actually being sold.

What you're being sold

Why the title doesn’t always carry the meaning

Somewhere in the last two years, "agent" became the term that unlocks budget, and predictably, a great deal of software migrated toward it. Gartner gave the practice a name in June 2025. The term for it is agent washing, and its description covers products that were already on the market – chatbots, assistants, and robotic process automation among them – repositioned under the agentic label without the capabilities that would justify it. 

At that time, Gartner estimated that only about 130 of the thousands of vendors claiming agentic AI were building the real thing. Anyone who was around for cloud washing in 2010 or AI washing in 2023 will recognize the shape of it.

This territory attracts a lot of dismissive commentary, and most of it is wrong. The research is optimistic about where the category lands. Gartner expects agents to be making at least 15% of everyday work decisions by 2028, and that agentic capability will be built into roughly a third of enterprise software products, up from almost none in 2024. The technology is real, and it's arriving quickly. The difficulty is that a fast-moving category and a blurred vocabulary make it hard to tell which specific product in front of you belongs to it.

One distinction to carry into the room

We suggest one question that can help make sense of it – does the system change its own plan when the plan stops working?

A chatbot answers. A workflow runs a route someone drew in advance and reaches the end of its instructions when reality departs from them. An agent holds an outcome and finds another way when a step fails.

Let’s look at two examples. A booking flow where the supplier API times out halfway through confirmation. A workflow retries, then stops, then queues a task for a person. An agent checks whether the reservation actually landed on the supplier side before retrying, because a blind retry risks a double booking. 

Or a return request where the order record says three items and the warehouse scan says two. A workflow follows whichever record it was pointed at. An agent notices the conflict, gathers what it can, and either resolves it against a rule you set or escalates with both records attached.

Notice what separates them. Both are useful, but one of them costs considerably more to build and run, so the question is which behavior your process actually requires.

The demo problem

Here's why this is hard to see from a seat in the audience. A good demo shows a system reading a message, understanding intent, drafting a response, and updating a record. That seems genuinely impressive, and it works. What the demo cannot show you is the call, where the supplier's API sends back something the integration can't read, or the input arrives in a format nobody configured. Autonomy only becomes visible against unscripted inputs, which is exactly what a rehearsed demonstration is designed to avoid.

So the single most useful thing you can ask for is a demo on your data. A vendor with a real product will find this a reasonable request. Products that have been relabeled tend to stop at the first input variant they weren't set up for.

Five questions about the product

Each of these has a strong answer and a familiar evasion. The evasions are the part worth learning, because they sound perfectly acceptable in the moment.

What happens when the system meets something it hasn't seen before? 

Strong answers get specific about the fallback: retry logic with conditions, an alternative path, a handoff to a person with the context attached. Evasion describes the model's general intelligence, or claims that it handles everything, which is a claim no serious engineer makes about any system.

Which of our systems does it write to, rather than only read from? 

This sorts advisory tools from operational ones in about a minute. A system that only reads is an assistant, and assistants are useful, but they don't remove work from anyone's queue. Strong answers name the systems and the scope of write access. Evasion talks about integrations without ever saying in which direction data moves.

What can it do without a person approving it, and how is that boundary enforced? 

Real products have an autonomy boundary you can configure, and the vendor can describe. If the boundary is presented as a philosophy of responsible AI rather than as a setting somebody in your team controls, it isn't a boundary.

What does it do when our data is wrong or missing? 

Strong answers describe validation, refusal behavior, and what the system does when two sources disagree. Evasion assumes clean inputs, which is a description of a pilot environment rather than your business.

What can we reconstruct afterward?

Logging, traceability, and the ability to explain three weeks later why a particular decision was made. This is the question that matters most the first time something goes wrong involving a customer's money, and it is the one buyers skip most often because it feels like an operational detail. It becomes the whole conversation during an audit.

What to ask about the deal, not just the product

Everything so far has been about what the system does. This part is about what you're left holding afterward, and these questions get asked far less often than they should, usually because they feel like something to sort out later with the contract. By then you're negotiating against a template, and the answers cost more.

You don't need legal language for any of this. Ask in plain terms and listen to how comfortable the answer sounds.

Who owns the workflows once they're built?
A vendor brings their platform, and reasonably enough, they keep it. What's less obvious is who ends up owning the layer you build together: the process logic, the exception rules, the mapping between your systems, the accumulated decisions about how your business actually handles a refund or a schedule change. That layer is where most of the project's value sits, and it was largely produced from your knowledge.

Ask directly whether you own it, whether you can take it with you, and in what form. Strong answers are specific, while vaguer ones tend to reframe the question as being about the platform, which isn't what you asked.

Will you use our data to improve your product?
Two versions of this, and they're different questions. The narrow one is whether your data trains models or improves systems that other customers benefit from. Vendors often want broad permission here, and there are honest reasons for it, but you should know what you're agreeing to rather than discover it in an appendix.

The broader one matters more in this market. Ask whether your operational patterns, your exception handling, your supplier quirks feed back into a product sold to your competitors. For example, in travel and ecommerce, that's a real consideration, since the vendor's next three clients may well be firms you compete with.

What does this cost at our real volume?
Pilot economics rarely survive production traffic, and cost overrun is often the first item on the list of reasons these projects get canceled.

Ask for the number at your actual monthly volume, including the busy month rather than the average one. A vendor who has run something at scale can produce it; one who hasn't will steer back toward value, which is a reasonable topic, but not the one you raised.

It's also worth understanding what drives the price. Per-agent and per-seat models reward adding more agents, which aligns the vendor's revenue with your integration complexity. That's not disqualifying, but it is something to see clearly before it shapes a roadmap.

If we want out in month nine, what do we actually get back?
The least popular question in any vendor meeting and the most revealing. There are three parts to it: 

• what you can export, and whether the format is usable by anyone other than them; 
• how much notice you need to give, and whether the agreement renews on its own while you're not paying attention
• what happens when the AI model underneath the system is retired by whoever makes it, since that timetable belongs to a third party and a system tuned against one version doesn't always behave the same way on the next one.

A vendor confident in their work answers this easily, because they expect you to stay for reasons other than difficulty leaving. Discomfort here tells you something worth knowing.

Who is doing the work

Everything above concerns what you're buying. This part concerns who you'll be working with, which turns out to matter just as much.

Will the people in this call be the people building it?
Sales teams are staffed with people who are good at conversations about outcomes, and delivery teams are made up of people who are good at systems. Both are necessary, and the gap between them is where a lot of projects lose their footing. Ask who does the implementation, whether you'll speak to them before signing, and who you call in month three when something behaves oddly.

What do you need from us?
The most revealing question of the entire evaluation, and the one most likely to separate a vendor who has done this before from one who hasn't. Experience produces specifics – your data gaps, your integration work, the exception cases somebody on your side will have to map because nobody outside your business knows them, which of your people will need to be available and for roughly how long. 

A vendor who says they need almost nothing from you is either selling something that does very little or hasn't looked closely at your systems yet.

Where does this leave our team?
Worth stating as a preference before signing rather than discovering afterward. Some engagements end with your people able to adjust a workflow, read a log, and understand why the system did what it did. Others end with a black box and a support email address.

Both models are legitimate, and permanent in-house ownership of every system isn't a realistic goal for most businesses. Some dependency is the normal cost of working with external specialists, and there's nothing wrong with it. The question is whether you chose it deliberately or accumulated it by default, which is a distinction you can only make while you're still deciding.

What only you can answer

If you read back through the questions, something becomes obvious. Some of them can't be answered well by a vendor at all, because they depend on things only you know: how the process really runs, where the records disagree, what an improvement would need to look like for anyone to call it one.

The quality of the answers you get from a vendor depends heavily on the quality of the brief you bring. Skip it and you end up evaluating vendors on how well they present, which is a skill unrelated to the one you're paying for.

So the honest advice is to do that work first. Map the process as it actually operates, look at the state of your data, and decide what number would tell you it worked.

If you already have the answers to those questions and would like to move on with an automation project, or if that sounds like more than you want to take on alone, leave us a message. A conversation with our team costs nothing, and most people come out of it with a sharper sense of what they're actually trying to solve, whether or not it goes anywhere. Bring the process you've been thinking about automating, and we'll tell you what we'd want to know before quoting on it.