Questions to ask a conversion rate optimization agency

A conversion rate optimization (CRO) recommendation changes nothing until it’s live on your store. So the most useful questions for a Shopify CRO agency cover the whole path from finding to live fix, and what you keep when the contract ends. Many agencies do this work well. Asking before you hire a CRO agency is how you find out what you’re buying.

The CRO agency questions below are grouped by stage of the engagement. Each comes with why it matters and what a good answer sounds like. Ask for the answers in writing so you can compare proposals side by side.

Who implements the changes

• Who writes the code for each recommendation? Some agencies hand over designs and specs for your developer to build; others implement the changes themselves, sometimes billed as extra development. Either can work if the proposal says which one you’re getting.
• How much of our developer’s time will each month need? If your developer builds the fixes, their queue sets the pace of the engagement. A good answer estimates the hours up front and says what happens when a launch takes priority.

How findings are prioritized and priced

The ranking decides which findings get built and which wait. Our post “How to price a conversion leak in $/month” shows one way to put a number on each.

• How will you decide what to fix first? A good answer starts from your store’s own data and weighs each fix’s likely impact against its effort and its risk to the brand.
• Will each finding come with an estimated value in $/month? A dollar figure with the arithmetic shown lets you rank a fix against everything else competing for developer time. A good agency also tells you when the data is too thin to size a finding.
• Can we see a prioritized backlog from a past engagement? An anonymized example shows how findings are written and ranked in practice.

How tests are run and measured

• Which changes will you test, and which will you ship without a test? Pages with little traffic may not support a reliable test, and a good agency tells you which ones up front.
• What is the control? Without one, a lift can’t be told apart from a promotion or a seasonal swing. A good answer describes a split between the current version and the change, one primary metric chosen before launch and a rule for when the test ends.
• Which testing tool will you use, and who holds the account? Some agencies run tests in a third-party A/B app you license separately, so ask who pays for it and what happens to running tests if you part ways.

How changes go live and how they’re rolled back

• Where does a change go before it’s live? A good answer is a draft or duplicate theme you can preview in your own store, with your own products, before any customer sees it.
• Who signs off before anything is published, and what is checked first? You want a named approver on your side and automated checks, such as Shopify’s Theme Check, on every change.
• How is a change undone, and by whom? Ask how changes are logged and how fast one can be reversed mid-campaign. A good answer is specific and doesn’t depend on one person being available.

What reporting looks like and what counts as a result

• What will we see each month? Reports built on your own Shopify and Google Analytics data let you check the figures yourself. A good answer gives you access to the data behind every summary.
• What counts as a result? Agree up front whether you’re paying for findings, shipped changes or measured lift against a control. In “Why most conversion rate optimization audits never ship,” we argue for counting the fixes that went live.
• How do you report a test that loses? A loss still tells you something about your customers. A good agency reports it plainly, rolls the change back and records what it learned.

What happens to the backlog when the engagement ends

• Who owns the research, test results and code? A good answer is you, in formats your team can use without the agency.
• What happens to recommendations that were never built? Ask for the open backlog, ranked and priced, as part of the handoff so the next person can pick it up.
• Will your changes keep working after the engagement ends? Code written into your theme stays with your store, where your developer can edit it. Changes that depend on an app or agency account may need that subscription to keep working.

An agency alternative that ships the fix

If the answers show the work will end in a document your developer has to implement, consider a product that ships the fix. HI Engine started inside a Shopify agency and is built on five years of CRO work for Shopify stores. It finds what’s costing your Shopify store conversions, sizes each issue in $/month, then writes and publishes the theme fix. You approve.

Each fix is written into your theme as native Liquid, JSON and CSS, then validated by Shopify Theme Check. It’s held on a draft theme for you to preview in your own store, and nothing goes live without your approval.

Every fix runs as an experiment against a control, with visitors split 50/50 per browser and no third-party A/B app needed. Winners stay and losers roll back. Every change is logged, and rollback is one click.

Pricing is custom after the trial, from $700/month – based on your store size, credit usage and results. The 14-day trial itself is free: we ship your first 3 fixes in the first month, and you pay only after they’re live and you’ve seen the results.