Shopify AI Coach: Coaching Merchants From Launch to Customer Care

Project overview

Shopify AI Coach is a Python and Django web app I built to answer Shopify merchants the way a mentor would: a direct answer, the reasoning behind it, and the next action worth taking. Answers come from a curated knowledge base of more than 300 question and answer pairs stored in Neon Postgres, and a language model covers the questions the knowledge base cannot answer well on its own.

The interesting engineering problem was not “add a chatbot.” It was deciding when the app is allowed to speak from vetted knowledge and when it is allowed to improvise — then making both paths sound like the same coach.

The problem

Merchants ask operational questions constantly: why traffic is not converting, whether a discount is worth running, what to fix on a product page, how to react when a supplier changes. Two common answers fail them. A documentation link explains a setting without explaining the decision. A generic AI reply sounds confident while staying too vague to act on.

Why a coach, not a support chatbot

Most AI support agents for Shopify stores are built to close a ticket: identify the issue, point at the setting, end the conversation. That is the right shape for “where do I change my shipping zones” and the wrong shape for “why is nobody buying.” A Shopify chatbot that only resolves tickets leaves the merchant with a corrected setting and no decision.

A coach behaves differently. It answers in the merchant’s language, commits to a recommendation, and names the next step — including when the honest recommendation is to do nothing yet. That is a content problem as much as a software one, which is why the app was designed around vetted answers first and generation second.

Merchants trying to find a Shopify coach are rarely looking for one isolated answer. They want to learn from experts across the decisions that connect an idea to a delivered order. Shopify AI Coach turns that Shopify coaching experience into an always-available product rather than a calendar of one-to-one calls.

What the app had to get right

• Answer from vetted knowledge whenever a vetted answer exists
• Recognize the same question asked in many different ways
• Never leave a merchant with “I don’t know” as the final word
• Stay consistent, so one question does not produce two different opinions
• Keep the language model a fallback rather than the default, for cost and for control

The seeded knowledge base

I authored and seeded more than 300 question and answer pairs into Neon Postgres in one consistent format. Each entry carries the canonical question, the phrasing variants merchants actually type, the coached answer, the topic it belongs to, and the next action to recommend.

Writing to a fixed format is what makes retrieval work later. Variants give the matcher something real to match against, topics stop related answers from competing with each other, and a required next action keeps entries from decaying into definitions.

Neon runs as the app’s Postgres, so loading the knowledge base is a repeatable management command against a serverless database rather than a hand-edited table nobody can reproduce.

One Shopify trainer from launch to customer care

The knowledge base is organized around the merchant journey, not around software menus. Each topic teaches a practical Shopify skill and connects it to the decision that comes next. The coach gives guidance; it does not place supplier orders, change store settings, or message customers without the merchant’s approval.

Store launch and Shopify login

For a new merchant, the coach answers Shopify login and account-access questions, explains Admin navigation, and guides the decisions behind store setup, theme selection, domains, policies, payments, and launch readiness.

Product validation and supplier sourcing

Before a product is listed, it helps the merchant evaluate demand, competition, margin, samples, minimum order quantities, supplier reliability, and delivery promises. The answer ends with a check or next action instead of declaring a product a guaranteed winner. Those decisions can draw on live Meta and TikTok ads filtered for dropshipping relevance rather than intuition alone.

Catalog, pricing, and merchandising

The coach covers product titles, descriptions, images, variants, collections, pricing, bundles, and discounts. It explains how each choice affects trust and margin so catalog work supports the offer rather than merely filling a page.

Marketing, conversion, and sales

Merchants can ask about traffic quality, product-page conversion, social proof, abandoned carts, email campaigns, paid ads, and promotions. Responses distinguish a traffic problem from an offer or trust problem before recommending more spend or a deeper discount.

Orders, fulfilment, and delivery

From supplier handoff to the customer’s door, the Shopify trainer explains inventory checks, order processing, shipping options, tracking, delivery delays, and how to communicate when fulfilment goes wrong.

Customer handling and retention

The coach helps frame replies for pre-sale questions, complaints, returns, refunds, chargebacks, and follow-up. It recommends a fair action and a clear response while leaving sensitive customer communication under merchant control.

Analytics and the next growth decision

Once the store is operating, it helps merchants read conversion rate, average order value, acquisition cost, repeat purchase behavior, and product performance. The goal is an all-in-one Shopify trainer that links launch, sourcing, sales, delivery, and customer care instead of treating them as unrelated tickets.

How Shopify AI Coach answers a question

1. Read the question

The app normalizes the incoming message and works out what the merchant is actually asking about, so a short, sloppy question is handled as well as a carefully written one.

2. Match it against the seed

The question is scored against the seeded pairs and their phrasing variants. A strong match returns the vetted coached answer as written. That is the entire point of authoring the knowledge base: the merchant gets the reviewed answer instead of a fresh improvisation.

3. Fall back to the model when the match is weak

When no seeded answer clears the confidence bar, the question goes to the language model rather than returning a dead end. The closest seeded material is passed in as grounding, together with the coaching rules, so the generated reply stays specific and keeps the same voice.

4. Answer as one coach

Both paths return the same shape of answer: the recommendation, the reasoning, and the next step. A merchant should not be able to tell which path served them, because inconsistency is what makes an assistant feel untrustworthy.

Architecture

Django owns the request cycle, the knowledge base models, and the seeding commands. Neon Postgres stores the seeded pairs. The model integration sits behind a small service layer, so the provider, the prompt, and the confidence threshold can change without touching views. Credentials and model settings are environment configuration, never code.

The design rule was simple: vetted knowledge first, generation second, and never an empty answer.

Tradeoffs

Authoring hundreds of pairs by hand is slower than pointing a model at a documentation dump, and I chose it deliberately — a vetted answer is reviewable, repeatable, and cheap to serve. The cost is maintenance: every pair is content somebody has to keep true.

The confidence threshold is the other real tradeoff. Set it too low and the app improvises when it should quote. Set it too high and it quotes a near-miss instead of thinking.

Outcome

• More than 300 vetted question and answer pairs seeded into Neon Postgres
• Merchant questions matched against vetted answers before any generation happens
• A model fallback that answers the long tail instead of returning nothing
• One consistent coaching voice across both answer paths
• Knowledge base loading repeatable as a Django management command

Where this coaching layer runs

Coaching is most useful next to the work it describes, so it runs inside the merchant dashboard alongside store building, catalog publishing, and stock-out recovery. The merchant can ask about the task already on screen instead of reconstructing its context in a separate chatbot.

That store automation uses the safer platform boundary developed by turning a rejected Theme Asset API request into an approved implementation. Product recommendations enter through the same filtered market signals the coach helps merchants evaluate.

What I learned

Retrieval quality turned out to be a writing problem more than an algorithm problem. The pairs that matched reliably were the ones written the way merchants actually ask, with variants included from the start rather than added after the first bad match.

The fallback also changed how I think about confidence. An assistant that admits a weak match internally and then does more work is far more useful than one that either guesses loudly or gives up politely.