The Knowledge Gap in Live Selling
Retail floors and technical B2B sales share one moment: a customer asks something specific, in front of the product, and the rep either knows the answer or doesn't. Three things make that moment harder than it should be:
- Catalogs move faster than training: specs, pricing and stock change weekly, while onboarding happens quarterly.
- The answer lives on a screen: reaching for a phone or laptop breaks eye contact and kills momentum.
- Coaching arrives after the deal: call intelligence improves the next conversation, not this one.
Why Glasses, Why Now
Smart glasses are projected to ship 112M units a year by 2030, up from 2.7M in 2024 (Citi Research), and in 2026 displays arrived in ordinary frames. Reps will wear these anyway. We designed Lookwise to be device-agnostic: a camera, a microphone and a display are enough. Phones work today; glasses make it invisible.
Design Principle: Never Break the Conversation
Everything follows from one constraint — the rep's attention belongs to the customer. Lookwise runs a four-step loop, and the only thing the rep ever sees is the last step.
- See: vision AI identifies the product the rep and customer are looking at.
- Know: it's matched to the brand catalog — live price, stock, specs and alternatives.
- Listen: the conversation builds a structured profile of budget, use case and concerns.
- Show: guidance appears on the glasses display, silently, with no wake-word gymnastics.
Decision: Every Turn Refines the Profile, Not Just the Answer
A chatbot answers the last question. A good salesperson remembers everything the customer has said. I designed the guidance model as a pipeline that carries the conversation's state forward:
- Materiality gate: a cheap, non-AI filter skips filler like "yeah", "ok" and "mm-hmm", so we only pay for turns that matter.
- Profile update: GPT-4 merges each turn into a structured customer profile — use case, priorities, concerns, budget — without losing earlier context.
- Budget and tier signals: amounts and ranges are mapped to value, mid or premium tiers.
- Catalog matching: the evolving profile is matched against the brand catalog for a best match and same-category alternatives.
- Discovery gate: when budget and key needs are still unknown, Lookwise holds back the product and suggests a clarifying question instead. A price-context check flags budgets below the cheapest in-stock model.
- Guidance composer: discovery questions if the profile is thin, a recommendation and pitch if it's confident — with the fields that changed this turn highlighted.
Designing the HUD: One Glance, One Action
The glasses display is small and sits in the rep's line of sight, so the HUD shows exactly one thing at a time — the guidance composer's output, not the profile behind it.
- Lead with the action: each card opens with a short label for what to do ("ASK") above a single line the rep can say out loud.
- Keep the face clear: guidance sits to the side of the view, so the rep never looks through text to see the customer.
- Show the system is listening: an "Updating…" state appears while the profile refreshes after a turn, so the rep doesn't act on stale guidance.
- Questions before products: when the discovery gate holds back a recommendation, the HUD shows the clarifying question instead — here, the customer's budget.
Architecture: A Swappable Capture Layer
The platform isn't tied to one headset. Capture can come from Meta Ray-Ban Display glasses, a phone camera or ambient audio, feeding a React Native (Expo) app and a Node.js backend on Azure Container Apps.
- AI engine: Azure OpenAI — GPT-4 Vision for product identification, Whisper for speech recognition, and vision grounding.
- Live price lookup through Google Shopping data, and catalog matching against brand catalogs in Azure Cosmos DB.
- Outputs: visual coaching on the glasses HUD, product and price cards in the app, and a brand admin portal.
Designing for Trust
A device that listens on a shop floor has to earn its place. We designed the risks in from the start:
- Recording customers: no video retention; the transcript stays client-side and is discarded per turn. Capture is rep-initiated, with in-store notice and a consent mode for EU deployments, hosted in an Azure EU region.
- Hardware timing: phone camera and earpiece deliver value today, and glasses are the upgrade path — whichever frames land first.
- Enterprise friction: pilots start with a flat catalog import (CSV or read-only API). No CRM write access, and no system of record to replace before value shows up.
What's Still Unproven
Clienteling tablets make the rep look down, XR field-service tools have no catalog or customer, and sales-enablement platforms coach after the conversation. Lookwise fills that gap on paper. What it hasn't done yet is prove itself on a real sales floor: the under-3-second answer loop, the HUD's readability mid-conversation and the effect on sales all still need testing with working reps.
Achievements
- Finalist in the online round of the XRCC 2026 hackathon.
- 1st place, Sales Track at the XRCC 2026 finals in Berlin — judged against teams from across the XR industry.
- Selected for the Builders Program at UnitedXR in Brussels (December 2026), where Lookwise will be pitched on stage and in front of investors.