Cartly — AI Shopping Assistant & Virtual Try-On Store

Role: Full-stack engineer (storefront, admin panel, commerce API and AI service)

Cartly is an online fashion and gifts store with Pip, an AI shopping assistant that talks with shoppers by text or voice in their own language, and a virtual fitting room where they can see items on themselves before buying.

The Problem

Online shoppers can't ask a salesperson for advice or try anything on, so they leave or return what doesn't fit. I wanted a store where an assistant knows the catalog, the shopper's cart and their orders, and where trying an item on takes a photo instead of a trip to a shop.

What I Built

• Pip, the AI shopping assistant: with Claude, Pip answers in whatever language and script the shopper uses. It picks products from the live catalog, adds them to the cart, answers questions about sizes and orders, and brings up what's timely for the shopper, such as the season or an upcoming Eid, Durga Puja, Diwali or 11.11 sale. Voice mode uses ElevenLabs speech-to-text and text-to-speech, and falls back to the browser's speech when no key is set.
• Virtual fitting room: with FASHN, the shopper's own photo is re-rendered wearing the look, with styling options like "tucked in" or "worn open" and a hold-to-compare view. An on-device quick preview places accessories (glasses, hats, watches, headphones, bags) on the body from MediaPipe pose landmarks, and the photo never leaves the device. A photo check flags shots that are cropped or taken from the side.
• Semantic search: pgvector in Postgres, so searches like "warm jacket" or "red dress under $100" work without a separate search vendor.
• Full commerce flow: catalog, cart, promo codes, checkout, orders, favourites, reviews, product requests and shared team accounts, all stored on the server.
• Admin panel: sales dashboard, orders, products and stock, customers, analytics, reviews, promotions, returns, delivery, payments, content, team roles, and a System page that shows today's AI spend.
• Bangladesh payments and delivery: SSLCommerz, bKash and Nagad payment gateways, plus Pathao, Steadfast and RedX courier integrations.

By the Numbers

• 4 services: a Vue storefront, a Vue admin panel, a NestJS commerce API and a FastAPI AI service
• ~150 REST endpoints with Swagger docs
• 27 data models in Prisma on PostgreSQL
• End-to-end tests that run against a real Postgres and Redis

Tech Stack

• Frontend: Vue 3, Vue Router, Vite, MediaPipe Tasks Vision
• Backend: NestJS (TypeScript) with Prisma, and FastAPI (Python) for the AI and voice service
• Data: PostgreSQL with pgvector, and Redis for carts, assistant sessions, rate limits, usage metering and caching
• AI: Anthropic Claude (assistant and photo review), FASHN (AI try-on), ElevenLabs (voice)

Key Decisions

• Every AI feature has a fallback: each one switches on with a single API key. Without it, the store still works: Pip uses a rule-based brain, try-on falls back to the on-device preview, and voice uses the browser. If the AI service is down, the same rules run in the browser.
• AI costs are controlled: every paid AI call is metered, each vendor can have a daily spending cap, and admins see today's spend.
• Privacy and honesty in try-on: a shopper's own photo is only used after they agree, and a flat cut-out is never presented as clothing being worn. Clothes and shoes only get "Try it on" when the AI try-on is enabled.

Outcome

A complete storefront, admin panel and AI stack in which a shopper can describe what they want in their own language, get picks from the real catalog, see the item on themselves and check out with local payment and delivery options.