← NextPrev →

PriceDrop - A/B Price Testing Engine for Shopify Stores Under $1M Revenue

Every Shopify store owner is either guessing their prices or leaving serious money on the table, and neither Shopify Analytics nor Google Analytics will tell them which. PriceDrop runs statistically valid price A/B tests across any product, shows real-time conversion lift, and declares a winner automatically — no data science degree required.

𝕏 Post Reddit HN

Difficulty

intermediate

What is it?

Shopify store operators on r/ecommerce and the Shopify Community forums consistently report pricing gut-feel as their biggest unsolved problem — they know $49 vs $59 matters but have no tooling that is not either enterprise-priced (VWO, Optimizely) or too generic (Google Optimize was shut down). PriceDrop is a Shopify app that lets merchants create a price A/B test in 3 clicks: pick a product, enter two prices, set a traffic split, and let it run. The app uses Shopify's storefront API to serve price variants per session, logs conversion events, and runs a Bayesian significance test client-side to declare a winner. Merchants see a live lift percentage and a clear win/lose verdict when statistical significance is reached. Buildable in 2 weeks as a Shopify embedded app using Shopify CLI, Polaris UI, and Supabase for test state.

Why now?

Google Optimize was shut down in September 2023 leaving a genuine gap in the Shopify ecosystem for affordable price testing — 200k+ mid-tier Shopify merchants have no credible alternative at under $100/month.

▸3-click price A/B test creation: select product, enter Price A and Price B, set split percentage (Implementation note: store variant assignment in Supabase keyed to Shopify customer session cookie).
▸Real-time conversion tracking dashboard showing orders, revenue, and conversion rate per price variant via Shopify webhook events.
▸Bayesian significance calculator that auto-declares a winner at 95% confidence and sends a Resend email notification to merchant.
▸One-click winner apply that updates the Shopify product price via Admin API and archives the losing variant.

Target Audience

Shopify merchants doing $50k-$1M annual revenue — roughly 200,000 stores in this band — who optimize conversion but have no price testing tool.

Example Use Case

Priya runs a Shopify candle store, creates a $34 vs $39 price test on her bestseller in 3 clicks, lets it run for 10 days, PriceDrop declares $39 the winner with 94% confidence, and she increases annual revenue by $8,400 without changing a single product.

User Stories

▸As a Shopify store owner, I want to run a price A/B test on my bestselling product without touching code, so that I can find the revenue-maximizing price with real data.
▸As a Shopify merchant, I want an automatic winner notification when my test reaches statistical significance, so that I do not have to manually check results every day.
▸As a store operator, I want to apply the winning price in one click directly from the results dashboard, so that I capture the revenue lift immediately without navigating Shopify admin.

Done When

✓Test creation: done when merchant selects a product, enters two prices, and sees the test appear as active in the dashboard within 10 seconds.
✓Conversion tracking: done when a test order placed on the Shopify dev store appears in the dashboard under the correct variant within 30 seconds of order creation.
✓Winner declaration: done when Bayesian significance hits 95% and merchant receives a Resend email with variant name, lift percentage, and a one-click Apply Winner button.
✓Price apply: done when merchant clicks Apply Winner and the Shopify product price updates to the winning variant price visible in Shopify admin within 5 seconds.

Is it worth building?

$29/month x 50 merchants = $1,450 MRR at month 2. $29/month x 200 merchants = $5,800 MRR at month 7.

Unit Economics

CAC: $15 via r/shopify DM outreach. LTV: $348 (12 months at $29/month). Payback: under 1 month. Gross margin: 87%.

Business Model

SaaS subscription via Shopify Billing API

Monetization Path

7-day free trial via Shopify Billing API, then $29/month. Upgrade triggered when merchant tries to run a second concurrent test on free tier.

Revenue Timeline

First dollar: week 4 via first Shopify Billing API charge. $1k MRR: month 3. $5k MRR: month 8.

Estimated Monthly Cost

Shopify Partner fees: $0, Vercel: $20, Supabase: $25, Resend: $10, Stripe fees (via Shopify Billing API): ~$0 at launch. Total: ~$55/month at launch.

Profit Potential

Full-time viable at $5k-$10k MRR with 170-345 paying merchants.

Scalability

High — expand to multivariate testing, bundled pricing tests, and a Shopify Plus tier with team access.

Success Metrics

Week 2: 10 beta installs. Week 4: first 5 paid conversions. Month 2: 40 paying merchants, under 8% monthly churn.

Launch & Validation Plan

Post in r/shopify and Shopify Community forum asking how merchants currently test prices — 20 responses validate demand before submitting to app store.

Customer Acquisition Strategy

First customer: DM 20 Shopify merchants in r/ecommerce offering free 30-day beta access in exchange for a recorded walkthrough review. Ongoing: Shopify App Store organic, r/shopify posts, ProductHunt, YouTube Shopify tutorial creators.

What's the competition?

Competition Level

Medium

What's the roadmap?

Feature Roadmap

V1 (launch): 2-variant price test, real-time dashboard, Bayesian winner, one-click price apply. V2 (month 2-3): scheduled test end dates, test history archive, CSV export of results. V3 (month 4+): multivariate price testing, bundle price testing, Shopify Plus team seats.

Milestone Plan

Phase 1 (Week 1-2): schema, Shopify API client, test creation, webhook handler — done when test order registers correctly. Phase 2 (Week 3): dashboard, Bayesian engine, Resend winner email — done when first mock winner declared. Phase 3 (Month 2): submit to Shopify App Store, 20 beta merchants installed — done when first Shopify Billing charge processes.

How do you build it?

Tech Stack

Shopify App CLI, Shopify Storefront API, Next.js, Supabase, Stripe, Shopify Polaris — build with Cursor for test engine, v0 for Polaris-compatible components

Suggested Frameworks

Shopify App Bridge, Shopify Polaris, Supabase Realtime

Time to Ship

3 weeks

Required Skills

Shopify App CLI, Shopify Storefront API session tokens, Bayesian A/B test statistics, Supabase Realtime.

Resources

Shopify App development docs, Shopify Polaris component library, Supabase Realtime docs, Bayesian A/B test npm libraries.

MVP Scope

app/page.tsx (embedded Shopify admin landing), app/tests/page.tsx (active tests list), app/tests/new/page.tsx (test creation form), app/api/shopify/webhook/route.ts (order event handler), app/api/tests/route.ts (CRUD for test management), lib/ab-engine.ts (Bayesian significance calculator), lib/shopify.ts (Admin and Storefront API client), lib/db/schema.ts (tests, variants, sessions, conversions), components/TestCard.tsx (active test display), seed.ts (2 demo tests with mock conversion data), .env.example (Shopify API key, Shopify secret, Supabase URL, Resend key)

Core User Journey

Install app -> connect Shopify store -> create price test in 3 clicks -> receive winner email after significance reached -> apply winner price in one click.

Architecture Pattern

Merchant creates test -> Supabase stores variant config -> Shopify storefront script assigns session to variant -> order webhook fires to Supabase -> Bayesian engine recalculates significance -> dashboard updates via Supabase Realtime -> winner email sent via Resend.

Data Model

Merchant has many Tests. Test has two Variants (Price A and Price B). Variant has many SessionAssignments. SessionAssignment has zero or one Conversion (order event).

Integration Points

Shopify Admin API for product price updates, Shopify Storefront API for session variant serving, Shopify Webhooks for order events, Supabase for test state and Realtime dashboard, Resend for winner notification emails, Shopify Billing API for subscription charges.

V1 Scope Boundaries

V1 excludes: multivariate testing, bundle pricing tests, team seats, custom significance thresholds, mobile app, non-Shopify platforms.

Success Definition

A merchant creates a price test, lets it run without any founder intervention, receives a winner notification email, applies the winning price in one click, and renews the subscription month two.

Challenges

Shopify app store review takes 2-5 days — submit early and test via direct install URL during review. The hardest non-technical problem is Shopify app store discoverability: organic search inside the app store is brutal for new apps without reviews, so LinkedIn and r/shopify outreach to get first 20 installs is mandatory before relying on store traffic.

Avoid These Pitfalls

Do not build multivariate testing in v1 — two-price A/B tests cover 90% of the use case and ship 4x faster. Do not underestimate Shopify app review turnaround — plan for a 5-day review window in your ship timeline. Finding first 10 installs before app store traffic exists requires direct community outreach — plan this from week one.

Security Requirements

Shopify session tokens for merchant auth. RLS on all test and conversion tables scoped to merchant Shopify shop domain. Shopify webhook HMAC signature verification on all incoming events. Rate limiting: 30 API calls per minute per merchant. GDPR: only anonymous session IDs stored, no PII.

Infrastructure Plan

Vercel for Next.js embedded app, Supabase for Postgres and Realtime, GitHub Actions for CI/CD, Sentry for errors — estimated $55/month at launch.

Performance Targets

Expected 30 DAU at launch. Dashboard Realtime update under 2s from order event. Page load under 2s. Bayesian calculation under 100ms client-side.

Go-Live Checklist

☐Security audit complete.
☐Shopify webhook HMAC verification tested.
☐Shopify Billing API subscription flow tested end-to-end.
☐Sentry error tracking live.
☐Supabase Realtime verified in production.
☐Custom domain with SSL active.
☐Privacy policy and terms published.
☐5 beta merchants ran full test cycle.
☐Shopify App Store listing submitted.

First Run Experience

On first run: 2 demo price tests with mock conversion data are pre-loaded showing a live dashboard with real-time variant stats and one test already at 94% significance. Merchant can click Apply Winner on the demo test to see the full flow. No manual config required: demo mode reads from seeded Supabase test data with no Shopify connection needed until merchant clicks Connect Store.

How to build it, step by step

1. Define Supabase schema for tests, variants, sessions, and conversions in lib/db/schema.ts before any UI. 2. Register Shopify Partner app with required Admin API and Webhook scopes, set embedded app redirect URI. 3. Run Shopify CLI create app command, integrate Polaris and App Bridge. 4. Build Shopify Admin API and Storefront API client in lib/shopify.ts. 5. Build test creation form at app/tests/new/page.tsx using Polaris components, saving test config to Supabase. 6. Build Bayesian significance calculator in lib/ab-engine.ts using beta distribution comparison. 7. Build Shopify order webhook handler at app/api/shopify/webhook/route.ts to record conversions per session variant. 8. Build Supabase Realtime dashboard at app/tests/page.tsx showing live conversion rates per variant. 9. Add winner auto-detection that fires Resend email when 95% confidence reached and exposes one-click price apply via Admin API. 10. Verify: create a test on a real Shopify dev store, place test orders on both variants, confirm Bayesian engine reaches significance, confirm winner email received, confirm price update applies via Admin API.

Generated

April 24, 2026

Model

claude-sonnet-4-6

← Back to All Ideas

ReplyMap - AI Agent That Maps Every Email Thread to a Next Action

Disclaimer: Ideas on this site are AI-generated and may contain inaccuracies. Revenue estimates, market demand figures, and financial projections are illustrative assumptions only — not financial advice. Do your own research before making any business or investment decisions. Technology availability, pricing, and market conditions change rapidly; always verify details independently.