User testing (sample testing)

Send a physical sample to a real person, have them agree to your requirements and answer a questionnaire on a hosted form (no login), then optionally sell them the sample with a Shopify draft order.

Where: setlist → User testing Related setting: Settings → Questionnaires

⚠️ "User testing" vs "Tests" — two different features. User testing (this page) = samples sent to real people for feedback. Tests = internal QA you run yourself. Similar names, different jobs.

What it is

User testing runs a small sample program. For each test you:

1. Pick a tester (a contact) and the sample under test (a product + a specific variant).
2. Optionally attach a questionnaire.
3. Generate a hosted form link and share it with the tester.
4. The tester opens the link (no login), agrees to your requirements, and fills out the questionnaire + optional notes/photos.
5. Once they submit, you can sell them the sample via a Shopify draft order.

The lifecycle

A test moves through five statuses:

Invited → Acknowledged → Submitted → Converted (plus Cancelled, off to the side)

Set it up: step by step

1. (Optional) Build a questionnaire

Create a reusable question set (up to 10 questions) in Settings → Questionnaires, or create one in a modal without leaving New test. You can also run "acknowledgment only" with no questionnaire.

2. Create the test

User testing → New test. Fill four sections:

• Tester — choose an existing contact or add a new contact (Name + Email required; Phone/Notes optional). A new contact is also synced to a Shopify customer (best-effort, once commerce access is granted) so a draft order later ties into your store CRM.
• Sample under test — choose the Product (required) and the Variant (the specific sample).
• Questionnaire — pick one, create one in place, or choose "No questions (acknowledgment only)". The questionnaire is snapshotted onto this test at creation, so editing it later never changes this tester's questions or answers.
• Requirements & timing — edit the requirements the tester must acknowledge (prefilled with sensible default copy), and set a Due date (required, no past dates).

Click Create test & generate link → you land on the test's detail page with the link ready to copy.

3. Share the link

On the detail page, copy the Tester form link (.../t/<token>) and send it to your tester. Anyone with the link can open the form — there's no tester login.

4. Watch results come in

As the tester acts, the status advances and their responses, notes, and photos appear on the detail page.

5. Sell the sample (optional)

Once the test is Submitted, use Sell the sample to create a Shopify draft order (unit price prefilled from the variant — edit it to comp or discount; set quantity). On success the test flips to Converted with links to the draft order and its invoice.

Selling the sample needs commerce access. The first time you create a draft order (or sync a tester to a Shopify customer), setlist asks Shopify for commerce access (read_customers, write_customers, write_draft_orders) via a Request commerce access card. Grant it once and the draft-order step appears. Everything else — invites, acknowledgment, questionnaires — works without it. See Shopify access & permissions.

The pages

List view

Four metrics — All tests, In progress (invited/acknowledged), Submitted (ready to sell), Converted — and a table: Tester · Product · Sample · Status · Completion (answered/total) · Due.

Detail view

Sections:

• Status — status badge, due date, tester info, the sample, and the Tester form link with Copy link.
• Acknowledgment — the requirements text and who signed/when (or "Not acknowledged yet").
• Responses — per-question answers once submitted (photo questions show a thumbnail gallery).
• Tester notes & photos — general feedback/photos the tester added.
• Sell the sample — the draft-order step (see above).
• Danger zone — Cancel this test (disables the link).

What the tester sees (the public form)

The link opens a clean, branded setlist page (not the Shopify admin, no login). It walks the tester through:

1. Step 1 — Acknowledge: read the requirements, check the box, and type their full name to sign.
2. Step 2 — Questionnaire (or a short confirm if acknowledgment-only): answer the questions. Question types include short text, long text, single-select, multi-select, and photo upload.
3. Notes & photos: an always-available spot for extra feedback and up to 6 photos (e.g. "it tarnished" / "arrived damaged").

When they submit, the test becomes Submitted and everything flows to your detail page. Photo limits: images only, under 10 MB each, up to 6 per field.

Tips & gotchas

• Snapshotting protects in-flight tests. Editing a questionnaire never changes a test that's already created — each test keeps its own copy of the questions.
• The link is the key. There's no tester account; anyone with the link can open the form. Cancel a test to disable its link.
• Submitted ≠ sold. "Sell the sample" is an explicit step that creates the draft order and moves the test to Converted.
• Unsynced variant? If the variant isn't in Shopify yet, the draft order uses a custom line item (title + price) instead of the real variant.

See also

• Settings → Questionnaires
• Tests — the other testing feature (internal QA)
• Products